An analysis method for intelligently identifying business links based on business traffic

Abstract

The present invention discloses an analysis method for intelligently identifying business links based on business traffic. Through business traffic and AI models, the real application call chain of the business can be intelligently identified, and the method has the following characteristics: 1) Traffic mirroring is implemented by a basic standard network protocol, with good application compatibility, and can well take into account both microservice architecture and traditional architecture; 2) The implementation operation is simple, and the traffic is obtained by bypassing the core switch in the form of traffic mirroring, without the need for agent intrusion, thus solving the problem of additional stability, security, and performance pressure caused by agent intrusion into application services; 3) Network traffic is all real and full data.

Description

Technical Field

[0001] The present invention relates to the field of network operation and maintenance technology, and in particular to an analysis method for intelligently identifying service links based on service traffic. Background Art

[0002] The business chain, also known as the business application call chain, is a topological diagram of the call relationship between application services of a specific business, which reflects the actual execution path of the business. In order to ensure the continuous stability of the business system, monitoring of the business chain is essential, and identifying the real application call chain of the business is the core of business chain monitoring. With the update and iteration of development technology and the emergence of microservices, IT systems have become more and more complex, and the identification of call chains has become more and more difficult.

[0003] The industry's solution for discovering call chains is through APM (Application Performance Management tools), which essentially connects application services through trace IDs to discover the relationship between application services. For example, Tencent's TSF microservice monitoring, Alibaba's application service monitoring ARMS, and Google's Dapper all use similar methods to achieve call chain discovery and monitoring.

[0004] APM first appeared in Google's public paper introducing Google's Dapper technology, and all existing APM solutions are based on this principle. Its advantages are:

[0005] 1. Google has many years of practical experience and mature solutions. At the same time, there are mature products or solutions on the market, such as oneAPM and Tingyun in China, and New Relic and AppDynamics abroad.

[0006] 2. High accuracy, reaching method level.

[0007] 3. Open source is mature, such as Skywalking in China.

[0008] But the disadvantages are also obvious:

[0009] 1. Difficult to implement. The trace ID of the application service requires the system architecture to be transformed into a microservice architecture. This transformation is unrealistic for most enterprise data centers. The risk of system architecture reconstruction to the original system cannot be assessed, and the workload is huge.

[0010] 2. Impact on the original application service. To obtain the trace ID, the application service must be deployed in an intrusive manner (agent intrusion), which is not allowed in industries with high security requirements. At the same time, this will inevitably affect the stability of the application service and cause additional consumption of resources. Although the additional consumption is not large, it is fatal for applications with high concurrency, massive requests, and performance loss sensitivity.

[0011] 3. The deployment is cumbersome and heavy. At the implementation level, agents need to be deployed for each application service. It is conceivable that the application service volume of a business system is huge. At the same time, the environment is complex, with containers, microservices, and ordinary services, spanning multiple systems and data centers. To identify the call chain, there are unpredictable difficulties in implementation.

[0012] Therefore, the prior art has defects and needs to be improved. Summary of the invention

[0013] The purpose of the present invention is to overcome the deficiencies of the prior art and provide an analysis method for intelligently identifying service links based on service traffic.

[0014] The technical solution of the present invention is as follows: Provide a method for analyzing service links based on service traffic intelligent identification, comprising the following steps:

[0015] Step 1: The traffic platform collects traffic to the business system through the core switch and TAP switch mirror traffic;

[0016] Step 2: Analyze the packet header information of the TCP message of the mirrored traffic and collect statistics on the core traffic indicators;

[0017] Step 3: Push the analyzed traffic data to the big data platform for storage and provide it to other applications;

[0018] Step 4: Filter business traffic through the CMDB platform;

[0019] Step 5: After the algorithm platform obtains the business flow data and CMDB data, it processes the data again on a daily basis;

[0020] Step 6: Filter the data after secondary processing using the valid business call chain path model;

[0021] Step 7: By adopting the business call chain path selection model, among the several businesses screened above and the several call chain paths of each business, the business call chain of each business is selected by the business call chain selection model to find out the real business call chain;

[0022] Step 8: After the model analysis is completed, the algorithm platform will output the results of the business call chain for verification by the operation and maintenance platform.

[0023] Furthermore, the specific steps of step 4 are as follows:

[0024] Step 4.1: The CMDB platform obtains the corresponding traffic information from the big data platform based on the business application IP and port;

[0025] Step 4.2: Filter out other non-business traffic from the acquired traffic information;

[0026] Step 4.3: The filtered traffic is business traffic, and the corresponding business start node and end node are marked in the business traffic node;

[0027] Step 4.4: The algorithm platform calls the filtered business traffic and CMDB information.

[0028] Furthermore, the specific steps of step 5 are as follows:

[0029] Step 5.1: After the algorithm platform obtains the business flow data and CMDB data, it merges nodes and paths for each node of the same cluster service and the paths flowing to the same node;

[0030] Step 5.2: Use minutes as time slices for the business traffic and draw the business call chain of the time slice to form call chain data for different time slices under the same business.

[0031] Furthermore, the specific steps of step 6 are as follows:

[0032] Step 6.1: Filter out multiple services with fluctuating traffic changes from the second-processed minute-time slice service traffic data;

[0033] Step 6.2: Use an algorithm to filter the call chain paths of the business call chain information of the above-screened business, and find out multiple valid call chain paths from the start node to the end node of the business.

[0034] Furthermore, the specific steps of step 7 are as follows:

[0035] Step 7.1: Uniquely identify each business call chain path topology;

[0036] Step 7.2: Detect the difference traffic of each marked service call chain, calculate the difference of service traffic in multiple time slice periods, form a service call chain difference traffic topology map, and use an algorithm to calculate several service lines of the difference traffic topology map;

[0037] Step 7.3: Summarize all business route diagrams in step 7.2, merge them, and keep adding the statistical times of the corresponding business call links. Finally, the one with the highest number of calls is taken as the final business route of the business. If there are multiple call chains with the highest number of call chains for a business, proceed to step 7.4, otherwise terminate;

[0038] Step 7.4: When there are multiple service call chains, the service route is monitored in real time in minutes. When it is found that there is traffic at the service endpoint but no traffic on a certain service line, it is eliminated until the last service line is left. During this process, all service lines that have not been eliminated are the service lines of this service.

[0039] Furthermore, the algorithm used in step 5 is the KPS algorithm, and the specific algorithm is as follows:

[0040] 1) Use Dijkstra algorithm to calculate the shortest link;

[0041] 2) Exclude the starting point and the end point, extract the intermediate node, remove the edge 4 between the intermediate node and the next node, and then use the Dijkstra algorithm to solve the shortest path from the intermediate node to the end point, and splice it with the line from the starting point to the intermediate node to form a candidate shortest path;

[0042] 3) Solving the j-th shortest path is based on the j-1-th shortest path.

[0043] Furthermore, the Dijkstra algorithm is as follows:

[0044] 1) Assume s is the starting point and e is the end point;

[0045] 2) Use a set F to store the nodes that have been visited. Initially, F only contains the starting point s. Use an array D to store the shortest path from the starting point s to all other nodes. At the beginning, the value of D is calculated using the following formula:

[0046]

[0047] 3) Find a node u that is not in F and whose D[u] is the smallest. D[u] is the shortest distance from the starting point s to the node u, and add u to F;

[0048] 4) Update the shortest distance in array D with node u:

[0049]

[0050] 5) If the end point e is already included in F, the shortest path has been found, otherwise continue to step 3).

[0051] By adopting the above scheme, the present invention intelligently identifies the real application call chain of the business through business traffic and AI models, and has the following characteristics: 1) Traffic mirroring is implemented by a basic standard network protocol, with good application compatibility, and can well take into account both microservice architecture and traditional architecture; 2) The implementation operation is simple, and the traffic is obtained by bypassing the core switch by traffic mirroring, without the need for agent intrusion, solving the problem of additional stability, security, and performance pressure caused by agent intrusion into application services; 3) Network traffic is all real and full data. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the effect of the present invention.

[0053] Figure 2 It is the data funnel diagram of the present invention.

[0054] Figure 3 It is a structural connection block diagram of the present invention. DETAILED DESCRIPTION

[0055] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0056] See also Figures 1 to 3 The present invention provides an analysis method for intelligently identifying a service link based on service traffic, comprising the following steps:

[0057] Step 1: The traffic platform collects traffic to the business system through the core switch and TAP switch mirror traffic.

[0058] Step 2: Analyze the packet header information of the TCP message of the mirrored traffic: source IP, destination IP, port triple information, and count the core traffic indicators such as the request volume, valid request number, response time, success rate, etc. of each IP.

[0059] Step 3: Push the analyzed traffic data to the big data platform for storage and provide it to other applications.

[0060] Step 4: Filter business traffic through the CMDB platform. The specific steps are as follows:

[0061] Step 4.1: The CMDB platform obtains the corresponding traffic information from the big data platform based on the business application IP and port.

[0062] Step 4.2: Filter out other non-business traffic from the acquired traffic information. For example, traffic from security applications and monitoring applications is non-business traffic and will affect the business call chain. It needs to be filtered to eliminate the impact.

[0063] Step 4.3: The filtered traffic is business traffic, and the corresponding business start node and end node are marked in the business traffic node.

[0064] Step 4.4: The algorithm platform calls the filtered business traffic and CMDB information and enters the model analysis phase.

[0065] Step 5: After the algorithm platform obtains the business flow data and CMDB data, it performs secondary processing on the data in units of days. The specific steps are as follows:

[0066] Step 5.1: After the algorithm platform obtains the business flow data and CMDB data, it merges the nodes and paths of each node serving the same cluster and the paths flowing to the same node.

[0067] Step 5.2: Use minutes as time slices for the business traffic and draw the business call chain of the time slice to form call chain data for different time slices under the same business.

[0068] Step 6: Filter the data after secondary processing through the effective business call chain path model. The specific steps are as follows:

[0069] Step 6.1: Filter out multiple services with fluctuating traffic changes from the second-processed minute time slice service traffic data. In this embodiment, it is assumed that s services are filtered out.

[0070] Step 6.2: Use an algorithm to filter the service call chain path of the service call chain information of the above-screened services, and find multiple valid call chain paths from the start node to the end node of the service. In this embodiment, it is assumed that each service has k call chain paths.

[0071] Step 7: By adopting the business call chain path selection model, among the s businesses selected above and the k call chain paths of each business, the business call chain selection model is used to select the business call chain of each business to find the real business call chain. The specific steps are as follows:

[0072] Step 7.1: Uniquely identify each business call chain path topology.

[0073] Step 7.2: Perform differential traffic detection on each marked service call chain. In different embodiments, take the actual service traffic time slices of 1, 5, 10, 30, and 60 cycles for multiple time slices, calculate the difference in service traffic, and form a service call chain differential traffic topology map. Use an algorithm to calculate the k service lines of the differential traffic topology map.

[0074] Step 7.3: Summarize all business route diagrams in step 7.2, merge them in the same way, and keep adding the statistical times of the corresponding business call links. Finally, take the one with the highest number of times as the final business route of the business. If there are multiple call chains with the highest number of call chains for a business, proceed to step 7.4, otherwise terminate.

[0075] Step 7.4: When there are multiple service call chains, the service route is monitored in real time in minutes. When it is found that there is traffic at the service endpoint but no traffic on a certain service line, it is eliminated until the last service line is left. During this process, all service lines that have not been eliminated are the service lines of this service.

[0076] Step 8: After the model analysis is completed, the algorithm platform will output the results of the business call chain for verification by the operation and maintenance platform.

[0077] Based on CMDB business attributes, CMDB information tables, CMDB resource associations, and application call chain information, IP+port is used as a node, and node merging is performed using CMDB information. For example, if all cluster nodes provide the same service, the cluster nodes can be merged into one node. Then, based on minutes, combined with call chain information, a new business topology map valid every minute is formed.

[0078] Taking the day as the unit, based on the effective business topology map formed every minute, select specific services, and use link traffic survival detection to select effective business routes. Traverse all effective business topology maps every minute, and select business topology maps with business traffic fluctuations. Assume that s business topology maps are selected. Use the KPS algorithm to solve the k shortest paths of the business in the calling information network for s business topology maps. K can be adjusted according to the actual situation, and k business lines can be selected from each business topology map. Summarize the above business lines and obtain a minimum of k and a maximum of k*s business lines as candidate business routes for the business. Assume that the number is d.

[0079] Use traffic correlation detection for final confirmation, mark all valid business topology maps, and subtract the business topology maps of multiple time slice intervals from each marked business topology map to form a link difference traffic topology map, and use the KPS algorithm to calculate the k business lines of the difference traffic topology map. Summarize all business line maps, merge the same ones, and continue to accumulate the number of times. The one with the highest number of times is the final business route of the business. If there are several business routes at the same time, if there are b routes, proceed to the next step, otherwise terminate. Monitor the b business routes in real time in minutes. When it is found that there is traffic at the end of the business, but there is no traffic on a certain business line, it will be eliminated until the last business line is left. All business lines that have not been eliminated in this process are business lines of the business.

[0080] The algorithm used in step 5 is the KPS algorithm, and the specific algorithm is as follows:

[0081] 1) Use Dijkstra algorithm to calculate the shortest link;

[0082] 2) Exclude the starting point and the end point, extract the intermediate node, remove the edge 4 between the intermediate node and the next node, and then use the Dijkstra algorithm to solve the shortest path from the intermediate node to the end point, and splice it with the line from the starting point to the intermediate node to form a candidate shortest path;

[0083] 3) Solving the j-th shortest path is based on the j-1-th shortest path.

[0084] The Dijkstra algorithm is as follows:

[0085] 1) Assume s is the starting point and e is the end point;

[0086] 2) Use a set F to store the nodes that have been visited. Initially, F only contains the starting point s. Use an array D to store the shortest path from the starting point s to all other nodes. At the beginning, the value of D is calculated using the following formula:

[0087]

[0088] 3) Find a node u that is not in F and whose D[u] is the smallest. D[u] is the shortest distance from the starting point s to the node u, and add u to F;

[0089] 4) Update the shortest distance in array D with node u:

[0090]

[0091] 5) If the end point e is already included in F, the shortest path has been found, otherwise continue to step 3).

[0092] In summary, the present invention intelligently identifies the real application call chain of the business through business traffic and AI models, and has the following characteristics: 1) Traffic mirroring is implemented by a basic standard network protocol, with good application compatibility, and can well take into account both microservice architecture and traditional architecture; 2) The implementation operation is simple, and the traffic is obtained by bypassing the core switch by traffic mirroring, without the need for agent intrusion, solving the problem of additional stability, security, and performance pressure caused by agent intrusion into application services; 3) Network traffic is all real and full data.

[0093] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An analysis method for intelligently identifying service links based on service traffic, characterized in that: The steps include: Step 1: The traffic platform collects traffic to the business system through the core switch and TAP switch mirror traffic; Step 2: Analyze the packet header information of the mirrored traffic TCP message and collect statistics on core traffic indicators; Step 3: Push the analyzed traffic data to the big data platform for storage and provide it to other applications; Step 4: Filter business traffic through the CMDB platform; the specific steps are as follows: Step 4.1: The CMDB platform obtains the corresponding traffic information from the big data platform based on the business application IP and port; Step 4.2: Filter out other non-business traffic from the acquired traffic information; Step 4.3: The filtered traffic is business traffic, and the corresponding business start node and end node are marked in the business traffic node; Step 4.4: The algorithm platform calls the filtered business traffic and CMDB information; Step 5: After the algorithm platform obtains the business flow data and CMDB data, it performs secondary processing on the data in units of days. The specific steps are as follows: Step 5.1: After the algorithm platform obtains the business flow data and CMDB data, it merges the nodes of the same cluster service and merges the paths flowing to the same node; Step 5.2: Use minutes as time slices for business traffic and draw the business call chain of the time slice to form call chain data of different time slices under the same business; Step 6: Filter the data after secondary processing using the valid business call chain path model; Step 7: By adopting the business call chain path selection model, among the several businesses screened above and the several call chain paths of each business, the business call chain of each business is selected by the business call chain path selection model to find out the real business call chain; Step 8: After the model analysis is completed, the algorithm platform will output the results of the business call chain for verification by the operation and maintenance platform.

2. The analysis method for intelligently identifying service links based on service traffic according to claim 1 is characterized in that: The specific steps of step 6 are as follows: Step 6.1: Filter out multiple services with fluctuating traffic changes from the second-processed minute-time slice service traffic data; Step 6.2: Use an algorithm to filter the call chain paths of the business call chain information of the above-screened business, and find out multiple valid call chain paths from the start node to the end node of the business.

3. The analysis method for intelligently identifying service links based on service traffic according to claim 1 is characterized in that: The specific steps of step 7 are as follows: Step 7.1: Uniquely identify each business call chain path topology; Step 7.2: Call the chain difference traffic detection program for each marked service, calculate the difference of service traffic in multiple time slice periods, form a service call chain difference traffic topology map, and use the algorithm to calculate several service lines of the difference traffic topology map; Step 7.3: Summarize all business route diagrams in step 7.2, merge them, and keep adding the statistical times of the corresponding business call links. Finally, the one with the highest number of calls is taken as the final business route of the business. If there are multiple call chains with the highest number of call chains for a business, proceed to step 7.4, otherwise terminate; Step 7.4: When a business has multiple call chains, the business route is monitored in real time on a minute-by-minute basis. When it is found that there is traffic at the end of the business but no traffic on a certain business line, it is eliminated until the last business line is left. During this process, all business lines that have not been eliminated are business lines of this business.

Images