Dynamic baseline monitoring method and system based on network protocol flow
By analyzing and quantifying network traffic data, the monitoring scheme with the highest score is selected, which solves the problem that static baseline monitoring cannot adapt to complex network environments. This achieves high efficiency, accuracy, and flexibility in dynamic baseline monitoring, ensuring stable network operation.
Patent Information
- Application Number
- CN202511026689.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-24
- Publication Date
- 2025-10-28
AI Technical Summary
Existing static baseline monitoring methods cannot meet the needs of complex and ever-changing network environments, and cannot achieve anomaly tracking and risk identification of network traffic changes and protocol interactions.
By acquiring network traffic data, it is decoded and analyzed into four elements: monitoring object, monitoring algorithm, monitoring probe, and monitoring index. Based on a preset evaluation matrix, quantitative evaluation is performed, and the sub-element with the highest score is selected as the monitoring scheme. Combined with dynamic baseline algorithm and dedicated traffic analysis equipment/software, dynamic monitoring is achieved.
It enables dynamic monitoring of network traffic, improves monitoring efficiency and accuracy, adapts to dynamic changes in network traffic, and provides reliable network stability assurance.
Smart Images
Figure CN120856604A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of network monitoring technology, and in particular to a dynamic baseline monitoring method and system based on network protocol traffic. Background Art
[0002] With the rapid development of digital services and the iteration of various technology protocol stacks, network environments are becoming increasingly complex and dynamic. This places higher demands on the accuracy and timeliness of network monitoring. Currently, static baseline monitoring is mainly used, which focuses on monitoring device performance indicators such as CPU utilization, memory usage, and process status by setting fixed thresholds. Mainstream operation and maintenance platforms, on the other hand, build their monitoring systems around device operating indicators, focusing on the hardware status and basic performance parameters of the devices.
[0003] However, traditional static baseline monitoring can no longer meet the monitoring needs of actual networks. This is because network equipment comes from a variety of brands and generates a lot of logs, and because network traffic is composed of complex components and its fluctuation characteristics vary greatly with business characteristics. Static baselines can only identify faults, anomalies, and risks based on equipment performance, but they cannot track and identify anomalies and risks in actual business traffic changes and protocol interactions. Summary of the Invention
[0004] To address the above problems, this application provides a dynamic baseline monitoring method based on network protocol traffic, including the following:
[0005] Firstly, this application provides a dynamic baseline monitoring method based on network protocol traffic, the method comprising:
[0006] Network traffic data is acquired, and the network traffic data is decoded, parsed and decomposed to obtain four elements of network monitoring. The four elements of network monitoring include monitoring objects, monitoring algorithms, monitoring probes and monitoring indicators. Each of the four elements of network monitoring includes one or more sub-elements.
[0007] The sub-elements are quantitatively evaluated based on a preset evaluation matrix. The sub-elements with the highest scores among the monitored objects, monitoring algorithms, monitoring probes, and monitoring indicators are selected as the final elements of the four elements of network monitoring in the monitoring scheme. The preset evaluation matrix is constructed based on the four elements and five key points of network monitoring. The five key points include predictive ability, interpretability, stability, localization ability, and operability.
[0008] The network traffic is monitored based on the aforementioned monitoring scheme.
[0009] Optionally, the quantitative evaluation of the sub-elements based on a preset evaluation matrix includes:
[0010] The ability to assess and locate elements of the monitored object;
[0011] Evaluate the interpretability, stability, and operability of the monitoring algorithm elements;
[0012] The stability and positioning capability of the monitoring probe elements were evaluated.
[0013] Assess the predictive ability, interpretability, stability, and location capability of monitoring indicators.
[0014] Optionally, the sub-elements in the monitored object include:
[0015] The system is divided into application system layer and network infrastructure layer. The application system layer includes the call information of each subsystem, microservice and application module, mainly based on the content of the data packet payload, including application requests, return data and business parameters. The network infrastructure layer includes each IP address and protocol layer, mainly based on the protocol encapsulation content of each data packet header, including TCP / UDP protocol, DNS protocol, HTTP protocol and 5-tuple.
[0016] Optionally, the sub-elements in the monitoring algorithm include a static baseline algorithm and a dynamic baseline algorithm;
[0017] The static baseline algorithm is based on the expected operation of business traffic and is designed according to business needs for simple monitoring scenarios.
[0018] The dynamic baseline algorithm takes the actual operation of business traffic as a reference, including using the mean-standard deviation model or Bayesian model to assess the actual business operation risk, and is used in complex monitoring scenarios.
[0019] Optionally, the monitoring probes are deployed as needed based on the amount of resources and the network architecture. The network architecture deployment should follow the basic principle of covering the north-south traffic, east-west traffic, important boundary link traffic of external exits, and application system server-side traffic of each network partition.
[0020] The deployment scheme includes dedicated traffic analysis equipment and dedicated traffic analysis software. The dedicated traffic analysis equipment is used for comprehensive analysis scenarios involving large volumes of traffic and big data, while the dedicated traffic analysis software is applied to server microservice scenarios.
[0021] Optionally, the sub-element of the monitoring metrics is the TCP protocol metric, which is calculated and determined by the connection no-response rate and the TCP packet loss rate.
[0022] Optionally, the calculation using connection no-response rate and TCP packet loss rate includes:
[0023] Connection failure rate = Number of three-way handshake failures / Total number of connections;
[0024] TCP packet loss rate = Number of packets lost with payload / Total number of TCP packets.
[0025] Secondly, this application provides a dynamic baseline monitoring system based on network protocol traffic, the system comprising:
[0026] The acquisition unit is used to acquire network traffic data, decode and decompose the network traffic data to obtain four elements of network monitoring. The four elements of network monitoring include monitoring object, monitoring algorithm, monitoring probe and monitoring index. Each of the four elements of network monitoring includes one or more sub-elements.
[0027] The processing unit is used to quantitatively evaluate the sub-elements based on a preset evaluation matrix, and select the sub-elements with the highest scores among the monitored objects, monitoring algorithms, monitoring probes and monitoring indicators as the final elements of the four elements of network monitoring in the monitoring scheme; the preset evaluation matrix is constructed based on the four elements and five key points of network monitoring, and the five key points include predictive ability, interpretability, stability, localization ability and operability.
[0028] The monitoring unit is used to monitor the network traffic based on the monitoring scheme.
[0029] Optionally, the processing unit performs quantitative evaluation of the sub-elements on a preset evaluation matrix, including:
[0030] The ability to assess and locate elements of the monitored object;
[0031] Evaluate the interpretability, stability, and operability of the monitoring algorithm elements;
[0032] The stability and positioning capability of the monitoring probe elements were evaluated.
[0033] Assess the predictive ability, interpretability, stability, and location capability of monitoring indicators.
[0034] Optionally, the sub-elements in the monitored object include:
[0035] The system is divided into application system layer and network infrastructure layer. The application system layer includes the call information of each subsystem, microservice and application module, mainly based on the content of the data packet payload, including application requests, return data and business parameters. The network infrastructure layer includes each IP address and protocol layer, mainly based on the protocol encapsulation content of each data packet header, including TCP / UDP protocol, DNS protocol, HTTP protocol and 5-tuple.
[0036] Optionally, the sub-elements in the monitoring algorithm include static baseline algorithms and dynamic baseline algorithms;
[0037] The static baseline algorithm is based on the expected operation of business traffic and is designed according to business needs for simple monitoring scenarios.
[0038] The dynamic baseline algorithm takes the actual operation of business traffic as a reference, including using the mean-standard deviation model or Bayesian model to assess the actual business operation risk, and is used in complex monitoring scenarios.
[0039] Optionally, monitoring probes can be deployed as needed based on the amount of resources and network architecture. The network architecture deployment should follow the basic principle of covering the north-south traffic, east-west traffic, important boundary link traffic of external exits, and application system server-side traffic in each network partition.
[0040] The deployment scheme includes dedicated traffic analysis equipment and dedicated traffic analysis software. The dedicated traffic analysis equipment is used for comprehensive analysis scenarios involving large volumes of traffic and big data, while the dedicated traffic analysis software is applied to server microservice scenarios.
[0041] Optionally, the sub-element of the monitoring metrics is the TCP protocol metric, which is calculated and determined by the connection no-response rate and the TCP packet loss rate.
[0042] Optionally, the processing unit calculates TCP protocol metrics based on connection no-response rate and TCP packet loss rate, including:
[0043] Connection failure rate = Number of three-way handshake failures / Total number of connections;
[0044] TCP packet loss rate = Number of packets lost with payload / Total number of TCP packets.
[0045] Thirdly, this application provides an apparatus comprising a memory and a processor, the memory for storing instructions or code, and the processor for executing the instructions or code to cause the apparatus to perform the dynamic baseline monitoring method based on network protocol traffic described in any of the implementations of the first aspect.
[0046] Fourthly, this application provides a computer-readable storage medium storing code, wherein when the code is executed, a device running the code implements the dynamic baseline monitoring method based on network protocol traffic described in any of the implementations of the first aspect.
[0047] This application provides a dynamic baseline monitoring method based on network protocol traffic. When executing the method, network traffic data is first acquired, and then decoded, parsed, and decomposed to obtain four elements of network monitoring. These four elements include a monitoring object, a monitoring algorithm, a monitoring probe, and a monitoring indicator. Each of the four elements includes one or more sub-elements. Then, based on a preset evaluation matrix, each sub-element is quantitatively evaluated. The sub-element with the highest score among the monitoring object, monitoring algorithm, monitoring probe, and monitoring indicator is selected as the final element of the four elements of network monitoring in the monitoring scheme. The preset evaluation matrix is constructed based on the four elements of network monitoring and five key points, including predictive ability, interpretability, stability, location ability, and operability. Finally, the network traffic is monitored based on the monitoring scheme. This method, on the one hand, cleverly breaks down dynamic network traffic monitoring into four elements: monitoring objects, monitoring algorithms, monitoring probes, and monitoring indicators. This enables structured management of monitoring work, making complex monitoring processes clear and orderly, and greatly improving monitoring efficiency and management convenience. On the other hand, it innovatively introduces a quantitative five-dimensional evaluation mechanism, which quantitatively evaluates sub-elements from five key points: predictive ability, interpretability, stability, location ability, and operability. This transforms subjective experience into objective quantitative scores, forming an optimization loop for the monitoring solution and ensuring that the monitoring solution can be continuously optimized according to dynamic changes in network traffic. Furthermore, this method provides a customizable dynamic baseline generation strategy, which can flexibly adapt to various types of business flow scenarios, comprehensively improving the accuracy, effectiveness, and adaptability of network traffic monitoring, and providing a reliable guarantee for stable network operation. Attached Figure Description
[0048] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0049] Figure 1 A flowchart illustrating a dynamic baseline monitoring method based on network protocol traffic, provided for embodiments of this application;
[0050] Figure 2 This application provides a schematic diagram of a network architecture for a monitoring probe.
[0051] Figure 3 This is a schematic diagram of the structure of a dynamic baseline monitoring system based on network protocol traffic, provided in an embodiment of this application. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0053] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0054] First, let's explain some of the terms used in this application:
[0055] Protocol: In network communication, the data transmission rules and conventions followed by the parties involved in the communication, such as TCP, UDP, HTTP, etc.
[0056] Baseline: Refers to the historical average or distribution range of various key performance indicators (such as traffic, latency, and number of connections) under normal network or system operation, serving as a reference standard for assessing whether future indicators are abnormal. It can be divided into static baseline (preset threshold) and dynamic baseline (statistical model that is dynamically adjusted according to business needs).
[0057] Network traffic capture devices: These are dedicated devices or modules deployed at critical network links or nodes to copy, mirror, or record raw data packets. Their main function is to collect real business traffic data losslessly for subsequent protocol parsing, metric extraction, and behavioral analysis. Common forms include bypass probes, traffic mirroring switches, and software probes.
[0058] Business: refers to the various functional modules of an enterprise that operate through the network, such as online banking, fund transfer, inquiry and other services.
[0059] System: A network infrastructure or business platform consisting of hardware and software that performs specific functions.
[0060] Network: A data transmission system composed of network devices such as switches, routers, and firewalls.
[0061] Mirrored traffic: refers to network traffic that copies the original network data packets through network devices (such as switches, routers, or TAPs) and sends them to a dedicated analysis port or device for detection and analysis.
[0062] Figure 1 A flowchart illustrating a dynamic baseline monitoring method based on network protocol traffic, provided as an embodiment of this application. (In conjunction with...) Figure 1 As shown, the dynamic baseline monitoring method based on network protocol traffic provided in this application embodiment may include:
[0063] S101. Obtain network traffic data, and decode, parse and decompose the network traffic data to obtain the four elements of network monitoring.
[0064] Network traffic data refers to the sum of data packets and related metadata transmitted between different network nodes such as computers, servers, and routers through various network protocols in a network environment. This data flows continuously within the network, forming the foundation of network communication. In this application, network traffic data is collected and aggregated by a network traffic capture device to obtain mirrored traffic. The mirrored traffic is then decoded, parsed, and decomposed into the four elements of network monitoring: the monitoring object, the monitoring algorithm, the monitoring probe, and the monitoring indicators.
[0065] Each of the four elements of network monitoring includes one or more sub-elements, and the sub-elements of the monitored object include:
[0066] The system layer is divided into application system layer classes and network infrastructure layer classes. The application system layer classes focus on the operational status of the application system, including the calls made by various subsystems, microservices, and application modules. For example, in an e-commerce system, the user order module calls the payment module, which in turn calls the bank's interface module. The application system layer classes primarily focus on the data packet payload content, which emphasizes the actual content transmitted within the data packet, such as application requests to user-initiated operations, results returned by the system, and business parameters like product ID and order amount.
[0067] The network infrastructure layer focuses on the operation of the network infrastructure. This includes IP addresses, the addresses of devices within the network; the protocol layer, which contains the protocol encapsulation of data packets, such as: TCP / UDP protocols (transport layer protocols that determine how data is transmitted across the network); DNS protocols (domain name resolution protocol that translates domain names into IP addresses); HTTP protocols (application layer protocols used for web browsing); and the 5-tuple, which includes source IP, destination IP, source port, destination port, and transport layer protocol type, used to uniquely identify a network connection.
[0068] The sub-elements in the monitoring algorithm include static baseline algorithm and dynamic baseline algorithm;
[0069] Static baselines are based on the expected operation of business traffic and are designed according to business needs. They are typically used for simple monitoring scenarios, such as business response rates of over 90% and API call success rates of 100%.
[0070] The dynamic baseline algorithm takes the actual operation of business traffic as a reference, including using the mean-standard deviation model or Bayesian model to assess the actual business operation risk, and is used in complex monitoring scenarios.
[0071] The use of the mean-standard deviation model to assess actual business operation risks includes using the historical traffic sample mean to predict the actual operation of future business traffic, and using the historical traffic mean-standard deviation to assess actual operation risks. If the actual operation indicators are greater than 3 standard deviations, it can be assessed that the business operation deviates too much from expectations and there is an operation risk.
[0072] Using Bayesian models to assess actual operational risks involves modeling the probability distribution of historical traffic data and combining it with current observation data to dynamically calculate the posterior probability of traffic anomalies. When the posterior probability exceeds a set risk threshold, the system can automatically identify a potential anomaly. For example, in the event of a micro-burst in network traffic, the system can use the normal distribution pattern of historical traffic to assess the probability of anomalies in the currently observed traffic in real time. If the probability exceeds a set risk threshold (e.g., a 95% confidence level), the system can determine that the micro-burst is not a normal fluctuation.
[0073] The deployment method for monitoring probes depends on the amount of resources and the specific requirements of the network architecture. During deployment, it's essential to ensure the network architecture can comprehensively cover the following traffic types: north-south traffic between network partitions (traffic from external to internal or vice versa); east-west traffic within a network partition (traffic between different areas); critical link traffic at the network egress point; and traffic on the application system server side (internal server traffic). There are two deployment options for monitoring probes. One is to use dedicated traffic analysis equipment, suitable for comprehensive analysis of high traffic and big data, capable of handling high-load network environments. The other is to use dedicated traffic analysis software, suitable for server microservice scenarios, primarily used to analyze internal server traffic, such as calls between microservices.
[0074] In this embodiment, to make the deployment method of the monitoring probe clearer to those skilled in the art, a network architecture for the monitoring probe is provided, such as... Figure 2 As shown, Figure 2 This application provides a schematic diagram of a network architecture for a monitoring probe, which includes three switches and a firewall. North-south traffic refers to traffic from one area of the network to another, while east-west traffic refers to traffic between different devices within the same area. Figure 2In this diagram, North-South Traffic 1 and North-South Traffic 2 represent traffic passing through different switches, which may traverse different network areas. East-West Traffic represents traffic between switches within the same network area. Furthermore, the firewall connects to one of these switches to monitor and control traffic on critical boundary links at the outbound exit point, which is a key point for the network to interact with the outside world.
[0075] Based on the network architecture and resource availability, dedicated traffic analysis equipment and software were deployed. The dedicated traffic analysis equipment was deployed at critical network nodes, such as near firewalls, to handle comprehensive analysis scenarios involving large volumes of traffic and data. This equipment can handle high-load network environments and is suitable for monitoring north-south and east-west traffic. The dedicated traffic analysis software was deployed on application system servers to monitor traffic in microservice scenarios, particularly server-side traffic. This deployment approach enables monitoring of critical network traffic, including north-south, east-west, critical boundary link traffic, and server-side traffic, thereby providing data support for network performance monitoring and optimization.
[0076] The sub-element of the monitoring metrics is the TCP protocol metric, which is calculated and determined using the connection non-response rate and the TCP packet loss rate. The specific calculation method is as follows:
[0077] Connection failure rate: refers to the ratio of the number of times the TCP three-way handshake fails due to various reasons to the total number of connection attempts. The formula is: Connection failure rate = Number of three-way handshake failures / Total number of connections.
[0078] TCP packet loss rate: refers to the ratio of the number of data packets with payloads lost to the total number of TCP data packets during TCP data transmission. The calculation formula is: TCP packet loss rate = number of lost payload packets / total number of TCP data packets.
[0079] S102. Based on the preset evaluation matrix, the sub-elements are quantitatively evaluated respectively, and the sub-elements with the highest scores among the monitored objects, monitoring algorithms, monitoring probes and monitoring indicators are selected as the final elements of the four elements of network monitoring in the monitoring scheme.
[0080] The evaluation matrix is used to quantitatively assess five key aspects of the four monitoring elements: predictive ability, interpretability, stability, location capability, and operability. Predictive ability evaluates whether the monitoring element can predict potential problems through changes in indicators before business issues occur. This is crucial for early problem detection and prevention. Interpretability evaluates whether the output of the monitoring element can be explained through technical logic. High interpretability means the monitoring results are easier to understand and trust. Stability evaluates whether the monitoring element is stable and reliable under normal operating conditions. Stability is a key factor in ensuring the long-term effective operation of the monitoring system. Location capability evaluates whether the monitoring element can accurately locate the specific location of the problem, which is crucial for rapid diagnosis and resolution. Operability evaluates whether the monitoring element is easy to adjust and optimize parameters. High operability means the monitoring system can be flexibly adjusted to adapt to different monitoring needs.
[0081] This five-point evaluation method allows for the quantitative scoring of sub-elements of each monitoring element. Finally, the highest-scoring sub-elements from the monitored object, monitoring algorithm, monitoring probe, and monitoring indicators are selected as the final four elements in the network monitoring solution. This method ensures that the selected elements are optimal in terms of predictive ability, interpretability, stability, location capability, and operability, thereby improving the overall effectiveness and reliability of the network monitoring solution. The constructed evaluation matrix is as follows:
[0082] Main points / elements Monitoring objects Monitoring Algorithm monitoring probes Monitoring indicators Predictive ability √ Explainability √ √ stability √ √ √ Positioning capability √ √ √ Operability √ √
[0083] Based on the aforementioned evaluation matrix, the quantitative evaluation of the sub-elements based on the preset evaluation matrix includes: evaluating the location capability of the monitored object element; evaluating the interpretability, stability, and operability of the monitoring algorithm element; evaluating the stability and location capability of the monitoring probe element; and evaluating the predictive capability, interpretability, stability, and location capability of the monitoring indicator element. By combining the four elements of network monitoring and the five key points of quantitative evaluation, commonly used network traffic monitoring schemes can be obtained, and continuous iteration can be performed to enhance the overall effectiveness of the monitoring scheme.
[0084] The following is an exemplary description of the process of quantitatively evaluating the sub-elements based on the above evaluation matrix, and then selecting the sub-element with the highest score among the monitored object, monitoring algorithm, monitoring probe, and monitoring indicator as the final element of the four elements of network monitoring in the monitoring scheme. The specific quantitative evaluation process is as follows. In the quantitative process, the quantitative range is 0 to 2, where 0 points indicates weak correlation, 1 point indicates some correlation, and 2 points indicates strong correlation.
[0085] For the monitored object, the main evaluation is its location capability. All sub-elements of the monitored object are evaluated separately to obtain a quantitative score. Ultimately, it was found that using the network 5-tuple as the monitored object can pinpoint the specific network IP, service port, and network protocol, demonstrating strong location capability in network monitoring. Therefore, the 5-tuple is selected as the monitored object in the monitoring scheme, as shown in the table below:
[0086] Main points / elements Monitoring objects Quantitative score Positioning capability √ 2
[0087] The monitoring algorithm was evaluated based on three aspects: interpretability, stability, and operability. The evaluation results showed that the mean-standard deviation model scored the highest, indicating that it effectively explained the actual operational and risk situations, demonstrated good stability of the mean, and allowed for rapid algorithm adjustments based on the sample size and the standard deviation. Therefore, the mean-standard deviation model was selected as the control algorithm in the final monitoring solution. The specific evaluation results are shown in the table below:
[0088] Main points / elements Monitoring Algorithm Quantitative score Explainability √ 2 stability √ 2 Operability √ 2
[0089] For monitoring probes, the main evaluation focuses on their stability and location capabilities. The basic principle of network architecture deployment is to monitor north-south and east-west traffic in each partition, as well as the traffic on critical boundary links at the external egress point. These deployment locations represent traffic from key network nodes, exhibiting strong traffic stability and clearly defined location boundaries. Therefore, the north-south, east-west, and critical boundary link traffic of each network partition are selected as the monitoring probes in the final monitoring solution. Specific evaluation results are shown in the table below:
[0090] Main points / elements monitoring probes Quantitative score stability √ 2 Positioning capability √ 2
[0091] To evaluate the predictive ability, interpretability, stability, and localization capability of monitoring indicators, the connection non-response rate during the TCP connection establishment process and the TCP packet loss rate due to payload loss after connection establishment are calculated. Based on the quantitative evaluation results of each indicator, indicators with a score of 8 or higher are retained. The specific evaluation results are shown in the table below:
[0092]
[0093] Ultimately, the connection non-response rate and TCP packet loss rate were retained as monitoring metrics in the monitoring scheme because their scores met the requirements.
[0094] Through the quantitative evaluation process based on the evaluation matrix described above, four key elements were selected in the network monitoring scheme: the monitoring object, the monitoring algorithm, the monitoring probe, and the monitoring indicators. Based on these four key elements, a dynamic baseline monitoring scheme can be established for monitoring network traffic. The dynamic baseline monitoring scheme based on network protocol traffic is as follows:
[0095]
[0096] S103. Monitor the network traffic based on the monitoring scheme.
[0097] The above describes a dynamic baseline monitoring method based on network protocol traffic provided in this application. This method, based on network traffic mirroring technology, decomposes dynamic network traffic monitoring into a four-element model: monitoring object, monitoring algorithm, monitoring probe, and monitoring indicators. It then combines this with a quantitative evaluation based on five key aspects: predictive capability, interpretability, stability, location capability, and operability, forming a dynamic baseline monitoring scheme based on network protocol traffic and quantifying the overall monitoring effectiveness. This method not only structures the complex network traffic monitoring task, making the monitoring process clearer, more organized, and easier to manage and maintain, but also transforms the previous subjective experience-based monitoring evaluation into quantifiable scores through a quantitative five-dimensional evaluation mechanism, forming an optimization loop that continuously improves the monitoring scheme and enhances monitoring effectiveness. Furthermore, the monitoring mechanism based on traffic mirroring, protocol decoding, and dynamic modeling can comprehensively and deeply analyze network traffic, ensuring the accuracy and reliability of monitoring results while adapting to dynamic changes in network traffic. More importantly, it provides a customizable dynamic baseline generation strategy, allowing for flexible adjustment of the monitoring scheme according to the characteristics and needs of different service flows, ensuring the monitoring system operates efficiently in various complex network environments and improving the versatility and adaptability of the monitoring system. In summary, this method significantly improves the efficiency, accuracy, and flexibility of network monitoring, providing strong support for stable network operation and optimized management. It also provides network operations and maintenance personnel with more scientific and reliable monitoring tools, reduces operation and maintenance costs, and improves network service quality.
[0098] The above are some specific implementations of a dynamic baseline monitoring method based on network protocol traffic provided in the embodiments of this application. Based on this, this application also provides a corresponding system. The system provided in the embodiments of this application will be described below from the perspective of functional modularity.
[0099] Figure 3 This is a schematic diagram of the structure of a dynamic baseline monitoring system based on network protocol traffic, provided as an embodiment of this application. (Combined with...) Figure 3 As shown in the embodiment of this application, the dynamic baseline monitoring system 300 based on network protocol traffic includes:
[0100] The acquisition unit 310 is used to acquire network traffic data, decode and decompose the network traffic data to obtain four elements of network monitoring. The four elements of network monitoring include monitoring object, monitoring algorithm, monitoring probe and monitoring index. Each element of the four elements of network monitoring includes one or more sub-elements.
[0101] The processing unit 320 is used to quantitatively evaluate the sub-elements based on a preset evaluation matrix, and select the sub-elements with the highest scores among the monitored objects, monitoring algorithms, monitoring probes and monitoring indicators as the final elements of the four elements of network monitoring in the monitoring scheme; the preset evaluation matrix is constructed based on the four elements and five key points of network monitoring, and the five key points include predictive ability, interpretability, stability, localization ability and operability;
[0102] The monitoring unit 330 is used to monitor the network traffic based on the monitoring scheme.
[0103] In one implementation of this application embodiment, the processing unit performs quantitative evaluation of the sub-elements on a preset evaluation matrix, including:
[0104] The ability to assess and locate elements of the monitored object;
[0105] Evaluate the interpretability, stability, and operability of the monitoring algorithm elements;
[0106] The stability and positioning capability of the monitoring probe elements were evaluated.
[0107] Assess the predictive ability, interpretability, stability, and location capability of monitoring indicators.
[0108] In one implementation of this application, the sub-elements in the monitored object include:
[0109] The system is divided into application system layer and network infrastructure layer. The application system layer includes the call information of each subsystem, microservice and application module, mainly based on the content of the data packet payload, including application requests, return data and business parameters. The network infrastructure layer includes each IP address and protocol layer, mainly based on the protocol encapsulation content of each data packet header, including TCP / UDP protocol, DNS protocol, HTTP protocol and 5-tuple.
[0110] In one implementation of this application, the sub-elements in the monitoring algorithm include a static baseline algorithm and a dynamic baseline algorithm;
[0111] The static baseline algorithm is based on the expected operation of business traffic and is designed according to business needs for simple monitoring scenarios.
[0112] The dynamic baseline algorithm takes the actual operation of business traffic as a reference, including using the mean-standard deviation model or Bayesian model to assess the actual business operation risk, and is used in complex monitoring scenarios.
[0113] In one implementation of this application, the monitoring probes are deployed as needed based on the amount of resources and the network architecture. The network architecture deployment must follow the basic principle of covering the north-south traffic, east-west traffic, important boundary link traffic of external exits, and application system server-side traffic of each network partition.
[0114] The deployment scheme includes dedicated traffic analysis equipment and dedicated traffic analysis software. The dedicated traffic analysis equipment is used for comprehensive analysis scenarios involving large volumes of traffic and big data, while the dedicated traffic analysis software is applied to server microservice scenarios.
[0115] In one implementation of this application, the sub-element of the monitoring indicators is the TCP protocol indicator, which is calculated and determined by the connection no-response rate and the TCP packet loss rate.
[0116] In one implementation of this application, the processing unit calculates TCP protocol metrics using connection no-response rate and TCP packet loss rate, including:
[0117] Connection failure rate = Number of three-way handshake failures / Total number of connections;
[0118] TCP packet loss rate = Number of packets lost with payload / Total number of TCP packets.
[0119] This application also provides corresponding devices and computer storage media for implementing the solutions provided in this application.
[0120] The device includes a memory and a processor. The memory stores instructions or code, and the processor executes the instructions or code to cause the device to perform the method described in any embodiment of this application.
[0121] The computer storage medium stores code, and when the code is run, the device running the code implements the method described in any embodiment of this application.
[0122] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.
[0123] It is understood that in the specific embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved need to obtain user permission or consent when the above embodiments of this application are applied to specific products or technologies, and the collection, use and processing of related data need to comply with the relevant laws, regulations and standards of relevant countries and regions.
[0124] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0125] It should also be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0126] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A dynamic baseline monitoring method based on network protocol traffic, characterized in that, The method includes: Network traffic data is acquired, and the network traffic data is decoded, parsed and decomposed to obtain four elements of network monitoring. The four elements of network monitoring include monitoring objects, monitoring algorithms, monitoring probes and monitoring indicators. Each of the four elements of network monitoring includes one or more sub-elements. The sub-elements are quantitatively evaluated based on a preset evaluation matrix. The sub-elements with the highest scores among the monitored objects, monitoring algorithms, monitoring probes, and monitoring indicators are selected as the final elements of the four elements of network monitoring in the monitoring scheme. The preset evaluation matrix is constructed based on the four elements and five key points of network monitoring. The five key points include predictive ability, interpretability, stability, localization ability, and operability. The network traffic is monitored based on the aforementioned monitoring scheme.
2. The method according to claim 1, characterized in that, The quantitative evaluation of the sub-elements based on the preset evaluation matrix includes: The ability to assess and locate elements of the monitored object; Evaluate the interpretability, stability, and operability of the monitoring algorithm elements; The stability and positioning capability of the monitoring probe elements were evaluated. Assess the predictive ability, interpretability, stability, and location capability of monitoring indicators.
3. The method according to claim 1, characterized in that, The sub-elements in the monitored object include: The system is divided into application system layer and network infrastructure layer. The application system layer includes the call information of each subsystem, microservice and application module, mainly based on the content of the data packet payload, including application requests, return data and business parameters. The network infrastructure layer includes each IP address and protocol layer, mainly based on the protocol encapsulation content of each data packet header, including TCP / UDP protocol, DNS protocol, HTTP protocol and 5-tuple.
4. The method according to claim 1, characterized in that, The sub-elements in the monitoring algorithm include static baseline algorithm and dynamic baseline algorithm; The static baseline algorithm is based on the expected operation of business traffic and is designed according to business needs for simple monitoring scenarios. The dynamic baseline algorithm takes the actual operation of business traffic as a reference, including using the mean-standard deviation model or Bayesian model to assess the actual business operation risk, and is used in complex monitoring scenarios.
5. The method according to claim 1, characterized in that, The monitoring probes are deployed as needed based on the amount of resources and the network architecture. The network architecture deployment must follow the basic principle of covering the north-south traffic, east-west traffic, important boundary link traffic of external exits, and application system server-side traffic of each network partition. The deployment scheme includes dedicated traffic analysis equipment and dedicated traffic analysis software. The dedicated traffic analysis equipment is used for comprehensive analysis scenarios involving large volumes of traffic and big data, while the dedicated traffic analysis software is applied to server microservice scenarios.
6. The method according to claim 1, characterized in that, The sub-element of the monitoring indicators is the TCP protocol indicator, which is calculated and determined by the connection no-response rate and the TCP packet loss rate.
7. The method according to claim 6, characterized in that, The calculation using connection non-response rate and TCP packet loss rate includes: Connection failure rate = Number of three-way handshake failures / Total number of connections; TCP packet loss rate = Number of packets lost with payload / Total number of TCP packets.
8. A dynamic baseline monitoring system based on network protocol traffic, characterized in that, The system includes: The acquisition unit is used to acquire network traffic data, decode and decompose the network traffic data to obtain four elements of network monitoring. The four elements of network monitoring include monitoring object, monitoring algorithm, monitoring probe and monitoring index. Each of the four elements of network monitoring includes one or more sub-elements. The processing unit is used to quantitatively evaluate the sub-elements based on a preset evaluation matrix, and select the sub-elements with the highest scores among the monitored objects, monitoring algorithms, monitoring probes and monitoring indicators as the final elements of the four elements of network monitoring in the monitoring scheme; the preset evaluation matrix is constructed based on the four elements and five key points of network monitoring, and the five key points include predictive ability, interpretability, stability, localization ability and operability. The monitoring unit is used to monitor the network traffic based on the monitoring scheme.
9. A computing device, characterized in that, The computing device includes: a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the method as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.