Network traffic sampling method, device and equipment and readable storage medium

Abstract

The invention discloses a network traffic sampling method, device and equipment and a readable storage medium, and is applied to the technical field of cloud platforms, and the method comprises the steps: obtaining a kernel programmable filtering program created based on a network monitoring instance; wherein the network monitoring instance is an instance created based on each network monitoring rule; the kernel programmable filtering program is a self-defined network flow filtering program dynamically loaded and executed in a kernel; and filtering the network traffic data of the cloud platform by using a kernel programmable filtering program to obtain target sampling traffic data. According to the invention, the kernel programmable filtering program created based on the network monitoring instance is utilized to sample the network traffic data of the cloud platform, and a user can determine the traffic sampling content corresponding to the kernel programmable filtering program based on requirements, so that the accuracy of network traffic sampling can be improved; and since filtering is directly completed in the kernel mode, data does not need to be copied in a cross-state manner, so that the performance of network traffic sampling is improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud platform technology, and in particular to a network traffic sampling method, device, equipment and readable storage medium. Background Art

[0002] Traditional cloud platform network monitoring generally uses a traffic sampling solution based on the sFlow (Sampled Flow) protocol. sFlow achieves traffic visualization by randomly sampling network packets and aggregating statistical information. Its core relies on hardware or software probes to periodically sample and forward packets. However, sFlow's network traffic collection accuracy is relatively low.

[0003] Therefore, how to improve the accuracy of network traffic collection by the cloud platform, thereby improving the analysis effect of network traffic, is a technical problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a network traffic sampling method, device, equipment and readable storage medium, which solve the technical problem of low accuracy of network traffic collection in the prior art.

[0005] To solve the above technical problems, the present invention provides a network traffic sampling method, comprising:

[0006] Obtaining a kernel programmable filter created based on a network monitoring instance; wherein the network monitoring instance is an instance created based on each network monitoring rule; the kernel programmable filter is a custom network traffic filter dynamically loaded and executed in the kernel;

[0007] The kernel programmable filter is used to filter the network traffic data of the cloud platform to obtain target sampling traffic data.

[0008] In some embodiments, before obtaining the kernel programmable filter program created based on the network monitoring instance, the method further includes:

[0009] generating a kernel programmable filter to be tested for filtering network traffic based on the network monitoring rule data in the network monitoring instance;

[0010] Performing a security test on the kernel programmable filter program to be tested, and obtaining the kernel programmable filter program when it is determined that the kernel programmable filter program to be tested is safe;

[0011] The kernel programmable filter is mounted into the operating system kernel in the cloud platform.

[0012] In some embodiments, the network monitoring rule is a rule including setting basic rule information and traffic filtering conditions; the network monitoring instance is an instance including setting basic instance information and monitoring configuration information.

[0013] In some embodiments, the parameters of the network monitoring rule include rule name, rule description, message direction, message source access control address, message destination access control address, message network address version, message source network address, message destination network address, message network protocol type, message source port and message destination port, and the parameters of the monitoring instance include instance name, instance description, monitoring rule list, sampling rate and instance running status.

[0014] In some embodiments, when the target sampled traffic data is the number and length of packets of each monitoring rule, after filtering the network traffic data of the cloud platform using the kernel programmable filtering program to obtain the target sampled traffic data, the following steps are further included:

[0015] Determine the sampling rate and number of monitoring rules for network monitoring instances;

[0016] Based on the sampling rate of the network monitoring instance, the number of monitoring rules, and the number of messages for each monitoring rule, a message count rate is determined using a first rate model; wherein the sampling rate is a proportion of sampling network traffic; and the first rate model is a model that sums the message count rate for the monitoring rule determined based on the number of messages for the monitoring rule and a statistical time interval;

[0017] Determine a message byte rate using a second quantity model based on the sampling rate of the network monitoring instance, the number of monitoring rules, and the message length of each monitoring rule; wherein the second rate model is a model that sums the message byte rates of the monitoring rules determined based on the message length of the monitoring rules;

[0018] A dynamic graph is generated based on the message number rate and the byte rate, and a visual display is performed based on the dynamic graph; wherein the line graph is a graph reflecting the trend of network traffic changes in the cloud platform.

[0019] In some embodiments, after filtering the network traffic data of the cloud platform using the kernel programmable filtering program to obtain target sampled traffic data, the method further includes:

[0020] Classify the target sampled traffic data according to different network monitoring instances and network monitoring rules to obtain classified data;

[0021] The classification data is stored in chronological order so as to be queried according to the interface, flow in / out direction, network monitoring instance and network monitoring rule.

[0022] In some embodiments, before using the kernel programmable filter to filter the network traffic data of the cloud platform to obtain target sampled traffic data, the method further includes:

[0023] The kernel programmable filter is deployed to all computing nodes and gateway nodes of the cloud platform.

[0024] An embodiment of the present invention further provides a network traffic sampling device, comprising:

[0025] A filter acquisition module is used to acquire a kernel programmable filter created based on a network monitoring instance; wherein the network monitoring instance is an instance created based on each network monitoring rule; the kernel programmable filter is a custom network traffic filter dynamically loaded and executed in the kernel;

[0026] The sampling module is used to filter the network traffic data of the cloud platform using the kernel programmable filtering program to obtain target sampling traffic data.

[0027] An embodiment of the present invention further provides a network traffic sampling device, comprising:

[0028] Memory for storing computer programs;

[0029] A processor is used to execute the computer program to implement the steps of the above-mentioned network traffic sampling method.

[0030] The present invention also provides a computer program product, comprising a computer program / instruction, which implements the steps of the above-mentioned network traffic sampling method when executed by a processor.

[0031] To solve the above technical problems, an embodiment of the present invention provides a network traffic sampling method, which may include: obtaining a kernel programmable filter created based on a network monitoring instance; wherein the network monitoring instance is an instance created based on various network monitoring rules; the kernel programmable filter is a custom network traffic filter dynamically loaded and executed in the kernel; and using the kernel programmable filter to filter the network traffic data of the cloud platform to obtain target sampling traffic data.

[0032] It can be seen from the above technical solution that the beneficial effect of the present invention is that: compared with the current use of sFlow, which leads to poor accuracy in network traffic sampling, the present invention obtains a kernel programmable filter created based on a network monitoring instance; thereby using the kernel programmable filter to sample the network traffic data of the cloud platform. Since the user can determine the traffic sampling content corresponding to the kernel programmable filter based on demand, the accuracy of network traffic sampling can be improved. Moreover, since the kernel programmable filter is located in the kernel, filtering is completed directly in the kernel state, and data does not need to be copied across states, thereby improving the performance of network traffic sampling. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0034] Figure 1 A flowchart of a network traffic sampling method provided by an embodiment of the present invention;

[0035] Figure 2 A flowchart illustrating a network traffic sampling method according to an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of a network traffic sampling system provided by an embodiment of the present invention;

[0037] Figure 4 A schematic diagram of the structure of a network traffic sampling device provided by an embodiment of the present invention;

[0038] Figure 5 A schematic diagram of the structure of a network traffic sampling device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0040] The terms "including" and "having," as used in the present description and accompanying drawings, and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements and may include steps or elements that are not listed.

[0041] In order to enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0042] Next, a network traffic sampling method provided by an embodiment of the present invention is introduced in detail. Figure 1 A flowchart of a network traffic sampling method provided by an embodiment of the present invention may include:

[0043] S101, obtaining a kernel programmable filter created based on a network monitoring instance; wherein the network monitoring instance is an instance created based on various network monitoring rules; the kernel programmable filter is a custom network traffic filter dynamically loaded and executed in the kernel.

[0044] The execution subject of this embodiment is a node of the cloud platform, for example, a computing node of the cloud platform, a gateway node of the cloud platform; or it can also be a filtering monitor service. The network monitoring instance in this embodiment is some combinations of filtering conditions determined according to demand. The network monitoring rules in this embodiment are different network traffic filtering condition rules. The network monitoring rules are structured policy entities used to describe network traffic filtering and monitoring strategies. They realize the identification, processing and statistics of specific network data packets through multi-dimensional matching conditions and action instructions. The logical architecture of the kernel programmable filtering program in this embodiment in the cloud platform network traffic monitoring can be divided into four layers: policy management layer: obtaining the definition, optimization and issuance of monitoring rules; kernel execution layer: the program processes network traffic in real time in the operating system kernel; data aggregation layer: collecting and structuring monitoring data; feedback optimization layer: dynamically adjusting strategies based on monitoring results.

[0045] It should be further explained that before obtaining the kernel programmable filter created based on the network monitoring instance, it can also include: generating a kernel programmable filter to be tested that filters network traffic based on the network monitoring rule data in the network monitoring instance; performing a security test on the kernel programmable filter to be tested, and when it is determined that the kernel programmable filter to be tested is safe, obtaining the kernel programmable filter; and mounting the kernel programmable filter into the operating system kernel in the cloud platform. This embodiment generates a kernel programmable filter to be tested based on the network monitoring rules in each network monitoring instance. The kernel programmable filter to be tested is a program that filters network traffic based on the network monitoring rules. This embodiment will perform a security test on the kernel programmable filter to be tested, so that the kernel programmable filter mounted in the kernel is safe, thereby improving the security of network traffic sampling. The kernel programmable filter is attached to the hook of the kernel system call, and can process network traffic data without affecting system operation. When the program needs to be updated, it can also be seamlessly upgraded.

[0046] It should be further explained that the above-mentioned generation of a kernel programmable filter to be tested for filtering network traffic based on the network monitoring rule data in the network monitoring instance may include: performing a rule semantic integrity check on the network monitoring rule data to obtain a verified network monitoring rule, and generating a kernel programmable filter to be tested based on the verified network monitoring rule and the network monitoring instance and the memory security constraint template and runtime protection instructions. The memory security constraint template is a pre-designed security programming paradigm used to forcibly inject a memory access protection mechanism during the code generation phase. Its core value lies in eliminating common memory security vulnerabilities through automated protection at compile time; in this embodiment, runtime protection instructions can be automatically injected, which refers to the dynamic insertion of security detection code during the program compilation / loading phase, to build a defense-in-depth system without modifying the source code. This technology achieves security enhancement through binary rewriting or compiler intermediate representation operations.

[0047] It should be further explained that the aforementioned network monitoring rules are rules that include basic rule information and traffic filtering conditions; the aforementioned network monitoring instances are instances that include basic instance information and monitoring configuration information. The basic rule information in this embodiment is information that determines the function and direction of action of the rule; the traffic filtering sampling in this embodiment is a filtering rule that is consistent with the action, and the basic instance information in this embodiment is information that determines the function and collection method of the instance. The parameters of the aforementioned network monitoring rules include rule name, rule description, message direction, message source access control address, message destination access control address, message network address version, message source network address, message destination network address, message network protocol type, message source port, and message destination port. The parameters of the aforementioned monitoring instances include instance name, instance description, monitoring rule list, sampling rate, and instance operation status. For example, the parameters of the monitoring instance in this embodiment may include: name: instance name; description: instance description; rules: monitoring rule list; sample: sampling rate; status: instance operation status. In this embodiment, a user sends a request to create a network monitoring rule and specifies the network monitoring instance to which the rule should be added. The network monitoring rule is then created, added to the corresponding network monitoring instance, and stored in the database. The main parameters for a monitoring rule include: name: rule name; description: rule description; bound: packet direction; source_mac_address: packet source MAC (Media Access Control) address; destination_mac_address: packet destination MAC address; ip_version: packet IP (Internet Protocol) version; source_ip_address: packet source IP address; destination_ip_address: packet destination IP address; ip_protocol: packet IP protocol type; source_port: packet source port; destination_port: packet destination port. By providing specific network monitoring rules and network monitoring instances, this embodiment improves the accuracy and comprehensiveness of monitoring by these rules and instances, and facilitates subsequent classification of network traffic based on network traffic data.

[0048] S102: Filter the network traffic data of the cloud platform using a kernel programmable filter program to obtain target sampled traffic data.

[0049] In this embodiment, using a kernel programmable filter to filter the cloud platform's network traffic data to obtain target sampled traffic data refers to filtering the cloud platform's network traffic data based on the kernel programmable filter to obtain filtered network traffic (target sampled traffic data). Alternatively, this can be understood as filtering out traffic that does not comply with network monitoring rules and obtaining traffic that complies with them. This embodiment acquires network traffic data through various ports on the cloud platform. This embodiment can generate kernel programmable filters based on user-defined rules, supporting flexible matching of traffic characteristics (such as IP addresses, ports, and protocols).

[0050] It should be further noted that, based on any of the above embodiments, before using the kernel programmable filter to filter the network traffic data of the cloud platform to obtain the target sampled traffic data, the process may further include: deploying the kernel programmable filter to all compute nodes and gateway nodes of the cloud platform. This embodiment improves the comprehensiveness of the deployed kernel programmable filter by deploying the kernel programmable filter to all compute nodes and gateway nodes of the cloud platform, thereby enabling all compute nodes and all gateway nodes to be monitored directly using the same device.

[0051] It should be further explained that, based on any of the above embodiments, when the target sampled traffic data is the number of packets and the length of packets for each monitoring rule, after filtering the network traffic data of the cloud platform using the kernel programmable filtering program to obtain the target sampled traffic data, the following steps may also be included:

[0052] S1: Determine the sampling rate and number of monitoring rules for a network monitoring instance; determine the message count rate using a first rate model based on the sampling rate of the network monitoring instance, the number of monitoring rules, and the number of messages for each monitoring rule; wherein the sampling rate is the ratio of sampling network traffic; and the first rate model is a model that sums the message count rate for the monitoring rule based on the number of messages for the monitoring rule and the statistical time interval;

[0053] S2: Determine a message byte rate using a second quantity model based on the sampling rate of the network monitoring instance, the number of monitoring rules, and the message length of each monitoring rule; wherein the second rate model is a model that sums the message byte rates of the monitoring rules determined based on the message length of the monitoring rules;

[0054] S3: Generates a dynamic graph based on the packet rate and byte rate, and performs a visual display based on the dynamic graph. The line graph reflects the trend of network traffic changes in the cloud platform.

[0055] In this example, the number of packets and the total length of packets within the specified time period of each network monitoring rule are calculated. The following formula is used to calculate the pps (number of packets per second) of the network packets of the interface monitored by the network monitoring instance, that is, the packet rate:

[0056] ;

[0057] P is the message rate monitored by the monitoring instance, Rsample is the sampling rate of the monitoring instance, T is the statistical time interval, and N is the number of monitoring rules. is the number of sampled packets for the i-th monitoring rule. The following formula is used to calculate the bps (bytes per second) of the network packets of the interface monitored by the monitoring instance, that is, the packet byte rate:

[0058] ;

[0059] B is the message byte rate monitored by the monitoring instance, Rsample is the sampling rate of the monitoring instance, T is the statistical time interval, and N is the number of monitoring rules. is the total length of the sampled packets for the i-th monitoring rule. This embodiment can calculate the average packet rate (packet rate) and average byte rate (byte rate) for each network traffic cache over a preset period of time based on the above formula, so that changes in each network traffic can be clearly displayed during data presentation. This embodiment generates a dynamic graph based on the packet rate and byte rate, and performs a visual display based on the dynamic graph. Because the packet rate and byte rate can demonstrate changes in network traffic, abnormal traffic can be detected through intuitive network traffic changes.

[0060] It should be further explained that after filtering the network traffic data of the cloud platform using the kernel programmable filter program to obtain the target sampled traffic data, the following steps may also be performed:

[0061] Step 1: Classify the target sampled traffic data according to different network monitoring instances and network monitoring rules to obtain classified data;

[0062] Step 2: Store the classified data in chronological order, enabling queries based on interface, traffic inbound and outbound direction, network monitoring instance, and network monitoring rule. This embodiment allows users to query by interface, inbound and outbound direction, or by network monitoring instance or network monitoring rule, helping users analyze OpenStack cloud platform services and providing real-time, effective network monitoring traffic data, improving the user experience. This embodiment enables more detailed analysis of each network traffic flow in each cloud host's inbound and outbound directions.

[0063] The network traffic sampling method provided by an embodiment of the present invention may include: S101, obtaining a kernel programmable filter created based on a network monitoring instance; wherein the network monitoring instance is an instance created based on various network monitoring rules; the kernel programmable filter is a custom network traffic filter dynamically loaded and executed in the kernel; S102, using the kernel programmable filter to filter the network traffic data of the cloud platform to obtain target sampled traffic data. Compared with the current use of sFlow, which results in poor accuracy in network traffic sampling, the present invention obtains a kernel programmable filter created based on a network monitoring instance; thereby using the kernel programmable filter to sample the network traffic data of the cloud platform. Since users can determine the traffic sampling content corresponding to the kernel programmable filter based on their own needs, the accuracy of network traffic sampling can be improved. Moreover, since the kernel programmable filter is located in the kernel, filtering is completed directly in the kernel state, and data does not need to be copied across states, thereby improving the performance of network traffic sampling.

[0064] The present invention is mainly used to implement programmable network monitoring functions on the OpenStack cloud platform and analyze cloud platform traffic.

[0065] As OpenStack cloud computing grows in scale, more and more services are deployed within OpenStack cloud platform clusters, and network traffic is becoming increasingly complex. Users urgently need a flexible network monitoring tool that can monitor cloud platform service network traffic and analyze current OpenStack cloud platform services based on user needs.

[0066] The present invention designs a method for securely, efficiently and flexibly monitoring the network traffic of the OpenStack cloud platform in the kernel, classifies and counts the network monitoring data according to traffic characteristics, interfaces, and ingress and egress directions, and calculates the detailed network traffic data of the cloud platform, providing users with real-time and effective network traffic data for analyzing the business status of the OpenStack cloud platform.

[0067] In order to make the present invention easier to understand, please refer to Figure 2 , Figure 2 A flowchart illustrating a network traffic sampling method provided by an embodiment of the present invention may include:

[0068] S201: Load a filter monitor service plug-in into the neutron-server service of the control node.

[0069] It should be noted that before sampling network traffic, this embodiment needs to load the filter monitor service plug-in in the neutron-server service (server providing the network) of the control node; deploy the filter monitor agent service on all computing nodes and gateway nodes; the filter monitor service plug-in is used to receive network monitoring creation requests, create network monitoring instances, and save them to the database; receive network monitoring association interface requests, dispatch the interface to the filter monitor agent service of the node where it is located, and send the monitoring instance to the filter monitor agent service; the filter monitor agent service starts the kernel programmable filter program for network monitoring, regularly reads network monitoring data, and reports to the filter monitor service plug-in; the filter monitor service plug-in classifies the target sampled traffic according to the network monitoring instance, sorts it by time, and saves it to the database. When receiving a request to view network monitoring data, it reads the corresponding network monitoring data from the database and returns it; it provides users with real-time and effective network monitoring traffic data for analyzing the business situation of the OpenStack cloud platform. For easier understanding, please refer to Figure 3 , Figure 3 A schematic diagram of a network traffic sampling system provided in an embodiment of the present invention. In the figure, Neutron: a module in OpenStack that provides network services; Ovn-nb: northbound database; Ovn-northd: a translation program between the northbound and southbound databases; Ovn-sb: southbound database; Rabbitmq: a message queue; Ovn-controller: a controller; Ovs: a virtual switch; Tap: a virtual network card (VM); Geneve: a virtual tunnel network card; Ovn-controller represents the database local control plane agent; Eth: a physical network card; Overlay: a tunnel network. It is understandable that the embodiment of the present invention divides the execution subject into two according to the executed action: the filter monitor service and the filter monitor agent, which meets the scalability, reliability, and security requirements of the cloud platform. This design enables the system to operate efficiently in large-scale, highly dynamic environments while reducing operation and maintenance complexity.

[0070] This step deploys the filter monitor service plug-in. The filter monitor service plug-in is deployed in the neutron-server service on the control node. It provides interfaces for network traffic monitoring based on kernel-programmable filters, such as creating network monitoring instances, creating network monitoring rules, and associating network monitoring instances with interfaces. The filter monitor service plug-in can access data. When receiving an interface request, it stores the requested data in the database. When receiving monitoring data reported by the filter monitoring proxy service, it also stores this data in the database. The filter monitor service plug-in maintains the status of the filter monitoring proxy service by periodically obtaining data written by the filter monitoring proxy service from the ovn-sb database to maintain and monitor the status of these filter monitoring proxy services. The filter monitor service plug-in schedules network monitoring instances. Once a network monitoring instance is associated with an interface, it schedules the network monitoring instance to the node where the interface resides and sends the network monitoring instance and interface data to the corresponding node's filter monitoring proxy service for interface network data monitoring.

[0071] S202, deploying filtering monitoring agent services on all computing nodes and gateway nodes.

[0072] This embodiment deploys a filtering monitoring agent service on the computing nodes of all gateway nodes. It is necessary to monitor the network conditions of the gateway interface and the cloud host interface, so it is necessary to deploy a filtering monitoring agent service on the computing nodes of all gateway nodes to ensure the integrity of the network traffic sampling function. After the filtering monitoring agent service is started, it will regularly write data such as Agent ID and timestamp to the chassis_private table of the ovn-sb database so that the filtering monitor service plug-in knows the existence of the filtering monitoring agent service; the filtering monitoring agent service receives the network monitoring instance and interface-related data issued by the filtering monitor service, and the filtering monitoring agent service obtains the data of the interface device from the ovs service, and then generates a monitoring code based on the monitoring instance data, starts the kernel programmable filtering program, monitors the network traffic of the interface, and organizes the monitoring data into reports to the filtering monitor service plug-in. It can be understood that in this embodiment, the filtering monitor service plug-in is deployed in the neutron-server service of the control node, which is responsible for receiving user requests, storing rule instances, and scheduling monitoring tasks; the filtering monitoring agent service is deployed on the computing node and the gateway node, which is responsible for generating kernel programmable filtering programs, performing traffic monitoring, and reporting data; the plug-in runs on the control node and is isolated from the core management components to prevent malicious requests from affecting the stability of the cloud platform.

[0073] S203, the filter monitor service plug-in receives the network monitoring creation request, creates a network monitoring instance using network monitoring rules according to the network monitoring request, and saves the instance to the database.

[0074] The network monitoring instance in this embodiment is an instance created based on various network monitoring rules.

[0075] S204, the filter monitor service plug-in receives the network monitoring association interface request, schedules the network monitoring association interface to the filter monitor proxy service of the node, and sends the network monitoring instance to the filter monitor proxy service.

[0076] In this step, the user can send a request to associate an interface with a network monitoring instance. Upon receiving the request, the filter monitoring service plug-in retrieves the host node data for the interface and further retrieves the filter monitoring proxy service data for that host node. It then dispatches the interface to the filter monitoring proxy service on that host node and stores the data in the database. The plug-in then determines whether the filter monitoring proxy service is alive. If so, it sends the network monitoring instance and associated interface data to the filter monitoring proxy service.

[0077] S205, the filtering monitoring agent service creates a kernel programmable filter based on the network monitoring instance, starts the kernel programmable filter to perform network monitoring, regularly reads target sampled traffic data, and reports it to the filtering monitor service plug-in.

[0078] The kernel programmable filter in this embodiment can be a Berkeley packet filter program. This step filters the monitoring agent service to receive the network monitoring instance and the associated interface data. The filtering monitoring agent service accesses the tap device data of the ovs acquisition interface. The filter monitoring agent service network monitoring instance is used to filter the rule data and device data, generate a kernel programmable filter for filtering messages, and after testing the kernel programmable filter safety, it will be hooked into the system call of the operating system kernel to monitor the network traffic on the device; and the sampled messages are matched with the fields in the monitoring rules in turn; if the match is successful, the message count is increased and the message length is recorded; if the match fails, no processing is performed; finally, the number and total length of the sampled messages are stored in chronological order under the MAP data block monitoring rule.

[0079] S206, the filter monitor service plug-in classifies the target sampled traffic data according to the network monitoring instance, and sorts them by time to obtain target network monitoring data, and saves the target network monitoring data to the database.

[0080] In this step, the filter monitor service plug-in receives network traffic monitoring data, categorizes it by network monitoring instances and network monitoring rules, and stores it in chronological order for user queries. Users can query by interface, inbound and outbound direction, or by network monitoring instance or network monitoring rule, providing real-time and effective network monitoring traffic data for user analysis of OpenStack cloud platform services.

[0081] S207, when the filter monitor service plug-in receives the request to view network monitoring data, it reads the corresponding network monitoring data from the database and returns it; it provides real-time and effective network monitoring traffic data for users to analyze the business situation of the OpenStack cloud platform.

[0082] Load the filter monitor service plug-in in the neutron-server service of the control node; deploy the filter monitor agent service on all computing nodes and gateway nodes; the filter monitor service plug-in receives the request to create a network monitoring, creates a network monitoring instance, and saves it to the database; the filter monitor service plug-in receives the network monitoring association interface request, dispatches the interface to the filter monitor agent service of the node, and sends the monitoring instance to the filter monitor agent service; the filter monitor agent service starts the kernel programmable filter program for network monitoring, and the filter monitor agent service regularly reads the network monitoring data and reports it to the filter monitor service plug-in; the filter monitor service plug-in classifies the network monitoring data according to the network monitoring instance, sorts it by time, and saves it to the database; when the filter monitor service plug-in receives the request to view the network monitoring data, it reads the corresponding network monitoring data from the database and returns it; it provides users with real-time and effective network monitoring traffic data for analyzing the business status of the OpenStack cloud platform.

[0083] This embodiment of the present invention is independent of the underlying CPU (central processing unit) architecture and supports multiple architectures, including x86 and ARM. It also designs a network traffic sampling method based on kernel programmable filtering. By utilizing kernel programmable filtering technology, network traffic on the OpenStack cloud platform is securely, efficiently, and flexibly monitored within the kernel. Network monitoring data is classified, counted, and calculated based on traffic characteristics, interfaces, and inbound and outbound directions, yielding detailed network traffic data for the cloud platform. This provides users with real-time, effective network traffic data for analyzing OpenStack cloud platform services.

[0084] A network traffic sampling device provided by an embodiment of the present invention is introduced below. The network traffic sampling device described below and the network traffic sampling method described above can be referenced to each other.

[0085] Figure 4A schematic diagram of the structure of a network traffic sampling device provided in an embodiment of the present invention may include:

[0086] The filter acquisition module 100 is used to acquire a kernel programmable filter created based on a network monitoring instance; wherein the network monitoring instance is an instance created based on each network monitoring rule; the kernel programmable filter is a custom network traffic filter dynamically loaded and executed in the kernel;

[0087] The sampling module 200 is used to filter the network traffic data of the cloud platform using the kernel programmable filtering program to obtain target sampling traffic data.

[0088] Furthermore, based on the above embodiment, the network traffic sampling device may further include:

[0089] A kernel programmable filter program generation module to be tested, configured to generate a kernel programmable filter program to be tested for filtering network traffic based on the network monitoring rule data in the network monitoring instance;

[0090] a security testing module, configured to perform a security test on the kernel programmable filter program to be tested, and obtain the kernel programmable filter program when it is determined that the kernel programmable filter program to be tested is safe;

[0091] The mounting module is used to mount the kernel programmable filter program into the operating system kernel in the cloud platform.

[0092] Further, based on any of the above embodiments, the network monitoring rule is a rule including setting basic rule information and traffic filtering conditions; the network monitoring instance is an instance including setting basic instance information and monitoring configuration information.

[0093] Further, based on any of the above embodiments, the parameters of the network monitoring rules include rule name, rule description, message direction, message source access control address, message destination access control address, message network address version, message source network address, message destination network address, message network protocol type, message source port and message destination port, and the parameters of the monitoring instance include instance name, instance description, monitoring rule list, sampling rate and instance running status.

[0094] Furthermore, based on the above embodiment, the network traffic sampling device may further include:

[0095] A sampling rate and quantity determination module, used to determine the sampling rate and quantity of monitoring rules for a network monitoring instance;

[0096] a message rate determination module, configured to determine a message rate using a first rate model based on a sampling rate of the network monitoring instance, the number of monitoring rules, and the number of messages for each monitoring rule; wherein the sampling rate is a proportion of network traffic sampled; and the first rate model is a model that sums the message rate for the monitoring rule determined based on the number of messages for the monitoring rule and a statistical time interval;

[0097] a message byte rate determination module, configured to determine a message byte rate using a second quantity model based on a sampling rate of the network monitoring instance, the number of monitoring rules, and the message length of each monitoring rule; wherein the second rate model is a model that sums the message byte rates of the monitoring rules determined based on the message length of the monitoring rules;

[0098] A visualization display module is used to generate a dynamic graph based on the message number rate and the byte rate, and perform visualization based on the dynamic graph; wherein the line graph is a graph reflecting the trend of network traffic changes in the cloud platform.

[0099] Furthermore, based on the above embodiment, the network traffic sampling device may further include:

[0100] A classification module, configured to classify the target sampled traffic data according to different network monitoring instances and network monitoring rules to obtain classified data;

[0101] The query module is used to store the classified data in chronological order so as to enable querying according to the interface, flow in / out direction, network monitoring instance and network monitoring rule.

[0102] Furthermore, based on the above embodiment, the network traffic sampling device may further include:

[0103] A deployment module is used to deploy the kernel programmable filter to all computing nodes and gateway nodes of the cloud platform.

[0104] It should be noted that the order of the modules and units in the above-mentioned network traffic sampling device can be changed without affecting the logic.

[0105] Figure 4 The description of the features in the corresponding embodiment can be found in Figure 4 The relevant descriptions of the corresponding embodiments will not be repeated here one by one.

[0106] The network traffic sampling device provided by an embodiment of the present invention may include: a filter program acquisition module 100 for acquiring a kernel programmable filter program created based on a network monitoring instance; wherein the network monitoring instance is an instance created based on various network monitoring rules; the kernel programmable filter program is a custom network traffic filter program dynamically loaded and executed in the kernel; and a sampling module 200 for using the kernel programmable filter program to filter the network traffic data of the cloud platform to obtain target sampled traffic data. Compared with the current use of sFlow, which results in poor accuracy in network traffic sampling, the present invention acquires a kernel programmable filter program created based on a network monitoring instance; thereby using the kernel programmable filter program to sample the network traffic data of the cloud platform. Since the user can determine the traffic sampling content corresponding to the kernel programmable filter program based on demand, the accuracy of network traffic sampling can be improved. Moreover, since the kernel programmable filter program is located in the kernel, filtering is completed directly in the kernel state, and data does not need to be copied across states, thereby improving the performance of network traffic sampling.

[0107] A network traffic sampling device provided by an embodiment of the present invention is introduced below. The network traffic sampling device described below and the network traffic sampling method described above can be referenced to each other.

[0108] Figure 5 A schematic diagram of the structure of a network traffic sampling device provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the network traffic sampling device includes: a memory 60 for storing computer programs;

[0109] The processor 61 is configured to implement the steps of the network traffic sampling method of the above embodiment when executing a computer program.

[0110] The network traffic sampling device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0111] The processor 61 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 61 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing content required to be displayed on the display screen. In some embodiments, the processor 61 may also include an artificial intelligence (AI) processor for handling computational operations related to machine learning.

[0112] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 60 is at least used to store the following computer program 601, wherein, after the computer program is loaded and executed by the processor 61, it can implement the relevant steps of the network traffic sampling method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 602 may include Windows, Unix, Linux, etc. The data 603 may include but is not limited to data of the network traffic sampling method process, etc.

[0113] In some embodiments, the network traffic sampling device may further include a display screen 62 , an input / output interface 63 , a communication interface 64 , a power supply 65 , and a communication bus 66 .

[0114] Those skilled in the art will understand that Figure 5 The structure shown in the figure does not constitute a limitation to the network traffic sampling device, and may include more or fewer components than shown in the figure.

[0115] It is understandable that if the network traffic sampling method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the current technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk or optical disk, etc. Various media that can store program code.

[0116] Based on this, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned network traffic sampling method are implemented.

[0117] The above describes in detail a network traffic sampling method provided by an embodiment of the present invention. The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. For similar or identical parts between the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For relevant details, refer to the method description.

[0118] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0119] The above is a detailed introduction to a network traffic sampling method, device, equipment and readable storage medium provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in several ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A network traffic sampling method, characterized in that: include: Obtaining a kernel programmable filter created based on a network monitoring instance; wherein the network monitoring instance is an instance created based on each network monitoring rule; the kernel programmable filter is a custom network traffic filter dynamically loaded and executed in the kernel; The kernel programmable filter is used to filter the network traffic data of the cloud platform to obtain target sampling traffic data.

2. The network traffic sampling method according to claim 1, characterized in that: Before obtaining the kernel programmable filter created based on the network monitoring instance, it also includes: generating a kernel programmable filter to be tested for filtering network traffic based on the network monitoring rule data in the network monitoring instance; Performing a security test on the kernel programmable filter program to be tested, and obtaining the kernel programmable filter program when it is determined that the kernel programmable filter program to be tested is safe; The kernel programmable filter is mounted into the operating system kernel in the cloud platform.

3. The network traffic sampling method according to claim 1, characterized in that: The network monitoring rule is a rule including setting basic rule information and traffic filtering conditions; the network monitoring instance is an instance including setting basic instance information and monitoring configuration information.

4. The network traffic sampling method according to claim 2, characterized in that: The parameters of the network monitoring rule include rule name, rule description, message direction, message source access control address, message destination access control address, message network address version, message source network address, message destination network address, message network protocol type, message source port and message destination port. The parameters of the monitoring instance include instance name, instance description, monitoring rule list, sampling rate and instance running status.

5. The network traffic sampling method according to any one of claims 1 to 4, characterized in that: When the target sampled traffic data is the number and length of messages of each monitoring rule, after filtering the network traffic data of the cloud platform using the kernel programmable filtering program to obtain the target sampled traffic data, the method further includes: Determine the sampling rate and number of monitoring rules for network monitoring instances; Based on the sampling rate of the network monitoring instance, the number of monitoring rules, and the number of messages for each monitoring rule, a message count rate is determined using a first rate model; wherein the sampling rate is a proportion of sampling network traffic; and the first rate model is a model that sums the message count rate for the monitoring rule determined based on the number of messages for the monitoring rule and a statistical time interval; Determine a message byte rate using a second quantity model based on the sampling rate of the network monitoring instance, the number of monitoring rules, and the message length of each monitoring rule; wherein the second rate model is a model that sums the message byte rates of the monitoring rules determined based on the message length of the monitoring rules; A dynamic graph is generated based on the message number rate and the byte rate, and a visual display is performed based on the dynamic graph; wherein the line graph is a graph reflecting the trend of network traffic changes in the cloud platform.

6. The network traffic sampling method according to claim 1, characterized in that: After filtering the network traffic data of the cloud platform using the kernel programmable filtering program to obtain target sampled traffic data, the method further includes: Classify the target sampled traffic data according to different network monitoring instances and network monitoring rules to obtain classified data; The classification data is stored in chronological order so as to be queried according to the interface, flow in / out direction, network monitoring instance and network monitoring rule.

7. The network traffic sampling method according to claim 1, characterized in that: Before using the kernel programmable filter program to filter the network traffic data of the cloud platform to obtain target sampled traffic data, the method further includes: The kernel programmable filter is deployed to all computing nodes and gateway nodes of the cloud platform.

8. A network traffic sampling device, characterized in that: include: A filter acquisition module is used to acquire a kernel programmable filter created based on a network monitoring instance; wherein the network monitoring instance is an instance created based on each network monitoring rule; the kernel programmable filter is a custom network traffic filter dynamically loaded and executed in the kernel; The sampling module is used to filter the network traffic data of the cloud platform using the kernel programmable filtering program to obtain target sampling traffic data.

9. A network traffic sampling device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the network traffic sampling method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the network traffic sampling method according to any one of claims 1 to 7.

Images