Network traffic distribution method, device and equipment

Abstract

The invention discloses a network traffic distribution method, device and equipment, and the method is applied to the technical field of logistics, and the method comprises the steps: collecting node-level and container-level network traffic information in each Pod in a Kubernetes cluster according to a first collection frequency through employing a data monitoring tool; collecting resource index data of each Pod in the Kubernetes cluster according to a second collection frequency by adopting the data monitoring tool; according to the network flow information and the resource index data, clustering each Pod according to the network flow and the resource use condition to obtain at least one load balancing group; a load balancing strategy is adopted to adjust a load balancer of Kubernetes, and network flow is uniformly distributed to each load balancing group. According to the invention, the problems of resource waste and network performance bottleneck caused by static IP allocation in a large-scale Kubernetes cluster are solved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to a network traffic allocation method, apparatus, and device. Background Art

[0002] In modern cloud computing environments, Kubernetes has become the standard for container orchestration. However, as the cluster scale expands, the management of IP addresses and the optimization of network performance become particularly important. Therefore, it is necessary to solve the problems of resource waste and network performance bottlenecks caused by static IP allocation in large-scale Kubernetes clusters. Summary of the Invention

[0003] The present invention provides a network traffic allocation method, apparatus, and device, which can adjust the load balancer of Kubernetes by using a load balancing strategy to evenly distribute network traffic to each load balancing group, and solve the problems of resource waste and network performance bottlenecks caused by static IP allocation.

[0004] On the one hand, the present invention provides a network traffic allocation method, the method comprising: Setting monitoring metrics and data collection frequencies of a data monitoring tool; the monitoring metrics include network traffic and resource metrics of a Kubernetes cluster, and the data collection frequencies include a first collection frequency corresponding to the network traffic and a second collection frequency corresponding to the resource metrics; Collecting network traffic information at the node level and container level in each Pod in the Kubernetes cluster by using the data monitoring tool according to the first collection frequency; Collecting resource metric data of each Pod in the Kubernetes cluster by using the data monitoring tool according to the second collection frequency; Clustering each Pod according to the network traffic information and the resource metric data according to network traffic and resource usage to obtain at least one load balancing group; Adjusting the load balancer of Kubernetes by using a load balancing strategy to evenly distribute network traffic to each load balancing group.

[0005] In an exemplary embodiment, the clustering each Pod according to the network traffic information and the resource metric data according to network traffic and resource usage to obtain at least one load balancing group includes: Using an anomaly detection algorithm to perform outlier identification processing on the network traffic information and the resource metric data, and removing the outliers to obtain processed data; Normalize the processed data to obtain normalized data; Use the K-Means algorithm to cluster each Pod according to the network traffic and resource usage in the normalized data, obtaining at least one load balancing group.

[0006] In an exemplary embodiment, the method further includes: For the normalized data corresponding to each Pod, extract the network attribute features, CPU usage rate, and memory usage rate in the normalized data; Use a dimensionality reduction algorithm to perform dimensionality reduction processing on the network attribute features, CPU usage rate, and memory usage rate, obtaining the target comprehensive feature corresponding to each Pod; Determine the IP address corresponding to each Pod according to the target comprehensive feature corresponding to each Pod.

[0007] In an exemplary embodiment, the determining the IP address corresponding to each Pod according to the target comprehensive feature corresponding to each Pod includes: Use a classification algorithm to perform classification processing on the target comprehensive feature corresponding to each Pod, obtaining the target traffic category of each Pod; Determine the Pod with the target traffic category of high traffic as a high traffic Pod, and determine the Pod with the target traffic category of low traffic as a low traffic Pod; Use a genetic algorithm or a simulated annealing algorithm to assign a first IP address to the high traffic Pod and assign a second IP address to the low traffic Pod; the performance of the first IP address is better than the performance of the second IP address.

[0008] In an exemplary embodiment, the method further includes: Use a convolutional neural network or a recurrent neural network to predict the traffic of each load balancing group, obtaining a prediction result; Dynamically adjust the load balancing strategy according to the prediction result, and reallocate the traffic of each load balancing group according to the adjusted load balancing strategy.

[0009] In an exemplary embodiment, the method further includes: Use a linear regression algorithm or a time series analysis algorithm to predict the network traffic demand of the Kubernetes cluster in a future period, obtaining a network traffic prediction result; Dynamically adjust the IP address pool capacity of the Kubernetes cluster according to the network traffic prediction result; Use a particle swarm optimization algorithm or an ant colony algorithm to determine the IP allocation strategy of the IP address pool.

[0010] In an exemplary embodiment, the method further includes: Obtaining data of an initial IP address pool of the Kubernetes cluster, and analyzing the data to obtain network traffic change data; Adjusting network bandwidth limits and packet priorities of the Kubernetes cluster according to the network traffic change data.

[0011] On the other hand, a network traffic allocation device is provided, and the device includes: A data setting module, configured to set monitoring metrics and data collection frequencies of a data monitoring tool; the monitoring metrics include network traffic and resource metrics of the Kubernetes cluster, and the data collection frequencies include a first collection frequency corresponding to the network traffic and a second collection frequency corresponding to the resource metrics; An information collection module, configured to collect network traffic information at node level and container level in each Pod in the Kubernetes cluster according to the first collection frequency by using the data monitoring tool; A data collection module, configured to collect resource metric data of each Pod in the Kubernetes cluster according to the second collection frequency by using the data monitoring tool; A clustering module, configured to cluster each Pod according to network traffic and resource usage conditions based on the network traffic information and the resource metric data to obtain at least one load balancing group; A traffic allocation module, configured to adjust a load balancer of Kubernetes by using a load balancing strategy to evenly distribute network traffic to each load balancing group.

[0012] In an exemplary embodiment, the clustering module includes: A data processing unit, configured to identify and remove outliers from the network traffic information and the resource metric data by using an outlier detection algorithm to obtain processed data; A normalization unit, configured to perform normalization processing on the processed data to obtain normalized data; A clustering unit, configured to cluster each Pod according to network traffic and resource usage conditions in the normalized data by using the K-Means algorithm to obtain at least one load balancing group.

[0013] In an exemplary embodiment, the device further includes: An information extraction module, configured to extract network attribute features, CPU usage rate, and memory usage rate in the normalized data for the normalized data corresponding to each Pod; A target feature determination module, which is used to perform dimensionality reduction processing on the network attribute features, CPU usage rate, and memory usage rate by using a dimensionality reduction algorithm to obtain the target comprehensive feature corresponding to each Pod; An address determination module, which is used to determine the IP address corresponding to each Pod according to the target comprehensive feature corresponding to each Pod.

[0014] In an exemplary embodiment, the address determination module includes: A traffic category determination unit, which is used to perform classification processing on the target comprehensive feature corresponding to each Pod by using a classification algorithm to obtain the target traffic category of each Pod; A classification unit, which is used to determine the Pod with the target traffic category of high traffic category as a high traffic Pod, and determine the Pod with the target traffic category of low traffic category as a low traffic Pod; An address allocation unit, which is used to allocate a first IP address to the high traffic Pod and a second IP address to the low traffic Pod by using a genetic algorithm or a simulated annealing algorithm; the performance of the first IP address is better than that of the second IP address.

[0015] In an exemplary embodiment, the device further includes: A prediction module, which is used to predict the traffic of each load balancing group by using a convolutional neural network or a recurrent neural network to obtain a prediction result; A traffic allocation module, which is used to dynamically adjust the load balancing strategy according to the prediction result, and re-allocate the traffic of each load balancing group according to the adjusted load balancing strategy.

[0016] In an exemplary embodiment, the device further includes: A network traffic prediction module, which is used to predict the network traffic demand of the Kubernetes cluster in a future period by using a linear regression algorithm or a time series analysis algorithm to obtain a network traffic prediction result; A capacity adjustment module, which is used to dynamically adjust the IP address pool capacity of the Kubernetes cluster according to the network traffic prediction result; A policy determination module, which is used to determine the IP allocation policy of the IP address pool by using a particle swarm optimization algorithm or an ant colony algorithm.

[0017] In an exemplary embodiment, the device further includes: A data analysis module, which is used to obtain the data of the initial IP address pool of the Kubernetes cluster and analyze the data to obtain network traffic change data; An information adjustment module is configured to adjust the network bandwidth limit and packet priority of the Kubernetes cluster according to the network traffic change data.

[0018] On the other hand, an electronic device is provided, which includes a processor and a memory. At least one instruction or at least one program segment is stored in the memory. The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the network traffic allocation method as described above.

[0019] On the other hand, a computer storage medium is provided. The computer storage medium stores at least one instruction or at least one program segment. The at least one instruction or the at least one program segment is loaded and executed by a processor to implement the network traffic allocation method as described above.

[0020] On the other hand, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device executes to implement the network traffic allocation method as described above.

[0021] The network traffic allocation method, device and equipment provided by the present invention have the following technical effects: The present invention sets the monitoring metrics and data collection frequencies of the data monitoring tool; the monitoring metrics include the network traffic and resource metrics of the Kubernetes cluster, and the data collection frequencies include the first collection frequency corresponding to the network traffic and the second collection frequency corresponding to the resource metrics; the data monitoring tool is used to collect the network traffic information at the node level and container level in each Pod in the Kubernetes cluster according to the first collection frequency; the data monitoring tool is used to collect the resource metric data of each Pod in the Kubernetes cluster according to the second collection frequency; according to the network traffic information and the resource metric data, each Pod is clustered according to the network traffic and resource usage conditions to obtain at least one load balancing group; a load balancing strategy is used to adjust the load balancer of Kubernetes, and the network traffic is evenly distributed to each load balancing group. The present invention solves the problems of resource waste and network performance bottleneck caused by static IP allocation in a large-scale Kubernetes cluster. Description of the Drawings

[0022] To more clearly illustrate the technical solutions and advantages in the embodiments of this specification or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0023] Figure 1 It is a schematic diagram of a network traffic allocation system provided by an embodiment of this specification; Figure 2 It is a schematic flowchart of a network traffic allocation method provided by an embodiment of this specification; Figure 3 It is a schematic flowchart of a method provided by an embodiment of this specification for clustering each Pod according to the network traffic information and the resource metric data according to the network traffic and resource usage conditions to obtain at least one load balancing group; Figure 4 It is a schematic flowchart of a method provided by an embodiment of this specification for determining the IP address corresponding to each Pod; Figure 5 It is a schematic flowchart of a method provided by an embodiment of this specification for determining the IP address corresponding to each Pod according to the target comprehensive feature corresponding to each Pod; Figure 6 It is a schematic diagram of the structure of a network traffic allocation device provided by an embodiment of this specification; Figure 7 It is a schematic diagram of the structure of a server provided by an embodiment of this specification. Detailed implementation manners

[0024] The following will clearly and completely describe the technical solutions in the embodiments of this specification with reference to the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0026] Please refer to Figure 1 , Figure 1 which is a schematic diagram of a network traffic distribution system provided by an embodiment of this specification. As Figure 1 shown, the network traffic distribution system may at least include server 01 and client 02.

[0027] Specifically, in the embodiment of this specification, the server 01 may include an independently operating server, or a distributed server, or a server cluster composed of multiple servers, and may also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 01 may include a network communication unit, a processor, a memory, and so on. Specifically, the server 01 may be used to adjust the load balancer of Kubernetes using a load balancing strategy and evenly distribute network traffic to each load balancing group.

[0028] Specifically, in the embodiment of this specification, the client 02 may include entity devices such as smart phones, desktop computers, tablet computers, laptop computers, digital assistants, smart wearable devices, smart speakers, vehicle terminals, smart TVs, etc., and may also include software running on the entity devices. For example, some web pages provided by some service providers to users, or applications provided by these service providers to users. Specifically, the client 02 may be used to display the load balancing strategy.

[0029] The following introduces a network traffic distribution method of the present invention. Figure 2It is a schematic flowchart of a network traffic allocation method provided by an embodiment of this specification. This specification provides the method operation steps as described in the embodiment or flowchart, but based on routine or non-creative labor, it may include more or fewer operation steps. The step order listed in the embodiment is only one of the execution orders of numerous steps and does not represent the only execution order. When the actual system or server product executes, it can be executed in the method order shown in the embodiment or the accompanying drawings or executed in parallel (for example, in an environment with parallel processors or multi-threaded processing). Specifically, as Figure 2 shown, the method may include: S201: Set the monitoring metrics and data collection frequency of the data monitoring tool; the monitoring metrics include the network traffic and resource metrics of the Kubernetes cluster, and the data collection frequency includes the first collection frequency corresponding to the network traffic and the second collection frequency corresponding to the resource metrics. In the embodiment of this specification, the network traffic data and resource usage of each Pod in the Kubernetes cluster can be collected in real time, the network traffic and resource metrics can be set, as well as the first collection frequency corresponding to the network traffic and the second collection frequency corresponding to the resource metrics.

[0030] S203: Use the data monitoring tool to collect the network traffic information at the node level and container level in each Pod of the Kubernetes cluster according to the first collection frequency.

[0031] S205: Use the data monitoring tool to collect the resource metric data of each Pod in the Kubernetes cluster according to the second collection frequency.

[0032] In the embodiments of this specification, the second collection frequency may be the same as or different from the first collection frequency, and monitoring tools such as Prometheus can be used to collect data. Based on the network monitoring plugin in the data monitoring tool, the network traffic information at the node level and container level in each Pod in the Kubernetes cluster is collected; based on the resource monitoring plugin in the data monitoring tool, the resource metric data of the Kubernetes cluster is collected; when using the Prometheus monitoring tool for data collection, it is first necessary to configure Prometheus to ensure that the network traffic data and resource usage of each Pod can be accurately collected. The configuration includes defining the metrics to be monitored, setting the data collection frequency, etc.; for network traffic data, network monitoring plugins of Prometheus, such as Node Exporter and cAdvisor, can be used to collect the network traffic information at the node level and container level respectively. These plugins can provide metrics such as network throughput, number of data packets, and number of network connections; for resource usage, the resource metrics of the Kubernetes cluster can be monitored through Prometheus, such as CPU usage, memory usage, disk I / O, etc.

[0033] S207: According to the network traffic information and the resource metric data, cluster each Pod according to the network traffic and resource usage to obtain at least one load balancing group.

[0034] In the embodiments of this specification, Pods can be clustered according to the network traffic and resource usage. Each cluster can be regarded as a load balancing group.

[0035] S209: Adjust the load balancer of Kubernetes using a load balancing strategy to evenly distribute the network traffic to each load balancing group.

[0036] In the embodiments of this specification, by adjusting the load balancer of Kubernetes, the network traffic is evenly distributed to each load balancing group; a convolutional neural network (CNN) or a recurrent neural network (RNN) is used to predict the network traffic, and the load balancing strategy is dynamically adjusted according to the prediction result to ensure the even distribution of the network traffic.

[0037] In the embodiments of this specification, as Figure 3 shown, the step of clustering each Pod according to the network traffic information and the resource metric data according to the network traffic and resource usage to obtain at least one load balancing group includes: SS2071: Use an anomaly detection algorithm to identify outliers in the network traffic information and the resource metric data, and remove the outliers to obtain processed data; S2073: Perform normalization processing on the processed data to obtain normalized data; S2075: Use the K-Means algorithm to cluster each Pod according to the network traffic and resource usage in the normalized data to obtain at least one load balancing group.

[0038] In the embodiments of this specification, the network attribute features include the current network traffic, average network traffic, peak network traffic, and the number of network connections. The collected data may contain noise and outliers, and data preprocessing is required. Machine learning anomaly detection algorithms, such as the Isolation Forest algorithm, can be used to identify and remove outliers. Normalize the data to map data of different metrics to the same numerical range. For example, the Min-Max normalization method can be used to map the data to the interval [0,1]. Use the K-Means algorithm to cluster Pods according to network traffic and resource usage. Each cluster can be regarded as a load balancing group.

[0039] In the embodiments of this specification, as Figure 4 shown, the method further includes: S401: For the normalized data corresponding to each Pod, extract the network attribute features, CPU usage rate, and memory usage rate in the normalized data; S403: Use a dimensionality reduction algorithm to perform dimensionality reduction processing on the network attribute features, CPU usage rate, and memory usage rate to obtain the target comprehensive feature corresponding to each Pod; S405: Determine the IP address corresponding to each Pod according to the target comprehensive feature corresponding to each Pod.

[0040] In the embodiments of this specification, features are extracted from the collected data for the dynamic IP address allocation algorithm. The features can include the current network traffic, average network traffic, peak network traffic, CPU usage rate, memory usage rate, number of network connections, etc. of the Pod; use dimensionality reduction algorithms such as principal component analysis (PCA) to process the features, reduce the dimensionality of the features, and improve the efficiency and accuracy of the algorithm.

[0041] In the embodiments of this specification, as Figure 5 shown, the determining the IP address corresponding to each Pod according to the target comprehensive feature corresponding to each Pod includes: S4051: Classify the target comprehensive features corresponding to each Pod using a classification algorithm to obtain the target traffic category of each Pod; S4053: Determine the Pods with the target traffic category of high traffic category as high - traffic Pods, and determine the Pods with the target traffic category of low traffic category as low - traffic Pods; S4055: Allocate a first IP address to the high - traffic Pods and a second IP address to the low - traffic Pods using a genetic algorithm or a simulated annealing algorithm; the performance of the first IP address is better than that of the second IP address.

[0042] In the embodiments of this specification, the network topology structure composed of each Pod can be determined by analyzing the network architecture of the Kubernetes cluster; a classification algorithm in machine learning, such as a support vector machine (SVM) or a random forest algorithm, is used to classify Pods into high - traffic Pods and low - traffic Pods according to their characteristics; for high - traffic Pods, considering their traffic requirements, resource usage, and network topology structure, an optimization algorithm, such as a genetic algorithm (Genetic Algorithm) or a simulated annealing algorithm (Simulated Annealing Algorithm), is used to allocate better - quality IP addresses; the network topology structure can be determined by analyzing the network architecture of the Kubernetes cluster. For example, factors such as the distance between Pods and other nodes and network latency are considered.

[0043] In the embodiments of this specification, the method further includes: Predict the traffic of each load - balancing group using a convolutional neural network or a recurrent neural network to obtain a prediction result; Dynamically adjust the load - balancing strategy according to the prediction result, and re - allocate the traffic of each load - balancing group according to the adjusted load - balancing strategy.

[0044] In the embodiments of this specification, a convolutional neural network (CNN) or a recurrent neural network (RNN) can be used to predict network traffic, and the load balancing policy can be dynamically adjusted according to the prediction results to ensure the uniform distribution of network traffic. A network traffic prediction model can also be trained based on the convolutional neural network or the recurrent neural network, and then the traffic of each load balancing group can be predicted according to the network traffic prediction model and the real-time collected data. Algorithms such as Q-Learning or Deep Q-Network (DQN) can be used to learn the optimal traffic control policy. The reinforcement learning algorithm can automatically adjust traffic control parameters such as bandwidth limits and packet priorities according to the network state and traffic requirements. By monitoring network traffic and resource usage, the reward function of the reinforcement learning algorithm can be adjusted in real time to encourage the algorithm to find the optimal traffic control policy.

[0045] In the embodiments of this specification, the method further includes: Using a linear regression algorithm or a time series analysis algorithm to predict the network traffic demand of the Kubernetes cluster in a future period to obtain a network traffic prediction result; Dynamically adjusting the capacity of the IP address pool of the Kubernetes cluster according to the network traffic prediction result; Using a particle swarm optimization algorithm or an ant colony algorithm to determine the IP allocation policy of the IP address pool.

[0046] In the embodiments of this specification, define the initial IP address pool of the Kubernetes cluster and configure the corresponding network plugin, set the time interval for data collection and analysis, and dynamically adjust the IP address allocation and network policy according to the analysis result.

[0047] Using a linear regression or a time series analysis algorithm to predict the future network traffic demand, and dynamically adjusting the size of the IP address pool of the Kubernetes cluster according to the prediction result to ensure that there are enough IP addresses available for allocation; using a particle swarm optimization (PSO) algorithm or an ant colony optimization algorithm to optimize the allocation policy of the IP address pool and improve the utilization rate of IP addresses.

[0048] In the embodiments of this specification, the method further includes: Obtaining the data of the initial IP address pool of the Kubernetes cluster and analyzing the data to obtain network traffic change data; Adjust the network bandwidth limit and packet priority of the Kubernetes cluster according to the network traffic change data.

[0049] In the embodiments of this specification, the C4.5 or ID3 algorithm can be used to automatically adjust the network policy of the Kubernetes cluster according to the results of data collection and analysis. For example, according to the change of network traffic, policies such as network bandwidth limit and packet priority can be automatically adjusted. The Autoencoder algorithm is used to compress and extract features from the network traffic data, and the network policy is automatically adjusted according to the extracted features to improve network performance and stability.

[0050] As can be seen from the technical solutions provided by the embodiments of this specification above, the embodiments of this specification set the monitoring metrics and data collection frequency of the data monitoring tool; the monitoring metrics include the network traffic and resource metrics of the Kubernetes cluster, and the data collection frequency includes the first collection frequency corresponding to the network traffic and the second collection frequency corresponding to the resource metrics; use the data monitoring tool to collect the network traffic information at the node level and container level in each Pod in the Kubernetes cluster according to the first collection frequency; use the data monitoring tool to collect the resource metric data of each Pod in the Kubernetes cluster according to the second collection frequency; cluster each Pod according to the network traffic information and the resource metric data according to the network traffic and resource usage conditions to obtain at least one load balancing group; use the load balancing strategy to adjust the load balancer of Kubernetes and evenly distribute the network traffic to each load balancing group. The present invention solves the problems of resource waste and network performance bottleneck caused by static IP allocation in a large-scale Kubernetes cluster.

[0051] The embodiments of this specification also provide a network traffic distribution device, as Figure 6 shown, the device includes: A data setting module 610, configured to set the monitoring metrics and data collection frequency of the data monitoring tool; the monitoring metrics include the network traffic and resource metrics of the Kubernetes cluster, and the data collection frequency includes the first collection frequency corresponding to the network traffic and the second collection frequency corresponding to the resource metrics; An information collection module 620, configured to use the data monitoring tool to collect the network traffic information at the node level and container level in each Pod in the Kubernetes cluster according to the first collection frequency; A data collection module 630, configured to collect resource metric data of each Pod in the Kubernetes cluster according to the second collection frequency by using the data monitoring tool; A clustering module 640, configured to cluster each Pod according to the network traffic information and the resource metric data according to network traffic and resource usage conditions, to obtain at least one load balancing group; A traffic allocation module 650, configured to adjust the load balancer of Kubernetes by using a load balancing strategy, and evenly allocate network traffic to each load balancing group.

[0052] In an exemplary embodiment, the clustering module includes: A data processing unit, configured to perform outlier identification processing on the network traffic information and the resource metric data by using an anomaly detection algorithm, and remove the outliers to obtain processed data; A normalization unit, configured to perform normalization processing on the processed data to obtain normalized data; A clustering unit, configured to cluster each Pod according to network traffic and resource usage conditions in the normalized data by using the K-Means algorithm, to obtain at least one load balancing group.

[0053] In an exemplary embodiment, the apparatus further includes: An information extraction module, configured to extract network attribute features, CPU usage rate, and memory usage rate in the normalized data for the normalized data corresponding to each Pod; A target feature determination module, configured to perform dimensionality reduction processing on the network attribute features, CPU usage rate, and memory usage rate by using a dimensionality reduction algorithm, to obtain a target comprehensive feature corresponding to each Pod; An address determination module, configured to determine an IP address corresponding to each Pod according to the target comprehensive feature corresponding to each Pod.

[0054] In an exemplary embodiment, the address determination module includes: A traffic category determination unit, configured to perform classification processing on the target comprehensive feature corresponding to each Pod by using a classification algorithm, to obtain a target traffic category of each Pod; A classification unit, configured to determine a Pod with a target traffic category of a high traffic category as a high traffic Pod, and determine a Pod with a target traffic category of a low traffic category as a low traffic Pod; An address allocation unit, configured to allocate a first IP address to the high traffic Pod and allocate a second IP address to the low traffic Pod by using a genetic algorithm or a simulated annealing algorithm; the performance of the first IP address is superior to the performance of the second IP address.

[0055] In an exemplary embodiment, the device further comprises: A prediction module, configured to predict the traffic of each load balancing group by using a convolutional neural network or a recurrent neural network to obtain a prediction result; A traffic allocation module, configured to dynamically adjust the load balancing policy according to the prediction result, and re-allocate the traffic of each load balancing group according to the adjusted load balancing policy.

[0056] In an exemplary embodiment, the device further comprises: A network traffic prediction module, configured to predict the network traffic demand of the Kubernetes cluster in a future period by using a linear regression algorithm or a time series analysis algorithm to obtain a network traffic prediction result; A capacity adjustment module, configured to dynamically adjust the IP address pool capacity of the Kubernetes cluster according to the network traffic prediction result; A policy determination module, configured to determine the IP allocation policy of the IP address pool by using a particle swarm optimization algorithm or an ant colony algorithm.

[0057] In an exemplary embodiment, the device further comprises: A data analysis module, configured to obtain the data of the initial IP address pool of the Kubernetes cluster and analyze the data to obtain network traffic change data; An information adjustment module, configured to adjust the network bandwidth limit and packet priority of the Kubernetes cluster according to the network traffic change data.

[0058] The device in the device embodiment is based on the same inventive concept as the method embodiment.

[0059] An embodiment of this specification provides an electronic device, which includes a processor and a memory. At least one instruction or at least one program segment is stored in the memory, and the at least one instruction or at least one program segment is loaded and executed by the processor to implement the network traffic allocation method provided in the above method embodiment.

[0060] An embodiment of the present invention further provides a computer storage medium, which can be set in a terminal to store at least one instruction or at least one program segment related to implementing a network traffic allocation method in a method embodiment. The at least one instruction or at least one program segment is loaded and executed by the processor to implement the network traffic allocation method provided in the above method embodiment.

[0061] Embodiments of the present invention also provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes to implement the network traffic allocation method provided in the above method embodiments.

[0062] Optionally, in the embodiments of this specification, the storage medium may be located in at least one of multiple network servers in a computer network. Optionally, in this embodiment, the above storage medium may include but is not limited to: various media that can store program codes such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs.

[0063] The memory in the embodiments of this specification can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory may further include a memory controller to provide the processor with access to the memory.

[0064] The method embodiments of network traffic allocation provided in the embodiments of this specification can be executed on a mobile terminal, a computer terminal, a server, or a similar computing device. Taking running on a server as an example, Figure 7 is the hardware structure block diagram of a server for the network traffic allocation method provided in the embodiments of this specification. As Figure 7As shown, the server 700 can vary significantly due to differences in configuration or performance. It can include one or more central processing units (CPUs) 710 (the central processing unit 710 can include, but is not limited to, processing devices such as microprocessor MCUs or programmable logic devices FPGAs), a memory 730 for storing data, and one or more storage media 720 for storing application programs 723 or data 722 (such as one or more mass storage devices). Among them, the memory 730 and the storage media 720 can be transient storage or persistent storage. The programs stored in the storage media 720 can include one or more modules, and each module can include a series of instruction operations on the server. Further, the central processing unit 710 can be set to communicate with the storage media 720 and execute a series of instruction operations in the storage media 720 on the server 700. The server 700 can also include one or more power supplies 760, one or more wired or wireless network interfaces 750, one or more input / output interfaces 740, and / or one or more operating systems 721, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and so on.

[0065] The input / output interface 740 can be used to receive or send data via a network. Specific examples of the above network can include a wireless network provided by the communication provider of the server 700. In one example, the input / output interface 740 includes a network interface controller (NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the input / output interface 740 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0066] Those of ordinary skill in the art can understand that Figure 7 the structure shown is only schematic and does not limit the structure of the above electronic device. For example, the server 700 can also include more or fewer components than Figure 7 shown in Figure 7 or have a different configuration from

[0067] As can be seen from the embodiments of the network traffic allocation method, apparatus, electronic device, or storage medium provided by the present invention described above, the present invention sets monitoring metrics and data collection frequencies of a data monitoring tool; the monitoring metrics include the network traffic and resource metrics of a Kubernetes cluster, and the data collection frequencies include a first collection frequency corresponding to the network traffic and a second collection frequency corresponding to the resource metrics; the data monitoring tool is used to collect the node-level and container-level network traffic information in each Pod in the Kubernetes cluster according to the first collection frequency; the data monitoring tool is used to collect the resource metric data of each Pod in the Kubernetes cluster according to the second collection frequency; according to the network traffic information and the resource metric data, each Pod is clustered according to the network traffic and resource usage conditions to obtain at least one load balancing group; a load balancing strategy is used to adjust the load balancer of Kubernetes, and the network traffic is evenly distributed to each load balancing group. The present invention solves the problems of resource waste and network performance bottleneck caused by static IP allocation in a large-scale Kubernetes cluster.

[0068] It should be noted that: the above sequence of the embodiments of the present specification is only for description and does not represent the superiority or inferiority of the embodiments. And the above specific embodiments of the present specification have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0069] Each embodiment in this specification is described in a progressive manner, and the same or similar parts among the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the apparatus, device, and storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0070] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiments can be completed by hardware, or can be completed by a program instructing relevant hardware. The program can be stored in a computer storage medium, and the above-mentioned storage medium can be a read-only memory, a magnetic disk, an optical disk, etc.

[0071] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A network traffic distribution method, characterized in that: The method comprises: Set monitoring indicators and data collection frequency of the data monitoring tool; the monitoring indicators include network traffic and resource indicators of the Kubernetes cluster, and the data collection frequency includes a first collection frequency corresponding to the network traffic and a second collection frequency corresponding to the resource indicators; Using the data monitoring tool to collect node-level and container-level network traffic information in each Pod in the Kubernetes cluster at the first collection frequency; Using the data monitoring tool to collect resource indicator data of each Pod in the Kubernetes cluster according to the second collection frequency; According to the network traffic information and the resource indicator data, each Pod is clustered according to the network traffic and resource usage to obtain at least one load balancing group; Use load balancing strategies to adjust the Kubernetes load balancer to evenly distribute network traffic to each load balancing group.

2. The method according to claim 1, characterized in that: According to the network traffic information and the resource indicator data, each Pod is clustered according to the network traffic and resource usage to obtain at least one load balancing group, including: Using an anomaly detection algorithm to perform outlier identification processing on the network traffic information and the resource indicator data, and removing the outliers to obtain processed data; Normalizing the processed data to obtain normalized data; The K-Means algorithm is used to cluster each Pod according to the network traffic and resource usage in the normalized data to obtain at least one load balancing group.

3. The method according to claim 2, characterized in that The method further comprises: For the normalized data corresponding to each Pod, extract the network attribute characteristics, CPU usage, and memory usage in the normalized data; A dimensionality reduction algorithm is used to reduce the dimension of the network attribute characteristics, CPU usage, and memory usage to obtain the target comprehensive characteristics corresponding to each Pod; Determine the IP address corresponding to each Pod based on the comprehensive characteristics of the target corresponding to each Pod.

4. The method according to claim 3, characterized in that Determining the IP address corresponding to each Pod according to the comprehensive target characteristics corresponding to each Pod includes: A classification algorithm is used to classify the target comprehensive features corresponding to each Pod to obtain the target traffic category of each Pod; Determine the Pod whose target traffic category is a high traffic category as a high traffic Pod, and determine the Pod whose target traffic category is a low traffic category as a low traffic Pod; A genetic algorithm or a simulated annealing algorithm is used to allocate a first IP address to the high-traffic Pod, and a second IP address is allocated to the low-traffic Pod; the performance of the first IP address is better than the performance of the second IP address.

5. The method according to claim 1, characterized in that The method further comprises: Using a convolutional neural network or a recurrent neural network to predict the traffic of each load balancing group to obtain a prediction result; The load balancing strategy is dynamically adjusted according to the prediction result, and the traffic of each load balancing group is redistributed according to the adjusted load balancing strategy.

6. The method according to claim 1, characterized in that The method further comprises: Use a linear regression algorithm or a time series analysis algorithm to predict the network traffic demand of the Kubernetes cluster in a future period to obtain a network traffic prediction result; Dynamically adjust the IP address pool capacity of the Kubernetes cluster according to the network traffic prediction result; The IP allocation strategy of the IP address pool is determined by using a particle swarm optimization algorithm or an ant colony algorithm.

7. The method according to claim 6, characterized in that The method further comprises: Obtaining data of the initial IP address pool of the Kubernetes cluster, and analyzing the data to obtain network traffic change data; According to the network traffic change data, the network bandwidth limit and the data packet priority of the Kubernetes cluster are adjusted.

8. A network traffic distribution device, characterized in that: The device comprises: A data setting module is used to set monitoring indicators and data collection frequency of the data monitoring tool; the monitoring indicators include network traffic and resource indicators of the Kubernetes cluster, and the data collection frequency includes a first collection frequency corresponding to the network traffic and a second collection frequency corresponding to the resource indicators; An information collection module, used to collect node-level and container-level network traffic information in each Pod in the Kubernetes cluster using the data monitoring tool at the first collection frequency; A data collection module, used to collect resource indicator data of each Pod in the Kubernetes cluster using the data monitoring tool according to the second collection frequency; A clustering module, configured to cluster each Pod according to network traffic and resource usage based on the network traffic information and the resource indicator data, to obtain at least one load balancing group; The traffic distribution module is used to adjust the Kubernetes load balancer using a load balancing strategy to evenly distribute network traffic to each load balancing group.

9. An electronic device, characterized in that: The device comprises: a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the network traffic distribution method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that: The computer storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the network traffic distribution method as described in any one of claims 1-7.

Images