A comprehensive evaluation method for container network plug-ins

Abstract

The present application proposes a comprehensive evaluation method for container network plug-ins, the method comprising: performing a functional evaluation on the container network plug-in to be evaluated according to pre-established functional evaluation indicators to obtain a functional evaluation result; performing a performance evaluation on the container network plug-in to be evaluated according to pre-established performance evaluation indicators to obtain a performance evaluation result; and determining a comprehensive evaluation result according to the functional evaluation result and the performance evaluation result. The method provided in the present application proposes a comprehensive evaluation of the container network plug-in from two aspects: functional evaluation and performance evaluation, taking into account a variety of scenarios, and can provide comprehensive reference factors for the evaluation of the container network plug-in.

Description

Technical Field

[0001] The present application relates to the technical field of container network plug-ins, and in particular to a comprehensive evaluation method for container network plug-ins. Background Art

[0002] In recent years, in order to adapt to the frequent changes in software requirements caused by the changing business environment, microservice architecture has gradually become the mainstream architecture for enterprise-level software design. Under the microservice architecture, a single enterprise application is generally composed of hundreds to thousands of micro programs with single functions. These single-function programs complete complex business requirements through remote calls between each other. Using containers to encapsulate such fine-grained programs and managing their lifecycles with the help of Kubernetes clusters is a widely used microservice architecture practice. The containers that make up the complete application collaborate through network communication, and in the Kubernetes cluster, the communication between containers running on multiple servers is based on the Container Network Interface (CNI).

[0003] The CNI specification was proposed by the Cloud Native Computing Foundation (CNCF), to which Kubernetes belongs. CNCF specifies the minimum functions that network plug-ins need to have through CNI, but does not provide specific implementations. Therefore, the open source community provides a variety of network plug-ins that comply with the CNI specification. In addition to implementing the minimum set of functions specified by CNI, each plug-in also extends different functions, including but not limited to network policies, traffic encryption, etc. In addition, the performance of the plug-ins implemented by the community also competes in many aspects. Therefore, different user scenarios have their own most suitable plug-ins. Existing research on network plug-ins basically focuses on performance benchmarks in specific hardware environments, and has not proposed a network plug-in selection strategy that adapts to different scenarios.

[0004] In the past, some literature has studied plug-in performance. The commonality of these studies is that they have conducted relatively detailed performance tests on CNI network plug-ins in different hardware environments and software platforms, and provided references for the performance benchmarks of CNI plug-ins. However, these studies did not provide solutions on how to select the optimal plug-in based on test data, and the most in-depth research on plug-in selection has also stopped at qualitative reference suggestions.

[0005] In terms of performance benchmarking, some literature has tested the performance of multiple CNI network plug-ins on different bare metal servers, and some have tested related performance on VMware ESXi virtual platforms. The above studies focus on the performance of communication through the host kernel network protocol stack. D et al. focused on the performance of CNI network plug-ins working on two kernel bypass high-performance network hardware, RDMA (Remote Direct Memory Access) and DPDK (Intel Data Plane Development Kit). In addition to the above literature on performance testing, a few literatures have also focused on the impact of some kernel or network card configuration options on performance, including packet size, protocol type, network policy, and communication mode.

[0006] However, the above research results basically use all the space to describe the test process and test data. Although a small number of literatures focus on the functional differences beyond basic performance and the impact of cluster characteristics on performance, they still only provide qualitative suggestions on plug-in selection. Summary of the invention

[0007] The present application provides a comprehensive evaluation method for a container network plug-in, the method comprising:

[0008] According to the pre-established functional evaluation indicators, the functional evaluation of the container network plug-in to be evaluated is performed to obtain the functional evaluation results;

[0009] According to the pre-established performance evaluation indicators, the performance of the container network plug-in to be evaluated is evaluated to obtain a performance evaluation result;

[0010] A comprehensive evaluation result is determined based on the function evaluation result and the performance evaluation result.

[0011] Optionally, the functional evaluation indicators include traffic encryption function, network policy function inbound direction, network policy function outbound direction, MTU setting, ease of deployment, ease of maintenance and cluster adaptability indicators.

[0012] Optionally, a functional evaluation is performed on the container network plug-in to be evaluated according to pre-established functional evaluation indicators, and the functional evaluation results obtained include:

[0013] The traffic encryption function is denoted as a1; the network policy function inbound direction is denoted as a2; the network policy function outbound direction is denoted as a3; the MTU setting is denoted as a4; the deployment ease is denoted as a5; and the maintenance ease is denoted as a6; and the analytical method is used to quantify them into fixed values ​​in the range of [0,1];

[0014] The cluster adaptability index, denoted as a7, is determined using a predetermined function a7=f(scale);

[0015] The functional evaluation results are determined by the following method:

[0016]

[0017] In the formula, s func The function is evaluated as a result.

[0018] Optionally, the performance evaluation indicators include bandwidth, broadband memory usage and CPU usage;

[0019] Among them, any performance evaluation index has different performance in different communication protocols;

[0020] The communication protocols include the TCP protocol for communication between Pods, the UDP protocol for communication between Pods, the TCP protocol for communication between Pods and Services, and the UDP protocol for communication between Pods and Services.

[0021] Optionally, the performance of the container network plug-in to be evaluated is evaluated according to a pre-established performance evaluation index to obtain a performance evaluation result, including:

[0022] The performance evaluation results are determined by the following method:

[0023] s perf =w1·bandwidth-w2·memory-w3·processor

[0024] In the formula, s perf is the performance evaluation result; bandwidth is the broadband score; memory is the broadband memory occupancy score; processor is the CPU occupancy score; w1 is the weight of the broadband, w2 is the weight of the broadband memory occupancy, and w3 is the weight of the CPU occupancy;

[0025] The corresponding relationship between the performance evaluation results and different communication protocols is:

[0026]

[0027] In the formula, s perf is the performance evaluation result; p is the communication ratio between Pods; Score the communication between Pods; Score the communication between Pod and Service;

[0028] The score of any one of the communication between the Pods or the communication between the Pods and the Service is determined by the following method:

[0029]

[0030] In the formula, is the score of any one of the communication between the Pods or between the Pods and the Service; f is the proportion of the UDP protocol; Score for UDP protocol; Score the TCP protocol;

[0031] In summary, the performance evaluation results are determined by the following method:

[0032]

[0033] In the formula, s perf is the performance evaluation result.

[0034] Optionally, determining a comprehensive evaluation result according to the function evaluation result and the performance evaluation result includes:

[0035] s=s func ·(s perf +1)*50,s∈[0,100]

[0036] Wherein, s is the comprehensive evaluation result; s func is the functional evaluation result; perf is the performance evaluation result.

[0037] Optionally, the communication protocol includes: a communication protocol in an encrypted state and a communication protocol in a non-encrypted state.

[0038] The method provided in this application proposes a comprehensive evaluation of the container network plug-in from two aspects: functional evaluation and performance evaluation, taking into account multiple scenarios, and can provide comprehensive reference factors for the evaluation of the container network plug-in. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of a comprehensive evaluation method for a container network plug-in applicable to an embodiment of the present application is provided;

[0040] Figure 2 A schematic diagram of a cluster adaptability indicator result provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of a Pod to Pod communication scenario evaluation provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of a Pod to Service communication scenario evaluation provided in an embodiment of the present application;

[0043] Figure 5 A communication performance evaluation result in an encryption scenario provided in an embodiment of the present application;

[0044] Figure 6 A schematic diagram of a comprehensive evaluation result of a container network plug-in provided in an embodiment of the present application;

[0045] Figure 7 A schematic diagram of a scenario verification result provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0047] Please refer to Figure 1 , which exemplarily shows a flow chart of a comprehensive evaluation method for a container network plug-in applicable to an embodiment of the present application.

[0048] It should be noted that before executing the method provided in the embodiment of the present application, it is necessary to set the working environment required by the embodiment of the present application. A feasible setting is that the embodiment of the present application is carried out in a small data center, and a cluster is formed on 50 ultra-micro servers connected by an H3C 10Gbit switch. The server is directly connected to the switch via SFP optical fiber and is set in the same VLAN, and a giant frame (MTU 9000) is configured from the switch. In terms of software, Kubernetes 1.19.0 was deployed on Ubuntu 18.04. Docker is installed from the official source of apt, docker.io. After each plug-in is tested, the entire cluster is reset, and then the cluster and CNI plug-in are redeployed. Each CNI plug-in is tested 3 times using different node combinations, and the average value is retained.

[0049] The method provided in this application comprises the following steps:

[0050] Step S101: perform a functional evaluation on the container network plug-in to be evaluated according to pre-established functional evaluation indicators to obtain a functional evaluation result.

[0051] In the embodiment of the present application, the functional evaluation indicators include traffic encryption function, network policy function inbound direction, network policy function outbound direction, MTU setting, ease of deployment, ease of maintenance, and cluster adaptability indicators.

[0052] Specifically, the traffic encryption function is denoted as a1; the network policy function inbound direction is denoted as a2; the network policy function outbound direction is denoted as a3; the MTU setting is denoted as a4; the ease of deployment is denoted as a5; and the ease of maintenance is denoted as a6; and the analytical method is used to quantify them into fixed values ​​in the range of [0,1].

[0053] Among them, the traffic encryption function, the network policy function inbound direction, the network policy function outbound direction, and the MTU setting function are quantified. If the function exists, the corresponding variable is 1, otherwise it is 0, as shown in Table 1. If the user scenario requires one of the three functions, the score of the plug-in without the function will be 0, and the user can be directly excluded from the selection.

[0054] The ease of deployment and maintenance mainly focus on the configuration complexity and operation complexity of the deployment, which are quantified from complex to simple as a5∈[0,1]. The number of deployment steps and the number of configuration items are actually tested and recorded. After actual testing, it is found that Canal and Flannel are the easiest to deploy. After downloading the official deployment file, a single command can be executed, and the deployment simplicity is quantified as 1; Calico and Weave deployment processes also require only a few steps, but since these two plug-ins have many configuration parameters, they need to be understood and reasonably configured during deployment, so the deployment complexity is quantified as a medium value of 0.95; Kube-Router and Cilium have complex deployment processes, including multiple components, many configuration parameters, and many commands to be executed during deployment. In addition, Kube-Router currently does not support MTU settings, so its deployment simplicity is quantified as 0.9. The deployment complexity quantification scores are shown in Table 1.

[0055] Maintenance difficulty mainly focuses on the complexity of modifying configuration during operation and the difficulty of troubleshooting after errors, which is quantified from simple to complex as a6∈[0,1]. From the perspective of configuration modification, the three network plug-ins Calico, Cilium and Weave Net have many configuration parameters and are relatively complex, while the other three plug-ins are relatively simple; from the perspective of error troubleshooting difficulty, plug-ins that use encapsulation technologies such as VXLAN are difficult to locate after errors, so they are scored lower, while plug-ins that are directly forwarded through the host network stack score higher. However, maintenance difficulty, as a subjective evaluation related to the ability of the maintenance team, should not affect the overall score too much. Therefore, when scoring, the initial score is full, and if the configuration modification or error location is complex, 0.05 points will be subtracted. The quantitative score of maintenance difficulty is shown in Table 1.

[0056] Table 1: Functional evaluation results (a1-a6)

[0057] Calico Canal Cilium Flannel Kube-Router WeaveNet <![CDATA[a1]]> 1 1 1 1 0 1 <![CDATA[a2]]> 1 0 1 0 0 1 <![CDATA[a3]]> 1 1 1 0 1 1 <![CDATA[a4]]> 1 1 1 0 1 1 <![CDATA[a5]]> 0.95 1.00 0.90 1.00 0.90 0.95 <![CDATA[a6]]> 0.990 0.995 0.995 0.995 1.000 0.990

[0058] The cluster adaptability index, denoted as a7, is determined using a predetermined function a7=f(scale).

[0059] The cluster adaptability index is the adaptability of large-scale clusters. It focuses on the ability of the container network plug-in to manage communication rules and synchronize Pod routing information when the cluster size continues to increase. This is closely related to the architecture of the plug-in. Focus on the adaptability score from 5 nodes to 1000 nodes, but the general experimental environment only allows testing of up to 50 nodes. Therefore, first theoretically analyze the functional form of the correlation between the cluster adaptability index and the cluster size, then add 5 nodes to each of the 50 nodes for actual testing, and finally fit according to the theoretical function form to obtain the correlation function between the cluster adaptability index and the cluster size for subsequent evaluation.

[0060] From a theoretical analysis, Flannel has the worst adaptability to large-scale clusters because its communication rules are related to the number of Pods. When there are many nodes communicating with each other, the growth complexity of synchronization information is at least O(n2). Therefore, a quadratic polynomial is used to fit the score curve.

[0061] Take a set of typical values ​​for the cluster scale, test each plug-in at each typical value, deploy 10k Nginx services on a cluster of k nodes, of which 5 Nginx form a Service, and then use a high-performance server to poll these 2k Services and count the number of processing times per second. In order to avoid negative scores when expanding to 500 nodes for evaluation, the worst-scoring Weave Net is tested to map the data of 50 nodes to 0.97, and the minimum value of the curve fitted to 500 nodes is 0. The fitted data is shown in Table 2, and then polynomial fitting is used to fit 5-50 nodes and 1-500 nodes. Figure 2 , which is a result schematic diagram of a cluster adaptability indicator provided in an embodiment of the present application.

[0062] Table 2: Functional evaluation results (a7)

[0063]

[0064] The functional evaluation results were determined using the following methods:

[0065]

[0066] In formula (1), s func Functional evaluation results.

[0067] Step S102: Perform performance evaluation on the container network plug-in to be evaluated according to the pre-established performance evaluation index to obtain a performance evaluation result.

[0068] Performance evaluation indicators include bandwidth, bandwidth memory usage, and CPU usage;

[0069] It should be noted that any performance evaluation indicator has different performance manifestations in different communication protocols.

[0070] The communication protocols include TCP protocol for communication between Pods, UDP protocol for communication between Pods, TCP protocol for communication between Pods and Services, and UDP protocol for communication between Pods and Services.

[0071] The communication protocol includes: a communication protocol in an encrypted state and a communication protocol in a non-encrypted state.

[0072] That is, there are eight corresponding situations corresponding to any performance evaluation index. For the convenience of description, the embodiments of the present application select some situations for introduction, and the method for determining the performance evaluation results in different situations is the same.

[0073] The performance evaluation results are determined using the following method:

[0074] s perf =w1·bandwidth-w2·memory-w3·processor (2)

[0075] In formula (2), s perf is the performance evaluation result; bandwidth is the broadband score; memory is the broadband memory occupancy score; processor is the CPU occupancy score; w1 is the weight of broadband, w2 is the weight of broadband memory occupancy, and w3 is the weight of CPU occupancy.

[0076] The corresponding relationship between the performance evaluation results and different communication protocols is:

[0077]

[0078] In formula (3), s perf is the performance evaluation result; p is the communication ratio between Pods; Score the communication between Pods; Score for communication between Pod and Service.

[0079] The score of any one of the communication between Pods or between Pods and Services is determined by the following method:

[0080]

[0081] In formula (4), It is the score of any one of the communication between Pods or between Pods and Services; f is the proportion of UDP protocol; Score for UDP protocol; Score for TCP protocol.

[0082] In summary, the performance evaluation results are determined by the following method:

[0083]

[0084] In formula (5), s perf The performance evaluation results are shown in Figure 2.

[0085] The following is an explanation with specific examples.

[0086] First, multiple scenarios are set, and three parameters w1, w2, and w3 are used to describe the importance of bandwidth, memory usage, and CPU usage in different user scenarios. When w1=1, it means that the bandwidth is currently very important, and wl=0 means that the bandwidth is not currently important. The plug-in with the worst bandwidth also meets the application requirements. w2 and w3 have similar meanings. When its value is 0, it means that the memory / CPU resource usage is not important at all, and when its value is 0.5, it means that the scenario is very sensitive to resource usage. Therefore, the six user scenarios shown in Table 4 are preset, and the performance scores are analyzed based on these scenarios.

[0087] Table 3 Typical user scenario comparison table

[0088]

[0089] The performance evaluation results of Pod-to-Pod communication in a non-encrypted scenario are taken as an example for analysis. As shown in Table 4, the performance data of the plug-in when Pod-to-Pod communicates via TCP. In terms of bandwidth, the bandwidth of the bare metal server reaches 9906Mbit / s. As a reference, except for Kuber-Router, the bandwidth loss of the other plug-ins is less than 5%, and the bandwidth loss of Calico and Canal is less than 100Mbit / s, which is very good. In contrast, the bandwidth of Kube-Router is only 8051Mbit / s, and the performance loss is as high as 18.7%. In terms of resource usage, the bare metal server has the lowest resource usage, with 390MB of memory and 3.1% of CPU usage. Among the six network plug-ins, except for Cilium, which uses 880M of memory, the memory usage of the other five network plug-ins is no more than 650MB. In terms of CPU usage, Cilium and Weave Net use more CPU resources, reaching 9.8% and 8.3% respectively.

[0090] Table 4: Plugin performance data for non-encrypted Pod-to-Pod communication via TCP

[0091] Plugins Bandwidth (Mbit / s) Memory usage (MB) CPU usage (%) Bare metal 9906 390 3.1 Calico 9834 648 5.9 Canal 9817 647 5.2 Cilium 9416 880 9.8 Flannel 9690 590 5.1 Kube-Router 8051 600 6.0 Weave Net 9759 650 8.3

[0092] In the non-encrypted case, Pod to Pod is transmitted via UDP protocol. Bandwidth is also used to describe performance, and CPU usage and memory usage are used to describe resource usage. The test data is shown in Table 5. The results show that the performance of Kube-Router is very low, only 2818Mbit / s. The performance loss is too large and cannot be accepted by users who pay attention to performance. Except for Kube-Router, the bandwidth of other plug-ins is greater than 9000Mbit / s, and except for Weave Net, the bandwidth of other plug-ins is above 9700Mbit / s. In terms of memory usage, Cilium performs poorly, reaching 870MB, while the memory usage of other plug-ins is between 600MB-700MB, which is basically the same. In terms of CPU usage, Cilium has the highest CPU usage, followed by Kube-Router and Weave Net. The CPU usage of Calico, Canal and Flannel does not exceed 10%, which is excellent.

[0093] Table 5: Plugin performance data for non-encrypted Pod-to-Pod communication via UDP

[0094] Plugins Bandwidth (Mbit / s) Memory usage (MB) CPU usage (%) Bare metal 9885 390 5.5 Calico 9757 670 9.9 Canal 9742 676 9.6 Cilium 9726 870 13.3 Flannel 9846 600 9.1 Kube-Router 2818 600 11.7 Weave Net 9393 663 11.9

[0095] From Table 4 and Table 5, we can see that the performance of each plug-in under TCP and UDP protocols is not completely consistent. Therefore, we first study the impact of the communication ratio of TCP and UDP protocols on the plug-in score. When the communication ratio p of TCP and UDP protocols changes, the scores of each plug-in are plotted according to formula (4) in the Pod to Pod communication scenario. Figure 3 As shown, it is a schematic diagram of a Pod to Pod communication scenario evaluation situation provided in an embodiment of the present application.

[0096] according to Figure 3 We can preliminarily draw the following conclusions about the performance differences of different container network plug-ins in different scenarios:

[0097] In scenario 1, performance is paramount and resources are not limited. When p changes, Calico and Canal always have the best performance, and the difference between the two is very small.

[0098] In scenario 2, performance is paramount, memory is unlimited, and CPU is limited. When p < 0.49, Flannel is the best, while when p > 0.49, Canal is the best.

[0099] Scenario 3: Performance first, memory limited, CPU unlimited, Flannel is always the best.

[0100] Scenario 4: Performance is of great concern and resources are slightly limited. Calico and Canal are basically the same. When p < 0.49, Flannel is the best, while when p > 0.49, Calico and Canal are the best.

[0101] Scenario 5: Performance is a little important, resources are limited, and Flannel performs best.

[0102] Scenario 6: Performance is not a concern and resources are very limited. Flannel performs best.

[0103] The performance of communication between Pod and Service is different from that between Pod and Pod. When Pod and Service communicate, because they will pass through ClusterIP service, the bandwidth peak and resource usage will be different. In addition, the communication process between Pod and Service is relatively more in line with the normal situation. The analysis process still uses actual measurement as the evaluation standard, and the data analysis method is similar to the performance analysis of Pod and Pod communication.

[0104] In the non-encrypted scenario, for the convenience of analysis, we fixed the Pod-to-Pod communication ratio p = 0.5, and then changed the TCP protocol communication ratio f. According to formula (3), we plotted Figure 4 The scores shown. Figure 4 As shown, it is a schematic diagram of a Pod to Service communication scenario evaluation provided by an embodiment of the present application. Figure 4 It can be found that when p=0.5, except for scenario 1, where Calico, Canal, and Flannel plugins competed with each other, Flannel scored the highest in all other scenarios. The scores under other p values ​​can also be analyzed in the same way.

[0105] In the encryption scenario, we first analyzed the feasibility of each plug-in in the encryption scenario and tested the network bandwidth when the encryption option was turned on, as shown in Table 6. The performance of the three plug-ins is very different. Calico TCP bandwidth and UDP bandwidth are far ahead of Cilium and Weave Net, and the former is about 5 times that of the latter two. It can be found that when Calico is encrypted, the TCP peak bandwidth loss is not large.

[0106] Table 6 Peak bandwidth of plug-ins in encryption scenario (Mbit / s)

[0107] Calico Cilium Weave Net Pod to Pod TCP 9786 1386 1384 Pod to Pod UDP 4877 869 432 Pod to Service TCP 9808 1661 1381 Pod to Service UDP 4872 866 451

[0108] Then pay attention to resource usage. As shown in the data in Table 7, when encryption is turned on, the memory usage of the three plug-ins is not much different, all within an acceptable range. However, the difference in CPU usage is quite obvious. Calico has a very high CPU usage rate under TCP and UDP protocols, reaching 37.6% and 32.4% respectively, which is much higher than the other two plug-ins. It can be inferred that Calico uses a high-performance encryption algorithm and has achieved performance far exceeding other plug-ins based on huge computing power requirements. However, this high bandwidth performance obtained with high CPU usage may not be suitable for all user situations.

[0109] Table 7 Resource usage in encryption scenario

[0110]

[0111] From the above analysis, we can conclude that in the encryption scenario, only three plug-ins meet the requirements, and the same is true for the six scenarios analyzed above. Figure 5 As shown in FIG. 1 , a communication performance evaluation result in an encryption scenario provided by an embodiment of the present application is shown. Figure 5 It can be seen that in the first four scenarios where performance is of great concern, Calico scores much higher than the other two plug-ins, which reflects that Calico is far ahead of other plug-ins in terms of performance. In the last two scenarios where resources are more limited and performance is not of great concern, it can be seen that Cilium scores the highest, while Calico scores lower due to excessive resource usage.

[0112] Step S103, determining a comprehensive evaluation result according to the function evaluation result and the performance evaluation result.

[0113] Specifically, the comprehensive evaluation results in the examples of this application are determined using the following method:

[0114] s=s func ·(s perf +1)*50,s∈[0,100] (6)

[0115] In formula (6), s is the comprehensive evaluation result; s func is the result of functional evaluation; perf The performance evaluation results are shown in Figure 2.

[0116] Among them, the functional evaluation results func ∈[0,1], performance evaluation result s perf ∈[-1,1], in order to facilitate comprehensive analysis, the performance score is remapped to [0,2], and then multiplied by the functional evaluation score and amplified by 50 times to obtain a final score s. The container network plug-in with the highest score s is the most suitable one.

[0117] Comprehensive analysis The various functional quantification results and performance quantification results in the previous article are comprehensively analyzed, and the comprehensive scores of each plug-in are calculated based on formula 6. When calculating sp2p and sp2s, it is assumed that TCP accounts for 80%, and then the proportion of Pod-to-Pod communication is specified as f=0.2. The cluster size is used as the independent variable, and the comprehensive score is calculated based on the above formula. The score curve is drawn as shown below: Figure 6 , which is a schematic diagram of a comprehensive evaluation result of a container network plug-in provided in an embodiment of the present application.

[0118] Analyze one by one Figure 6 The first two subgraphs preset the need to set MTU, encryption, and network policy. The subgraph "Encryption, Scenario 1" presets the user scenario as Scenario 1, that is, performance first, and unlimited resources. In this case, due to the excellent encryption performance of Calico, the score is always the highest. Since WeaveNet has slightly better performance, WeaveNet is second best when the cluster scale is small, but when the cluster scale expands, Cilium has better adaptability to large-scale clusters, and Cilium surpasses Weave Net to become second best. In the subgraph "Encryption, Scenario 6", the preset user scenario is not concerned with performance, and resources are very limited. At this time, it can be found that Weave Net has become the best choice due to its low resource usage. The last four sub-graphs are preset to not pay attention to MTU settings, encryption, and network policies, and then four different user scenarios are introduced respectively. It can be found that when performance is paramount and resources are unlimited (sub-graph "Non-encryption, Scenario 1") and performance is very important and resources are slightly limited (sub-graph "Non-encryption, Scenario 4"), small-scale clusters are all Flannel with simple configuration and maintenance and good performance. The best score, and when the cluster scale becomes larger, Calico becomes the best choice due to its good adaptability to large-scale clusters. When performance is slightly concerned and resources are relatively limited (sub-graph "Non-encryption, Scenario 5") and performance is not concerned and resources are completely limited (sub-graph "Non-encryption, Scenario 6"), Flannel becomes the best choice when the cluster is small due to its simple deployment and maintenance and low resource usage. When the cluster scale increases, Calico becomes the best choice, and the more limited the resources, the wider the cluster scale covered by Flannel as the best choice.

[0119] Figure 6 It reflects the correlation between the evaluation of the six plug-ins and the cluster size in six typical user scenarios, in a 10G interconnected data center environment, with the preset TCP and UDP communication ratios, and the preset Pod to Pod and Pod to Service communication ratios. Based on the evaluation, users can quickly find the most suitable plug-in according to their own scenarios.

[0120] In order to illustrate the feasibility of this application, four typical scenarios are selected for verification: Verification scenario 1: The user cluster has about 10 nodes, traffic encryption is required, and performance is required to be paramount; Verification scenario 2: The user cluster has about 20 nodes, traffic encryption is required, but only 1000MB of memory and 10% of CPU resources can be reserved for the plug-in; Verification scenario 3: The user cluster has about 50 nodes, the user needs network policy, no encryption is required, and performance is paramount; Verification scenario 4: The user cluster has about 100 nodes, and performance is very limited.

[0121] First, determine the score ranking for each verification scenario, and then determine the verification strategy. For performance-first verification scenarios, the verification indicator is the peak bandwidth. For scenarios with limited resources and given limits, verify the performance of all plug-ins after limiting resource usage. For situations where performance is not a concern and resources are very limited, test all plug-ins with the peak bandwidth of the plug-in with the worst bandwidth, and then verify resource usage.

[0122] The scores drawn based on the above scenarios and strategies are as follows Figure 7 As shown. Figure 7 As shown in Figure 1, it is a schematic diagram of a scenario verification result provided by an embodiment of the present application. For test scenario 1, in a cluster of 10 nodes, the comprehensive peak bandwidth of three types of traffic encryption (TCP and UDP ratio is fixed, Pod to Pod and Pod to Service ratio is fixed), it can be seen that Calico's comprehensive bandwidth is much larger than the other two plug-ins, and the ranking is in line with Figure 7 For test scenario 2, in a cluster of 20 nodes, traffic is encrypted, and then the resource limit function of Kubernetes is used to limit the resource usage of the three plug-ins to 1000MB of memory and the CPU utilization rate to no more than 10%. The resource limit causes a significant drop in Calico performance, and the resulting comprehensive bandwidth ranking is in line with Figure 6 For test scenario 3, the comprehensive bandwidth of the five plug-ins with network policy functions also meets the scores in 5. For scenario 4, the Iperf3 tool was used to simulate the peak bandwidth of the worst-performing Kube-Router, and the six plug-ins were tested. The CPU and memory usage were quantified using formula 3 and remapped to 0-1, and the bandwidth usage was also consistent with the situation in 6. The actual test rankings of the four scenarios are consistent with the method provided in this application.

[0123] The method provided in this application proposes a comprehensive evaluation of the container network plug-in from two aspects: functional evaluation and performance evaluation, taking into account multiple scenarios, and can provide comprehensive reference factors for the evaluation of the container network plug-in.

[0124] The above-described embodiments of the present application do not constitute a limitation on the protection scope of the present application.

Claims

1. A comprehensive evaluation method for container network plug-ins, characterized in that: The method comprises: According to the pre-established functional evaluation indicators, the functional evaluation of the container network plug-in to be evaluated is performed to obtain the functional evaluation results; According to the pre-established performance evaluation indicators, the performance of the container network plug-in to be evaluated is evaluated to obtain a performance evaluation result; Determining a comprehensive evaluation result according to the function evaluation result and the performance evaluation result; According to the pre-established functional evaluation indicators, the functional evaluation of the container network plug-in to be evaluated is performed, and the functional evaluation results obtained include: The traffic encryption function is denoted as a1; the network policy function inbound direction is denoted as a2; the network policy function outbound direction is denoted as a3; the MTU setting is denoted as a4; the ease of deployment is denoted as a5; and the ease of maintenance is denoted as a6. The analytical method is used to quantify them into fixed values ​​in the range of [0,1]. The cluster adaptability index, denoted as a7, is determined using a predetermined function a7 = f(scale); The functional evaluation results are determined by the following method: In the formula, s func evaluating the results of the functions; According to the pre-established performance evaluation indicators, the performance of the container network plug-in to be evaluated is evaluated to obtain the performance evaluation results, including: The performance evaluation results are determined by the following method: s perf =w1·bandwidth-w2·memory-w3·processor In the formula, s perf is the performance evaluation result; bandwidth is the broadband score; memory is the broadband memory occupancy score; processor is the CPU occupancy score; w1 is the weight of the broadband, w2 is the weight of the broadband memory occupancy, and w3 is the weight of the CPU occupancy; The corresponding relationship between the performance evaluation results and different communication protocols is: In the formula, s perf is the performance evaluation result; p is the communication ratio between Pods; Score the communication between Pods; Score the communication between Pod and Service; The score of any one of the communication between the Pods or the communication between the Pods and the Service is determined by the following method: In the formula, is the score of any one of the communication between the Pods or between the Pods and the Service; f is the proportion of UDP protocol; Score the TCP protocol; Score for UDP protocol; In summary, the performance evaluation results are determined by the following method: In the formula, s perf is the performance evaluation result.

2. The method according to claim 1, characterized in that The functional evaluation indicators include traffic encryption function, network policy function inbound direction, network policy function outbound direction, MTU setting, deployment simplicity, maintenance simplicity and cluster adaptability indicators.

3. The method according to claim 1, characterized in that The performance evaluation indicators include bandwidth, bandwidth memory usage and CPU usage; Among them, any performance evaluation index has different performance in different communication protocols; The communication protocols include the TCP protocol for communication between Pods, the UDP protocol for communication between Pods, the TCP protocol for communication between Pods and Services, and the UDP protocol for communication between Pods and Services.

4. The method according to claim 1, characterized in that: Determine a comprehensive evaluation result based on the function evaluation result and the performance evaluation result, including: s=s func ·(s perf +1)*50,s∈[0,100] Wherein, s is the comprehensive evaluation result; s func is the functional evaluation result; perf is the performance evaluation result.

5. The method according to claim 1, characterized in that: The communication protocol includes: a communication protocol in an encrypted state and a communication protocol in a non-encrypted state.

Images