Elastic capacity expanding and shrinking method and device for container, electronic equipment and storage medium

Abstract

The invention provides an elastic capacity expansion and contraction method and device for a container, electronic equipment and a storage medium, and belongs to the technical field of cloud computing. In the elastic capacity expansion and contraction method for the container, the influence of the network transmission rate and the network packet loss rate on capacity expansion and contraction of the container is considered; according to the method, the corresponding container can be subjected to capacity expansion and shrinkage according to the network transmission rate and the network packet loss rate monitored in real time and the preset capacity expansion and shrinkage strategy, namely, dynamic adjustment of capacity expansion and shrinkage of the container is achieved, capacity expansion and shrinkage are more reasonable, operation and maintenance personnel do not need to manually intervene or set complex automatic scripts, and the operation efficiency is improved. In other words, the complexity of operation and maintenance management is simplified through the automatic dynamic adjustment process.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing, and in particular to a method, device, electronic device and storage medium for elastically scaling a container. Background Art

[0002] With the rapid development of cloud computing, big data and microservice architecture, container technology (such as Docker) has become the mainstream way to build and deploy applications due to its lightweight, portability and efficient resource utilization. In cloud environments, dynamic scaling of containers is one of the key technologies to ensure application performance, optimize resource utilization and reduce operating costs. However, traditional container scaling methods are often based on the utilization of resources such as CPU and memory.

[0003] The above container scaling methods only rely on the utilization rate of resources such as CPU and memory. This results in that in some scenarios, even if CPU and memory resources are sufficient, application performance may degrade due to network bottlenecks, and traditional methods cannot respond to such changes in a timely manner.

[0004] Traditional expansion and contraction strategies usually make decisions based on preset thresholds or fixed rules, and lack the ability to dynamically adjust according to real-time network conditions and application needs. This may result in the inability to effectively expand or reduce resources in a timely manner when the network fluctuates or when sudden traffic increases;

[0005] Traditional methods often lead to irrational resource allocation due to the inability to accurately predict and respond to changes in network transmission rates. On the one hand, when the network load is low, too many resources are allocated to containers, resulting in resource waste; on the other hand, when the network load is high, insufficient resources are allocated, resulting in degraded application performance.

[0006] Traditional scaling methods require manual intervention by operation and maintenance personnel or the setting of complex automation scripts to adapt to different application scenarios and requirements. This not only increases operation and maintenance costs, but also reduces system stability and reliability.

[0007] In summary, traditional container scaling methods do not consider the impact of network transmission rate on containers, unreasonable scaling, and high O&M complexity. Summary of the invention

[0008] In view of this, the purpose of the present invention is to provide a method, device, electronic device and storage medium for elastically scaling containers, so as to alleviate the technical problems that traditional container scaling methods do not consider the impact of network transmission rate on containers, unreasonable scaling and high operation and maintenance complexity.

[0009] In a first aspect, an embodiment of the present invention provides a method for elastically expanding and contracting a container, comprising:

[0010] Dockerfile packages the executable program compiled by the user and the base image that the executable program depends on into a Docker image for network performance indicator collection, wherein the executable program is used to obtain network performance indicator data, and then encapsulates it in the Prometheus indicator format and exposes it through the HTTP protocol;

[0011] Kubernetes obtains the Docker image, and starts the Docker image according to the Deployment configuration file to obtain a network performance indicator collection container, wherein the Deployment configuration file is provided with a startup command for the Docker image;

[0012] Prometheus collects the target network performance indicator data of the corresponding container from the network performance indicator data emitted by the network performance indicator collection container according to the configured PodMonitor resource object, and calculates the key network performance indicators according to the target network performance indicator data, wherein the key network performance indicators include: network transmission rate and network packet loss rate;

[0013] HPA obtains the key network performance indicators collected by Prometheus through Prometheus Adapter, and scales the corresponding container according to the key network performance indicators and preset key network performance indicator thresholds according to the preset scaling strategy.

[0014] Furthermore, the executable program is developed and compiled using the GO language, and the executable program is used to monitor the / proc / net / dev file of the Linux system in real time, and parse the / proc / net / dev file to obtain the network performance indicator data, and then encapsulate the network performance indicator data according to the Prometheus indicator format and expose it through the HTTP protocol;

[0015] The base image is an open source base image obtained from a warehouse.

[0016] Furthermore, the parameters in the PodMonitor resource object include: a port, a path, and a crawling frequency for data collection.

[0017] Furthermore, the network performance indicator data includes: the total number of bytes sent, the total number of data packets sent and the total number of packet losses.

[0018] Furthermore, key network performance indicators are calculated based on the target network performance indicator data, including:

[0019] The network transmission rate is calculated according to the network transmission rate calculation formula: network transmission rate = total number of bytes sent at the current moment - total number of bytes sent at the previous moment, wherein the time interval between the current moment and the previous moment is 1 minute;

[0020] The network packet loss rate is calculated according to the network packet loss rate calculation formula: Network packet loss rate = (total number of packet losses per unit time / total number of sent data packets per unit time)*100, wherein the unit time is 1 minute.

[0021] Further, scaling the corresponding container according to the key network performance indicator and the preset key network performance indicator threshold according to the preset scaling strategy includes:

[0022] If the network transmission rate for a preset number of consecutive monitoring cycles is lower than a fixed ratio of the network transmission rate threshold, the number of corresponding containers is expanded to a target number, wherein the target number=(current network transmission rate / the network transmission rate threshold)*current number of containers.

[0023] Further, scaling the corresponding container according to the key network performance indicator and the preset key network performance indicator threshold according to the preset scaling strategy includes:

[0024] If the network transmission rate for a preset number of consecutive monitoring cycles is higher than a fixed ratio of the network transmission rate threshold, the number of corresponding containers is reduced to a first target number, wherein the first target number = (current network transmission rate / the network transmission rate threshold)*the current number of containers;

[0025] If the network packet loss rate for a preset number of consecutive monitoring periods is higher than the network packet loss rate threshold, the number of corresponding containers is reduced to a second target number, wherein the second target number = (current network packet loss rate / the network packet loss rate threshold)*the current number of containers;

[0026] A target quantity of a corresponding container is determined according to the first target quantity and / or the second target quantity.

[0027] In a second aspect, an embodiment of the present invention further provides a container elastic expansion and contraction device, comprising:

[0028] A packaging unit is used to package the executable program compiled by the user and the base image that the executable program depends on into a Docker image for network performance indicator collection through Dockerfile, wherein the executable program is used to obtain network performance indicator data, and then encapsulates it according to the Prometheus indicator format and exposes it through the HTTP protocol;

[0029] A startup unit, used for Kubernetes to obtain the Docker image, and to start the Docker image according to a Deployment configuration file to obtain a network performance indicator collection container, wherein the Deployment configuration file is provided with a startup command for the Docker image;

[0030] A collection and calculation unit, which is used for Prometheus to collect target network performance indicator data of the corresponding container from the network performance indicator data emitted by the network performance indicator collection container according to the configured PodMonitor resource object, and calculate key network performance indicators according to the target network performance indicator data, wherein the key network performance indicators include: network transmission rate and network packet loss rate;

[0031] The expansion and contraction unit is used for HPA to obtain the key network performance indicators collected by Prometheus through Prometheus Adapter, and to expand and contract the corresponding containers according to the key network performance indicators and the preset key network performance indicator thresholds according to the preset expansion and contraction strategies.

[0032] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0033] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to execute any method described in the first aspect above.

[0034] In an embodiment of the present invention, a method for elastically scaling a container is provided, comprising: Dockerfile packages a user-compiled executable program and a basic image on which the executable program depends into a Docker image for network performance indicator collection, wherein the executable program is used to obtain network performance indicator data, and is encapsulated in a Prometheus indicator format and then exposed through an HTTP protocol; Kubernetes obtains the Docker image, and starts the Docker image according to a Deployment configuration file to obtain a network performance indicator collection container, wherein a startup command for the Docker image is set in the Deployment configuration file; Prometheus collects target network performance indicator data of a corresponding container from the network performance indicator data emitted by the network performance indicator collection container according to a configured PodMonitor resource object, and calculates key network performance indicators according to the target network performance indicator data, wherein the key network performance indicators include: network transmission rate and network packet loss rate; HPA obtains the key network performance indicators collected by Prometheus through a Prometheus Adapter, and scales the corresponding container according to a preset scaling strategy according to the key network performance indicators and preset key network performance indicator thresholds. From the above description, it can be seen that the elastic scaling method of containers of the present invention takes into account the influence of network transmission rate and network packet loss rate on container scaling. The corresponding container can be scaled according to the preset scaling strategy based on the real-time monitored network transmission rate and network packet loss rate, thereby realizing dynamic adjustment of container scaling, making the scaling more reasonable and eliminating the need for manual intervention by operation and maintenance personnel or setting up complex automation scripts. The process of automated dynamic adjustment simplifies the complexity of operation and maintenance management, and alleviates the technical problems of traditional container scaling methods that do not consider the influence of network transmission rate on containers, unreasonable scaling, and high operation and maintenance complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 A flow chart of a method for elastically expanding and shrinking a container provided by an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of a container elastic expansion and contraction device provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Traditional container scaling methods do not consider the impact of network transmission rate on containers, the scaling is unreasonable, and the operation and maintenance complexity is high.

[0041] Based on this, the container elastic scaling method of the present invention takes into account the impact of network transmission rate and network packet loss rate on container scaling. The corresponding container can be scaled up or down according to the preset scaling strategy based on the real-time monitored network transmission rate and network packet loss rate, thereby realizing dynamic adjustment of container scaling, making the scaling more reasonable and eliminating the need for manual intervention by operation and maintenance personnel or setting up complex automation scripts. The process of automated dynamic adjustment simplifies the complexity of operation and maintenance management.

[0042] To facilitate understanding of this embodiment, a method for elastic expansion and contraction of a container disclosed in an embodiment of the present invention is first introduced in detail.

[0043] Embodiment 1:

[0044] According to an embodiment of the present invention, an embodiment of a method for elastically scaling up and down a container is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] Figure 1 is a flow chart of a method for elastically scaling a container according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0046] Step S102, Dockerfile packages the executable program compiled by the user and the base image on which the executable program depends into a Docker image for network performance indicator collection, wherein the executable program is used to obtain network performance indicator data, and is encapsulated in the Prometheus indicator format and then exposed through the HTTP protocol;

[0047] Specifically, the executable program is developed and compiled using the GO language. The executable program is used to monitor the / proc / net / dev file of the Linux system in real time, and parse the / proc / net / dev file to obtain network performance indicator data, and then encapsulate the network performance indicator data in the Prometheus indicator format and expose it through the HTTP protocol; the basic image is an open source basic image obtained from the warehouse.

[0048] In order to build a network performance indicator collection container, a process was first designed, which focused on real-time monitoring of the content changes of the / proc / net / dev file in the Linux system. This file, as a special resource dynamically generated by the Linux system kernel, records in detail various statistical information including: network interface device eth0 (i.e., network performance indicator data), such as the total number of bytes sent, the total number of packets sent, and the total number of packet losses, etc., providing key data for key network performance indicator monitoring.

[0049] Next, a special indicator interface is published using the HTTP protocol. This indicator interface is responsible for encapsulating the network performance indicator data parsed from the / proc / net / dev file in the Prometheus indicator format to ensure data standardization and compatibility. Finally, the encapsulated data is exposed through the HTTP protocol.

[0050] In order to realize the above functions (i.e. the functions of the indicator interface), the GO language is used for development. Taking advantage of the high efficiency and concurrency of the GO language, the indicator interface is compiled into a binary executable program. Subsequently, the executable program and its dependent basic images are packaged into a Docker image (for network performance indicator collection) through Dockerfile. This process not only simplifies the deployment process, but also ensures the consistency of the environment.

[0051] Finally, the built Docker image is published to the warehouse for quick retrieval and deployment when needed.

[0052] Step S104, Kubernetes obtains the Docker image, and starts the Docker image according to the Deployment configuration file to obtain a network performance indicator collection container, wherein the Deployment configuration file is provided with a startup command for the Docker image;

[0053] Specifically, in the vast ecosystem of Kubernetes (K8s), Pod plays the role of the smallest and most flexible deployable unit. It can elegantly encapsulate multiple containers and let these containers share a unified network namespace, thereby achieving efficient network communication and resource sharing. Based on this feature, integrating a dedicated network performance indicator collection container inside the Pod can achieve fine capture of business container network resources without directly intervening in the core logic of the business system, thereby maintaining the purity and independence of business operations.

[0054] In order to seamlessly integrate the network performance indicator collection function, first introduce a network performance indicator collection container specifically used for network performance indicator collection in the business system's Deployment configuration file (specifically, set the startup command of the Docker image in the Deployment configuration file). This network performance indicator collection container uses the previously released Docker image as its operating basis to ensure that the necessary network monitoring tasks can be performed. When the Pod is scheduled and started, all containers in it, including the newly added network performance indicator collection container, will be started synchronously. In this way, the network performance indicator collection container can instantly and meticulously monitor the Pod's network activities.

[0055] Step S106, Prometheus collects the target network performance indicator data of the corresponding container from the network performance indicator data emitted by the network performance indicator collection container according to the configured PodMonitor resource object, and calculates the key network performance indicators according to the target network performance indicator data, wherein the key network performance indicators include: network transmission rate and network packet loss rate;

[0056] Specifically, the network performance indicator data includes: the total number of bytes sent, the total number of data packets sent, and the total number of packet losses.

[0057] During implementation, in order to further simplify the process of Prometheus grabbing network performance indicator data from Pod, the powerful functions of Prometheus Operator are utilized, especially the PodMonitor resource object it provides, which gives Prometheus powerful automation capabilities, enabling Prometheus to intelligently identify and monitor those Pod containers that meet the preset labels (i.e., the Pod containers corresponding to the parameters in the PodMonitor resource object). By configuring the PodMonitor resource object, you can easily bind the network performance indicator collection container to the monitoring task, and set the following parameters in the PodMonitor resource object: the port, path, and frequency of data collection. This flexible configuration method enables Prometheus to automatically discover and collect these target network performance indicator data according to the configured PodMonitor resource object (i.e., collect the target network performance indicator data of the corresponding container from the network performance indicator data emitted by the network performance indicator collection container), thereby greatly improving the efficiency and accuracy of monitoring.

[0058] Specifically, in terms of network performance indicator data collection, the following is a detailed description of the network performance indicator data exposed by the container:

[0059] network_transmit_bytes_total: This metric represents the total number of bytes sent, and its type is Counter. It records the total number of bytes sent by the container since the container was started until the current moment, providing a direct basis for evaluating network traffic.

[0060] network_package_transmit_total: This metric is the total number of packets sent, and also uses the Counter type. It records the total number of packets sent by the container from the start of the container to the current moment.

[0061] network_package_drop_total: This metric measures the total number of packet losses, and its type is also Counter. It records the total number of data packets lost during the transmission of the container since the container was started to the current moment, and is an important indicator for evaluating network stability and reliability.

[0062] Through these carefully designed indicators, the network performance of Pods in the Kubernetes cluster can be monitored and analyzed more comprehensively and accurately, providing strong support for the stable operation and optimization of the system.

[0063] The above corresponding container is specifically the container corresponding to the parameters in the PodMonitor resource object.

[0064] After obtaining the target network performance indicator data, key network performance indicators are further calculated based on the target network performance indicator data, wherein the key network performance indicators include: network transmission rate and network packet loss rate.

[0065] As a key parameter to measure the speed of data transmission, network transmission rate directly affects the response time of applications and user experience. Especially in scenarios with high concurrency and low latency requirements, such as online games, financial trading systems, and real-time data analysis, changes in network transmission rate have an important impact on container scaling decisions.

[0066] In step S108, HPA obtains the key network performance indicators collected by Prometheus through Prometheus Adapter, and scales the corresponding container according to the preset scaling strategy based on the key network performance indicators and the preset key network performance indicator thresholds.

[0067] Specifically, using the Horizontal Pod Autoscaler (HPA) mechanism, we designed a set of custom scaling strategies driven by key network performance indicators to optimize network performance and resource utilization efficiency. This scaling strategy closely tracks the fluctuations of key network performance indicators and intelligently adjusts the number of Pod containers to cope with dynamic demand.

[0068] Given that HPA natively only supports automatic scaling of CPU and memory utilization, in order to incorporate key network performance indicator considerations, the Prometheus Adapter is introduced as a bridge, which can seamlessly convert Prometheus metrics data (such as key network performance indicators) into an HPA-compatible format. Subsequently, the PromQL query language provided by Prometheus is used to accurately extract and convert network transmission rate (network_transmit_bytes_rate) and network packet loss rate (network_package_drop_rate) as key network performance indicators. Furthermore, HPA scales the corresponding containers according to the preset scaling strategies based on the key network performance indicators and the preset key network performance indicator thresholds.

[0069] In an embodiment of the present invention, a method for elastically scaling a container is provided, comprising: Dockerfile packages a user-compiled executable program and a basic image on which the executable program depends into a Docker image for network performance indicator collection, wherein the executable program is used to obtain network performance indicator data, and is encapsulated in a Prometheus indicator format and then exposed through an HTTP protocol; Kubernetes obtains the Docker image, and starts the Docker image according to a Deployment configuration file to obtain a network performance indicator collection container, wherein a startup command for the Docker image is set in the Deployment configuration file; Prometheus collects target network performance indicator data of a corresponding container from the network performance indicator data emitted by the network performance indicator collection container according to a configured PodMonitor resource object, and calculates key network performance indicators according to the target network performance indicator data, wherein the key network performance indicators include: network transmission rate and network packet loss rate; HPA obtains the key network performance indicators collected by Prometheus through a Prometheus Adapter, and scales the corresponding container according to a preset scaling strategy according to the key network performance indicators and preset key network performance indicator thresholds. From the above description, it can be seen that the elastic scaling method of containers of the present invention takes into account the influence of network transmission rate and network packet loss rate on container scaling. The corresponding container can be scaled according to the preset scaling strategy based on the real-time monitored network transmission rate and network packet loss rate, thereby realizing dynamic adjustment of container scaling, making the scaling more reasonable and eliminating the need for manual intervention by operation and maintenance personnel or setting up complex automation scripts. The process of automated dynamic adjustment simplifies the complexity of operation and maintenance management, and alleviates the technical problems of traditional container scaling methods that do not consider the influence of network transmission rate on containers, unreasonable scaling, and high operation and maintenance complexity.

[0070] The above content briefly introduces the container elastic expansion and contraction method of the present invention, and the specific contents involved are described in detail below.

[0071] In an optional embodiment of the present invention, the key network performance indicator is calculated according to the target network performance indicator data, which specifically includes the following steps:

[0072] (1) The network transmission rate is calculated according to the network transmission rate calculation formula: Network transmission rate = total number of bytes sent at the current moment - total number of bytes sent at the previous moment, where the time interval between the current moment and the previous moment is 1 minute;

[0073] (2) The network packet loss rate is calculated according to the network packet loss rate calculation formula: Network packet loss rate = (total number of packet losses per unit time / total number of data packets sent per unit time) * 100, where the unit time is 1 minute.

[0074] Specifically, the total number of bytes sent by the container in the last minute is used to calculate the network sending rate, and the total number of packets sent and the total number of packet losses are combined to calculate the corresponding network packet loss rate. These two key network performance indicators - network sending rate and network packet loss rate, are then configured as custom elastic scaling indicators, which directly reflect the health of container network performance.

[0075] Network transmission rate: calculated by rate(network_transmit_bytes_total[1m]), reflecting the average growth rate of network transmission bytes within 1 minute.

[0076] Network packet loss rate: calculated by the expression increase(network_package_drop_total[1m]) / increase(network_package_transmit_total[1m])*100, which measures the change in the ratio of the total number of packet drops to the total number of transmitted packets within 1 minute.

[0077] In an optional embodiment of the present invention, scaling the corresponding container according to the key network performance indicator and the preset key network performance indicator threshold according to the preset scaling strategy specifically includes the following steps:

[0078] (1) If the network transmission rate for a preset number of consecutive monitoring cycles is lower than a fixed ratio of the network transmission rate threshold, the number of corresponding containers is expanded to the target number, where the target number = (current network transmission rate / network transmission rate threshold)*current number of containers;

[0079] Specifically, the Pod expansion logic is: when the network_transmit_bytes_rate (network transmission rate) is lower than 10% (i.e. a fixed ratio) of the network transmission rate threshold for multiple consecutive monitoring cycles (for example, 5 minutes), the strategy will automatically trigger the expansion mechanism, driving the growth of network throughput by increasing the number of containers, thereby improving network utilization efficiency.

[0080] The final number of containers after expansion (ie, the target number) = (current network transmission rate / network transmission rate threshold)*the current number of containers.

[0081] (2) If the network transmission rate for a preset number of consecutive monitoring periods is higher than a fixed ratio of the network transmission rate threshold, the number of corresponding containers is reduced to a first target number, where the first target number = (current network transmission rate / network transmission rate threshold)*current number of containers;

[0082] (3) If the network packet loss rate for a preset number of consecutive monitoring periods is higher than the network packet loss rate threshold, the number of corresponding containers is reduced to a second target number, where the second target number = (current network packet loss rate / network packet loss rate threshold)*current number of containers;

[0083] Specifically, the logic of Pod scaling down is: if the network_transmit_bytes_rate (network transmission rate) is higher than 10% (i.e., a fixed ratio) of the network transmission rate threshold for multiple consecutive monitoring cycles (e.g., 5 minutes), the Pod container is reduced to balance the network load. In addition, if the network_package_drop_rate (network packet loss rate) exceeds the standard continuously (more than 1%, i.e., 1% is the network packet loss rate threshold) and is not caused by a fault, the number of containers will also be considered to be reduced to reduce the risk of single container packet loss and ensure network quality.

[0084] (4) Determine a target quantity for the corresponding container according to the first target quantity and / or the second target quantity.

[0085] Specifically, if only the above-mentioned reduction logic (2) is satisfied, the first target quantity is used as the target quantity of the corresponding container; if only the above-mentioned reduction logic (3) is satisfied, the second target quantity is used as the target quantity of the corresponding container; if both the above-mentioned reduction strategies (2) and (3) are satisfied, the larger one between the first target quantity and the second target quantity is used as the target quantity of the corresponding container.

[0086] By implementing the above strategies, we can achieve fine management and efficient use of Pod resources in the Kubernetes cluster, ensuring stable and optimized network performance.

[0087] By using the Horizontal Pod Autoscaler (HPA), an automatic expansion and contraction mechanism for Pod resources is implemented. The core of this mechanism is that the HPA controller can compare and analyze a series of customized key network performance indicators with the preset key network performance indicator thresholds in real time, so as to intelligently evaluate whether the number of container instances needs to be adjusted to cope with dynamically changing load demands.

[0088] The thresholds of key network performance indicators are carefully calculated to optimize the set values ​​based on in-depth consideration of factors such as the current network environment status, the maximum supported transmission rate, and the proportion of the service's transmission rate in the system. Once HPA detects that key network performance indicators - such as network transmission rate or network packet loss rate - deviate from the preset optimization range, its built-in automatic response mechanism will be triggered immediately. If the network transmission rate exceeds the upper limit of the optimization range, the system will make intelligent decisions to reduce the number of container instances to avoid resource waste and maximize cost-effectiveness, so as to effectively regulate the network transmission rate and ensure that it remains within a reasonable threshold range; on the contrary, if the network transmission rate does not reach the lower limit of the optimization range, the system will automatically increase the container instance to improve the network throughput, ensure smooth data transmission, and finely manage the network packet loss rate to keep it at the minimum acceptable to business operations, thereby ensuring the continuous stable operation of the service and efficient and reliable performance.

[0089] The above process designs a set of custom elastic scaling strategies (i.e., expansion and contraction strategies) based on key network performance indicators. The core of this strategy is to use the collected network performance indicator data as the key elastic scaling basis for the HPA (Horizontal Pod Autoscaler) controller in the Kubernetes platform. Thanks to the flexibility of Kubernetes, it already supports multiple standard metrics and allows users to define additional metrics to achieve more refined automatic expansion and contraction management.

[0090] The above-mentioned automated scaling process is designed to ensure that the network transmission rate is maintained at an efficient and stable level, while strictly controlling the network packet loss rate so that it always remains within the acceptable range for the business. In this way, high availability of services and optimization of cost-effectiveness are ensured.

[0091] Traditional technologies have many deficiencies in container expansion and contraction, especially when dealing with the key indicator of network transmission rate. Therefore, the present invention proposes a method for elastic expansion and contraction of containers based on the network transmission rate indicator. The core of this method is to monitor the network transmission rate of the container in real time, and use it as an important basis for resource scheduling and dynamic adjustment of capacity. By deeply analyzing the fluctuation trend of bandwidth utilization (network transmission rate and network packet loss rate), the system can intelligently predict and execute container expansion or contraction operations, aiming to achieve optimal configuration and efficient utilization of bandwidth resources. This method is not only an effective supplement to the elastic scaling mechanism of traditional network-intensive applications, but also a significant improvement and enhancement of its performance and efficiency.

[0092] The container elastic expansion and contraction method proposed in the present invention has significant beneficial effects compared with the traditional container resource management method, which is specifically reflected in the following aspects:

[0093] Accurate resource allocation to improve system performance: By real-time monitoring of network transmission rate, a key indicator that directly reflects the application load status, the present invention can achieve accurate dynamic adjustment of container resources. Automatically expand capacity when load increases to ensure that the application can efficiently process requests and avoid performance bottlenecks; automatically shrink capacity when load decreases to release idle resources and improve resource utilization. This precise resource allocation strategy significantly improves the overall performance and response speed of the system;

[0094] Enhanced system stability and reliability: In the face of sudden changes in network traffic or load, the present invention can quickly respond and adjust container resources, effectively preventing service interruptions or performance degradation caused by insufficient resources. At the same time, through intelligent monitoring and expansion and contraction strategies, it reduces system fluctuations caused by improper resource allocation, significantly enhancing system stability and reliability;

[0095] Optimize resource cost-effectiveness: The elastic scaling method based on the network transmission rate index can dynamically adjust the resource scale according to the actual load demand and realize on-demand allocation of resources. This method avoids long-term idleness and waste of resources and reduces operating costs. At the same time, through precise scaling operations, it ensures sufficient resource support under high load, thereby improving the overall cost-effectiveness;

[0096] Simplify the complexity of operation and maintenance management: The automated and intelligent scaling process of the present invention greatly reduces the burden on operation and maintenance personnel. Operation and maintenance personnel do not need to frequently manually adjust resource configurations, reducing the risk of human intervention and errors. This allows the operation and maintenance team to focus more on core business and high-level operation and maintenance tasks, such as system optimization, security reinforcement, etc., thereby improving overall operation and maintenance efficiency and management level;

[0097] Supports diverse application scenarios: Whether it is a high-concurrency Web service, a real-time data processing system or a large-scale distributed application, the present invention can accurately schedule container resources according to specific network transmission rate indicators. This flexibility and adaptability enable the method to be widely used in various complex business scenarios to meet the resource management needs of different applications.

[0098] In summary, the container elastic scaling method based on network transmission rate indicators proposed in the present invention has shown significant beneficial effects in improving system performance, enhancing stability, optimizing cost-effectiveness, simplifying operation and maintenance management, and supporting diversified application scenarios, and has important practical application value and technological innovation significance.

[0099] Embodiment 2:

[0100] The embodiment of the present invention further provides a container elastic expansion and contraction device, which is mainly used to execute the container elastic expansion and contraction method provided in the first embodiment of the present invention. The container elastic expansion and contraction device provided in the embodiment of the present invention is specifically introduced below.

[0101] Figure 2 is a schematic diagram of a container elastic expansion and contraction device according to an embodiment of the present invention. Figure 2 As shown, the device mainly includes: a packaging unit 10, a startup unit 20, a collection and calculation unit 30, and a capacity expansion and contraction unit 40, wherein:

[0102] The packaging unit is used to package the user-compiled executable program and the basic image that the executable program depends on into a Docker image for network performance indicator collection. The executable program is used to obtain network performance indicator data, and is packaged in the Prometheus indicator format and then exposed through the HTTP protocol;

[0103] A startup unit is used for Kubernetes to obtain a Docker image and start the Docker image according to a Deployment configuration file to obtain a network performance indicator collection container, wherein a startup command of the Docker image is set in the Deployment configuration file;

[0104] The collection and calculation unit is used for Prometheus to collect the target network performance indicator data of the corresponding container from the network performance indicator data emitted by the network performance indicator collection container according to the configured PodMonitor resource object, and calculate the key network performance indicators according to the target network performance indicator data, where the key network performance indicators include: network transmission rate and network packet loss rate;

[0105] The scaling unit is used by HPA to obtain the key network performance indicators collected by Prometheus through the Prometheus Adapter, and scale the corresponding containers according to the preset scaling strategies based on the key network performance indicators and the preset key network performance indicator thresholds.

[0106] In an embodiment of the present invention, a container elastic expansion and contraction device is provided, including: Dockerfile packages a user-compiled executable program and a basic image on which the executable program depends into a Docker image for network performance indicator collection, wherein the executable program is used to obtain network performance indicator data, and is encapsulated in a Prometheus indicator format and then exposed through an HTTP protocol; Kubernetes obtains the Docker image, and starts the Docker image according to a Deployment configuration file to obtain a network performance indicator collection container, wherein a startup command of the Docker image is set in the Deployment configuration file; Prometheus collects target network performance indicator data of a corresponding container from the network performance indicator data emitted by the network performance indicator collection container according to a configured PodMonitor resource object, and calculates key network performance indicators according to the target network performance indicator data, wherein the key network performance indicators include: network transmission rate and network packet loss rate; HPA obtains the key network performance indicators collected by Prometheus through a Prometheus Adapter, and expands and contracts the corresponding container according to a preset expansion and contraction strategy according to the key network performance indicators and preset key network performance indicator thresholds. From the above description, it can be seen that in the container elastic expansion and contraction device of the present invention, the influence of network transmission rate and network packet loss rate on container expansion and contraction is taken into account, and the corresponding container can be expanded and contracted according to the preset expansion and contraction strategy based on the real-time monitored network transmission rate and network packet loss rate, that is, dynamic adjustment of container expansion and contraction is realized, and the expansion and contraction is more reasonable. There is no need for operation and maintenance personnel to manually intervene or set up complex automation scripts. The process of automated dynamic adjustment simplifies the complexity of operation and maintenance management, and alleviates the technical problems that traditional container expansion and contraction methods do not consider the influence of network transmission rate on containers, unreasonable expansion and contraction, and high operation and maintenance complexity.

[0107] Optionally, the executable program is developed and compiled using the GO language. The executable program is used to monitor the / proc / net / dev file of the Linux system in real time, and parse the / proc / net / dev file to obtain network performance indicator data, and then encapsulate the network performance indicator data in accordance with the Prometheus indicator format and expose it through the HTTP protocol; the basic image is an open source basic image obtained from the warehouse.

[0108] Optionally, the parameters in the PodMonitor resource object include: a port, a path, and a crawling frequency for data collection.

[0109] Optionally, the network performance indicator data includes: the total number of bytes sent, the total number of data packets sent, and the total number of packet losses.

[0110] Optionally, the collection and calculation unit is also used to: calculate the network transmission rate according to the network transmission rate calculation formula: network transmission rate = total number of bytes sent at the current moment - total number of bytes sent at the previous moment, wherein the time interval between the current moment and the previous moment is 1 minute; calculate the network packet loss rate according to the network packet loss rate calculation formula: network packet loss rate = (total number of packet losses per unit time / total number of data packets sent per unit time)*100, wherein the unit time is 1 minute.

[0111] Optionally, the scaling unit is further used to: if the network transmission rate for a consecutive preset number of monitoring cycles is lower than a fixed proportion of the network transmission rate threshold, expand the number of corresponding containers to a target number, wherein the target number = (current network transmission rate / network transmission rate threshold)*current number of containers.

[0112] Optionally, the scaling unit is further used to: if the network transmission rate for a continuous preset number of monitoring cycles is higher than a fixed proportion of the network transmission rate threshold, the number of corresponding containers is scaled down to a first target number, wherein the first target number=(current network transmission rate / network transmission rate threshold)*current number of containers; if the network packet loss rate for a continuous preset number of monitoring cycles is higher than the network packet loss rate threshold, the number of corresponding containers is scaled down to a second target number, wherein the second target number=(current network packet loss rate / network packet loss rate threshold)*current number of containers; determine the target number of corresponding containers according to the first target number and / or the second target number.

[0113] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0114] like Figure 3 As shown, an electronic device 600 provided in an embodiment of the present application includes: a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the above-mentioned container elastic expansion and contraction method.

[0115] Specifically, the above-mentioned memory 602 and processor 601 can be general-purpose memory and processor, which are not specifically limited here. When the processor 601 runs the computer program stored in the memory 602, the above-mentioned container elastic expansion and contraction method can be executed.

[0116] The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 601. The above processor 601 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0117] Corresponding to the above-mentioned container elastic expansion and contraction method, an embodiment of the present application also provides a computer-readable storage medium, which stores machine-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned container elastic expansion and contraction method.

[0118] The container elastic expansion and contraction device provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application has the same implementation principle and technical effects as the aforementioned method embodiment. For the sake of brief description, for the parts not mentioned in the device embodiment, reference can be made to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0119] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0120] For another example, the flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0121] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0122] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0123] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or partly contribute to the prior art or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the vehicle marking method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0124] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0125] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solution of the present application, rather than to limit it. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solution recorded in the aforementioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiment of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for elastically expanding and contracting a container, characterized in that: include: Dockerfile packages the executable program compiled by the user and the base image that the executable program depends on into a Docker image for network performance indicator collection, wherein the executable program is used to obtain network performance indicator data, and then encapsulates it in the Prometheus indicator format and exposes it through the HTTP protocol; Kubernetes obtains the Docker image, and starts the Docker image according to the Deployment configuration file to obtain a network performance indicator collection container, wherein the Deployment configuration file is provided with a startup command for the Docker image; Prometheus collects the target network performance indicator data of the corresponding container from the network performance indicator data emitted by the network performance indicator collection container according to the configured PodMonitor resource object, and calculates the key network performance indicators according to the target network performance indicator data, wherein the key network performance indicators include: network transmission rate and network packet loss rate; HPA obtains the key network performance indicators collected by Prometheus through Prometheus Adapter, and scales the corresponding container according to the key network performance indicators and preset key network performance indicator thresholds according to the preset scaling strategy.

2. The method according to claim 1, characterized in that The executable program is developed and compiled using the GO language. The executable program is used to monitor the / proc / net / dev file of the Linux system in real time, parse the / proc / net / dev file, obtain the network performance indicator data, and then encapsulate the network performance indicator data according to the Prometheus indicator format and expose it through the HTTP protocol; The base image is an open source base image obtained from a warehouse.

3. The method according to claim 1, characterized in that The parameters in the PodMonitor resource object include: the port, path and crawling frequency of data collection.

4. The method according to claim 1, characterized in that: The network performance indicator data includes: the total number of bytes sent, the total number of data packets sent and the total number of packet losses.

5. The method according to claim 1, characterized in that The key network performance indicators are calculated based on the target network performance indicator data, including: The network transmission rate is calculated according to the network transmission rate calculation formula: network transmission rate = total number of bytes sent at the current moment - total number of bytes sent at the previous moment, wherein the time interval between the current moment and the previous moment is 1 minute; The network packet loss rate is calculated according to the network packet loss rate calculation formula: Network packet loss rate = (total number of packet losses per unit time / total number of sent data packets per unit time)*100, wherein the unit time is 1 minute.

6. The method according to claim 1, characterized in that Scaling the corresponding container according to the key network performance indicator and the preset key network performance indicator threshold according to the preset scaling strategy, including: If the network transmission rate for a preset number of consecutive monitoring cycles is lower than a fixed ratio of the network transmission rate threshold, the number of corresponding containers is expanded to a target number, wherein the target number=(current network transmission rate / the network transmission rate threshold)*current number of containers.

7. The method according to claim 1, characterized in that Scaling the corresponding container according to the key network performance indicator and the preset key network performance indicator threshold according to the preset scaling strategy, including: If the network transmission rate for a preset number of consecutive monitoring cycles is higher than a fixed ratio of the network transmission rate threshold, the number of corresponding containers is reduced to a first target number, wherein the first target number = (current network transmission rate / the network transmission rate threshold)*the current number of containers; If the network packet loss rate for a preset number of consecutive monitoring periods is higher than the network packet loss rate threshold, the number of corresponding containers is reduced to a second target number, wherein the second target number = (current network packet loss rate / the network packet loss rate threshold)*the current number of containers; A target quantity of a corresponding container is determined according to the first target quantity and / or the second target quantity.

8. A container elastic expansion and contraction device, characterized in that: include: A packaging unit is used to package the executable program compiled by the user and the base image that the executable program depends on into a Docker image for network performance indicator collection through Dockerfile, wherein the executable program is used to obtain network performance indicator data, and then encapsulates it according to the Prometheus indicator format and exposes it through the HTTP protocol; A startup unit, used for Kubernetes to obtain the Docker image, and to start the Docker image according to a Deployment configuration file to obtain a network performance indicator collection container, wherein the Deployment configuration file is provided with a startup command for the Docker image; A collection and calculation unit, which is used for Prometheus to collect target network performance indicator data of the corresponding container from the network performance indicator data emitted by the network performance indicator collection container according to the configured PodMonitor resource object, and calculate key network performance indicators according to the target network performance indicator data, wherein the key network performance indicators include: network transmission rate and network packet loss rate; The expansion and contraction unit is used for HPA to obtain the key network performance indicators collected by Prometheus through Prometheus Adapter, and to expand and contract the corresponding containers according to the key network performance indicators and the preset key network performance indicator thresholds according to the preset expansion and contraction strategies.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to execute the method according to any one of claims 1 to 7.

Images