Automatic maintenance method and device of container and computer program product

By monitoring container performance indicators in real time and using target operation and maintenance models for status classification, automatically predicting and diagnosing faults, the problem of low maintenance efficiency in container system in K8s environment is solved, and fast response and efficient fault repair are achieved.

CN119806880APending Publication Date: 2025-04-11BEIJING PACTERA JINXIN TECH LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411885269.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-12-19
Publication Date
2025-04-11

AI Technical Summary

Technical Problem

The maintenance efficiency of container systems in K8s environment is low, and operation and maintenance personnel need to manually troubleshoot problems and perform maintenance operations, which takes a long time.

Method used

By monitoring the container performance indicators in the cluster in real time, obtaining the preprocessed indicator data, and inputting them into the pretrained target operation and maintenance model, and computing the status classification results of the current cluster content machine, including potential faults and actual faults. Based on the fault prediction and diagnosis results, alarm information is issued and automatic maintenance is carried out.

Benefits of technology

It quickly captures the dynamic changes in the operating state of the container, reduces the time of failure impact, improves the pertinence and efficiency of operation and maintenance work, avoids potential failures from developing into actual failures, ensures service stability, and improves fault repair efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119806880A_ABST
    Figure CN119806880A_ABST
Patent Text Reader

Abstract

The invention discloses an automatic maintenance method and device of a container and a computer program product, and relates to the technical field of cloud computing and containers, and the method comprises the following steps: monitoring a container performance index in a cluster in real time, and obtaining preprocessed index data; inputting the index data into a pre-trained target operation and maintenance model, and calculating a state classification result of a current cluster inner container, the state classification result comprising a potential fault and an actual fault; if the containers in the cluster have potential faults, alarm information is sent out according to fault prediction information, and corresponding maintenance is carried out on the target container in combination with current operation state information; and if the containers in the cluster have actual faults, performing corresponding maintenance on the target container according to a fault diagnosis result. The dynamic change of the running state of the container can be captured by monitoring the performance index of the container in the cluster in real time. Faults are predicted and diagnosed based on state classification results, fault causes are automatically positioned, corresponding maintenance operation is carried out, and the fault repair efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of cloud computing and container technology, and in particular to an automated maintenance method, device, and computer program product for a container. Background Art

[0002] With the popularity of cloud computing and container technology, K8s (Kubernetes) container orchestration platform has become the mainstream choice for enterprises to deploy and manage microservices. However, the complexity of operation and maintenance in the K8s environment has also increased. When a problem occurs in the system, the operation and maintenance personnel need to manually troubleshoot the root cause of the problem and perform maintenance operations, which is time-consuming and leads to low maintenance efficiency.

[0003] Therefore, how to improve the maintenance efficiency of container systems is a problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the present application is to provide a method, device and computer program product for automated maintenance of a container, aiming to solve the technical problem of low maintenance efficiency of the container system.

[0005] To achieve the above objectives, the present application proposes an automated maintenance method for a container, the method comprising:

[0006] Monitor container performance indicators in the cluster in real time and obtain pre-processed indicator data;

[0007] Input the indicator data into the pre-trained target operation and maintenance model to calculate the status classification results of the containers in the current cluster, wherein the status classification results include potential faults and actual faults;

[0008] If the container in the cluster has the potential fault, an alarm message is issued according to the fault prediction information, and corresponding maintenance is performed on the target container in combination with the current running status information;

[0009] If the actual fault exists in the container in the cluster, corresponding maintenance is performed on the target container according to the fault diagnosis result.

[0010] In one embodiment, the state classification result includes a normal state, the potential fault, and the actual fault, and the step of inputting the indicator data into a pre-trained target operation and maintenance model to calculate the state classification result of the container in the current cluster includes:

[0011] Obtain data features through indicator data, input preset weights and the data features into the target operation and maintenance model, and output the operation status information of the current cluster;

[0012] Based on the operation status information, a status classification result of the container in the current cluster is generated, and the status classification result is identified as the normal state, the potential fault or the actual fault.

[0013] In one embodiment, the current running status information includes the current number of service requests, and the step of issuing an alarm message according to the fault prediction information if the container in the cluster has the potential fault, and performing corresponding maintenance on the target container in combination with the current running status information includes:

[0014] If the container in the cluster has the potential fault, an alarm message is issued according to the fault prediction information;

[0015] Detecting whether the current number of service requests reaches a preset request number threshold;

[0016] If the current number of service requests reaches a preset threshold of the number of requests, the containers in the cluster are ranked according to the frequency of container use to determine a priority container and a secondary container;

[0017] By reducing the number of container replicas of the secondary container, resources are allocated to the priority container.

[0018] In one embodiment, after the step of detecting that the current number of service requests reaches a threshold of a preset number of requests, the method further includes:

[0019] If the current service request quantity reaches a preset request quantity threshold, a rate limiter is configured for the target container, and real-time monitoring information of the rate limiter is obtained;

[0020] The current limiting parameters are dynamically adjusted according to the real-time monitoring information.

[0021] In one embodiment, if the actual fault exists in the container in the cluster, the step of performing corresponding maintenance on the target container according to the fault diagnosis result includes:

[0022] If the actual fault exists in the container in the cluster, locate the faulty container according to the fault diagnosis result;

[0023] If the fault diagnosis result is a transient fault, restart the faulty container and record the number of restarts;

[0024] If the number of restarts reaches the preset restart threshold, the corresponding traffic of the failed container is directed to the new container copy.

[0025] In one embodiment, after the step of locating the fault container according to the fault diagnosis result, the following steps are included:

[0026] Check whether the service response time of the faulty container reaches the preset response time;

[0027] If the service response time of the faulty container reaches the preset response time, a circuit breaker is set to block new requests, a new container copy is created, and the corresponding traffic of the faulty container is directed to the new container copy.

[0028] In one embodiment, if the actual fault exists in the container in the cluster, the step of performing corresponding maintenance on the target container according to the fault diagnosis result includes:

[0029] After the system version is updated, if the fault diagnosis result is a transient fault, restart the fault container and record the number of restarts;

[0030] If the number of restarts reaches a preset restart threshold, determining the fault diagnosis result as an update failure;

[0031] If the fault diagnosis result is that the update failed, roll back to the previous version until the system returns to normal.

[0032] In one embodiment, before the step of inputting the indicator data into the pre-trained target operation and maintenance model, the following steps are included:

[0033] Adding labels to the preprocessed indicator data to obtain labeled data, wherein the labels are used to distinguish between normal states, potential faults, and actual faults;

[0034] The historical data and the labeled data are input into the initial operation and maintenance model for training, and the model weights are optimized through a supervised learning method to obtain the target operation and maintenance model.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes an automatic maintenance device for a container, the automatic maintenance device for a container comprising:

[0036] The data acquisition module is used to monitor the performance indicators of containers in the cluster in real time and obtain pre-processed indicator data;

[0037] A fault classification module is used to input the indicator data into a pre-trained target operation and maintenance model to calculate the status classification results of the containers in the current cluster, wherein the status classification results include potential faults and actual faults;

[0038] A fault prediction module is used to issue an alarm message according to the fault prediction information if the container in the cluster has the potential fault, and perform corresponding maintenance on the target container in combination with the current running status information;

[0039] The fault diagnosis module is used to perform corresponding maintenance on the target container according to the fault diagnosis result if the container in the cluster has the actual fault.

[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes an automated maintenance device for a container, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the automated maintenance method for the container as described above.

[0041] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the automated maintenance method of the container as described above are implemented.

[0042] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the automated maintenance method of the container as described above are implemented.

[0043] One or more technical solutions proposed in this application have at least the following technical effects:

[0044] Monitor the performance indicators of containers in the cluster in real time and obtain preprocessed indicator data; input the indicator data into the pre-trained target operation and maintenance model to calculate the status classification results of the containers in the current cluster. The status classification results include potential faults and actual faults; if there is a potential fault in the container in the cluster, issue an alarm based on the fault prediction information, and perform corresponding maintenance on the target container in combination with the current operating status information; if there is an actual fault in the container in the cluster, perform corresponding maintenance on the target container based on the fault diagnosis results. By monitoring the performance indicators of containers in the cluster in real time, the dynamic changes of the container operation status can be quickly captured to ensure timely response of the system and reduce the impact time of faults. According to the status classification results, the container operation status is divided into potential faults and actual faults, and differentiated maintenance strategies are adopted for different fault types to improve the pertinence and efficiency of operation and maintenance work. When a potential fault occurs, the user is prompted by an alarm message and maintenance is performed in combination with the operating status information, which can intervene in advance to prevent the potential fault from developing into an actual fault and ensure service stability. For actual faults, the cause of the fault is automatically located and corresponding maintenance operations are performed through the model diagnosis results, effectively reducing the time for manual troubleshooting and processing and improving the efficiency of fault repair. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0047] Figure 1 This is a flow chart of the first embodiment of the automated maintenance method for a container of the present application;

[0048] Figure 2 This is a flow chart of a second embodiment of the automated container maintenance method of the present application;

[0049] Figure 3 This is a flow chart of a third embodiment of the automated container maintenance method of the present application;

[0050] Figure 4 This is a schematic diagram of the module structure of the automated maintenance device for the container according to an embodiment of the present application.

[0051] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0053] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0054] There are many problems with traditional operation and maintenance, including: Traditional operation and maintenance usually requires a lot of manual intervention. From monitoring system status, identifying problems to solving problems, operation and maintenance personnel need to do manual operations. The time between discovering problems and solving problems is long, and it is impossible to respond to emergencies quickly. For services with high requirements for business continuity and user experience, this method is not only inefficient but also prone to errors. Secondly, as the scale of the system expands, the traditional operation and maintenance model is difficult to effectively manage and maintain a large number of servers and applications. During the expansion process, more human resources need to be continuously invested, which is costly and inflexible. Thirdly, in complex distributed systems, it is very difficult to locate the source of the problem by manual means, especially when the problem involves multiple components or services. The troubleshooting process may take a long time and key information is easy to be missed. Finally, traditional operation and maintenance usually does not make full use of data analysis to predict potential problems or optimize performance. It is often based on personal experience and lacks unified standards and processes. This leads to large differences in practices between different teams. The lack of standardization makes it difficult to share and inherit knowledge. Operation and maintenance decisions are more based on intuition rather than objective data, which affects the accuracy of decisions. Therefore, how to improve the maintenance efficiency of container systems while ensuring the accuracy of decisions is a problem that needs to be solved at present.

[0055] The present application provides a solution to monitor the performance indicators of containers in the cluster in real time and obtain preprocessed indicator data; input the indicator data into the pre-trained target operation and maintenance model, calculate the status classification results of the containers in the current cluster, and the status classification results include potential faults and actual faults; if there is a potential fault in the container in the cluster, issue an alarm message according to the fault prediction information, and perform corresponding maintenance on the target container in combination with the current operation status information; if there is an actual fault in the container in the cluster, perform corresponding maintenance on the target container according to the fault diagnosis result. By monitoring the performance indicators of the containers in the cluster in real time, the dynamic changes of the container operation status can be quickly captured to ensure timely response of the system and reduce the impact time of the fault. According to the status classification results, the container operation status is divided into potential faults and actual faults, and differentiated maintenance strategies are adopted for different fault types to improve the pertinence and efficiency of operation and maintenance work. When a potential fault occurs, the user is prompted by an alarm message and maintenance is performed in combination with the operation status information, so that early intervention can be made to avoid the potential fault from developing into an actual fault and ensure service stability. For actual faults, the cause of the fault is automatically located and corresponding maintenance operations are performed through the model diagnosis results, which effectively reduces the time for manual troubleshooting and processing and improves the efficiency of fault repair.

[0056] Based on this, the embodiment of the present application provides an automated maintenance method for a container, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the automated container maintenance method of the present application.

[0057] In this embodiment, the automated maintenance method for the container includes steps S10 to S40:

[0058] Step S10: monitor the performance indicators of containers in the cluster in real time and obtain pre-processed indicator data.

[0059] It should be noted that a cluster can be composed of a group of server nodes for hosting and managing a computing resource pool for containerized applications, such as a cluster environment running on a Kubernetes platform. Container performance indicators can be key parameters for measuring the operating status of a container. For example, container performance indicators can include CPU usage, memory usage, storage usage, container health, etc. Prometheus or other monitoring tools can be deployed in a production environment to continuously monitor all containers in the cluster and record data points every minute or shorter. Prometheus can control the frequency of data collection by configuring scrape_interval. The preprocessed indicator data can be understood as container performance indicators after cleaning, standardization, and feature extraction. The Prometheus raw data (container performance indicators) can be cleaned, invalid or abnormal values ​​can be removed, and converted into a unified format. For example, operations such as removing negative CPU usage values ​​and unifying memory units can be performed to obtain preprocessed indicator data.

[0060] Step S20, input the indicator data into the pre-trained target operation and maintenance model, and calculate the status classification results of the containers in the current cluster, where the status classification results include potential faults and actual faults.

[0061] It should be noted that the indicator data can be a pre-processed container performance indicator, and the target operation and maintenance model can be a model trained in advance by machine learning based on normal operation data and fault data. The target operation and maintenance model can be used to determine the fault type of the container in the current cluster. It can be understood that the status classification results can include potential faults and actual faults. Among them, potential faults can be understood as container performance indicators approaching the fault threshold, but have not yet caused actual faults. At this time, early warning can be used to avoid the expansion of the problem. Actual faults can be understood as the container has experienced abnormal operation, resulting in significant impact on service performance or function.

[0062] Step S30: If there is a potential fault in the container in the cluster, an alarm message is issued according to the fault prediction information, and corresponding maintenance is performed on the target container in combination with the current running status information.

[0063] It should be noted that the fault prediction information can be understood as the output result based on the target operation and maintenance model, indicating the cause and impact range of the potential fault. The alarm information can be a warning information to remind the operation and maintenance personnel to pay attention to potential risks. For example, the alarm can be issued to the operation and maintenance personnel through pop-up windows, voice and other reminders to prevent the potential fault from becoming an actual fault. The system performs corresponding maintenance on the target container based on the fault prediction information and the current operating status information. The target container can be the container that is predicted to fail, and the current operating status information can be the indicator data of the target container.

[0064] Step S40: If there is an actual fault in the container in the cluster, corresponding maintenance is performed on the target container according to the fault diagnosis result.

[0065] It should be noted that the fault diagnosis result can be the specific fault type and cause obtained by the target operation and maintenance model based on the performance indicator analysis, and an alarm message will also be issued based on the fault diagnosis result. Exemplarily, the fault diagnosis result may include the fault location, such as the cluster, namespace, and container where the fault occurred. Exemplarily, the code segment is as follows:

[0066] pod_cpu{cluster="mobile-phone",namespace="mobile",pod="mobile-pod"}95. Cluster="mobile-phone",namespace="mobile",pod="mobile-pod" represent the location information of the cluster, namespace, and container (pod). 95 represents the CPU usage of 95%. When compared with the threshold (such as 80), the fault diagnosis result is obtained and an alarm message is issued. The threshold can be selected based on experience and can usually be set within the model.

[0067] In this embodiment, by real-time monitoring of container performance indicators in the cluster, dynamic changes in the container's operating status can be quickly captured to ensure timely system response and reduce the impact time of failures. According to the status classification results, the container's operating status is divided into potential failures and actual failures, and differentiated maintenance strategies are adopted for different failure types to improve the pertinence and efficiency of operation and maintenance work. When a potential failure occurs, the user is prompted by an alarm message and maintenance is performed in combination with the operating status information, which can intervene in advance to prevent potential failures from developing into actual failures and ensure service stability. For actual failures, the cause of the failure is automatically located through the model diagnosis results and corresponding maintenance operations are performed, effectively reducing the time for manual troubleshooting and processing and improving the efficiency of fault repair.

[0068] Reference Figure 2 , Figure 2This is a flow chart of the second embodiment of the automated maintenance method for containers of the present application, based on the above Figure 1 The first embodiment shown is a second embodiment of the automated maintenance method for a container of the present application.

[0069] In the second embodiment, the step S20 includes:

[0070] Step S201, obtain data features through indicator data, input preset weights and data features into the target operation and maintenance model, and output the operation status information of the current cluster.

[0071] It should be noted that data features can be understood as key features extracted from indicator data for analyzing the operating status of containers. For example, data features may include trend features, distribution features, or time series features. The preset weights may be parameters obtained through training in the target operation and maintenance model, indicating the degree of contribution of specific data features to the status classification. For example, if CPU usage is the main influencing factor of potential failures, the CPU usage weight is set to a higher value. If the network delay is more important for real-time interactive services, the network delay weight is set to a higher value. By inputting the preset weights and data features into the target operation and maintenance model, the mapping relationship between the preset weights and data features can be obtained, and the classification probability can be calculated through the linear combination of weights and data features. The operating status information of the current cluster can be the overall operating status of the containers in the current cluster.

[0072] Step S202: generating a status classification result of the container in the current cluster based on the operation status information, and identifying the status classification result as a normal state, a potential fault, or an actual fault.

[0073] It should be noted that the status classification result identification may include a normal state, a potential fault or an actual fault. For example, the normal state may be identified as "0", the potential fault may be identified as "1", and the actual fault may be identified as "2".

[0074] In this embodiment, by obtaining data features through indicator data, key features can be extracted from complex raw monitoring data, irrelevant or redundant information can be effectively removed, and the accuracy of data analysis can be improved. Based on the preset weight and feature mapping relationship, in-depth analysis of data features can be achieved, thereby quickly identifying the global status of cluster operation, greatly improving operation and maintenance efficiency.

[0075] In one embodiment, step S30 includes: if there is a potential fault in a container in the cluster, issuing an alarm message according to the fault prediction information; detecting whether the current number of service requests reaches a threshold of a preset number of requests; if the current number of service requests reaches the threshold of the preset number of requests, ranking the containers in the cluster according to the frequency of container use, and determining a priority container and a secondary container; and allocating resources to the priority container by reducing the number of container copies of the secondary container.

[0076] It should be noted that the current operation status information includes the current number of service requests, and the situation of detecting whether the current number of service requests has reached the threshold of the preset number of requests includes whether the current number of service requests is greater than or equal to the maximum threshold of the preset number of requests, and whether the current number of service requests is less than or equal to the minimum threshold of the preset number of requests. It can be understood that whether the current number of service requests is greater than or equal to the maximum threshold of the preset number of requests, or the current number of service requests is less than or equal to the minimum threshold of the preset number of requests, it can be regarded as the current number of service requests reaching the threshold of the preset number of requests. The priority container can be a container with a higher frequency of use and serving core business services, and the secondary container can be a container with a lower frequency of use and serving non-core functions.

[0077] Exemplarily, service applications are divided into A, B, C, and D applications, with the importance of Class A applications being the highest and Class D applications being the lowest. When these four types of applications are deployed in a cluster, the available resources (CPU, memory) of the cluster are fixed. When the traffic increases (such as during Double Eleven events), various types of applications are competing for resources. At this time, the use of resources by applications with low importance can be appropriately reduced, or the number of container copies can be reduced to save resources. Resources are given priority to Class A applications with high importance. Among them, the number of container copies can be the number of instances of the container, which is used to share service requests. Exemplarily, the current number of service requests (the number of user requests currently being processed by the target container) can be obtained through a load balancer (such as Kubernetes Ingress) or the container's own monitoring tools (such as Prometheus and Grafana).

[0078] In this implementation, by predicting potential failures and issuing warning information in a timely manner, intervention can be made before the problem occurs, preventing potential failures from evolving into actual failures, and significantly improving the reliability of the system. Combined with the real-time operating status, by reducing the number of copies of secondary containers, resources can be freed up to guarantee the performance requirements of high-priority containers, thereby achieving optimal resource utilization. By ranking the frequency of container use, resource allocation is dynamically adjusted according to the number of requests and frequency of use, avoiding waste of resources caused by blind expansion or reduction.

[0079] In one embodiment, after the step of detecting that the current number of service requests reaches a threshold value of a preset number of requests, the method further includes: if the current number of service requests reaches the threshold value of the preset number of requests, configuring a flow limiter for the target container and obtaining real-time monitoring information of the flow limiter; and dynamically adjusting the flow limiting parameters according to the real-time monitoring information.

[0080] It should be noted that the preset threshold of the number of requests may include the maximum number of requests that the container can normally process and the reasonable number of requests when the container is under low load. It is understandable that exceeding the maximum number of requests may cause response delays or service failures, and being lower than the reasonable number of requests may result in waste of resources.

[0081] Exemplarily, the maximum number of requests is set to 100, that is, a service can normally process 100 requests per second. When the number of requests per second reaches 1000, an exception will be displayed. The target container can be a container instance that is detected to need to be limited in the current running state. The limiter can be a mechanism for limiting the rate at which the container processes requests. Exemplarily, the rate at which the container processes requests can be limited by a leaky bucket algorithm. Specifically, requests can be processed at a fixed rate, and requests that exceed the rate will be discarded. The maximum number of requests within a certain time window can also be limited by a fixed window or sliding window algorithm. Real-time monitoring information may include the current request rate and the number of requests that are limited. Based on the real-time monitoring information, the configuration parameters of the limiter are dynamically adjusted to adapt to the current load changes.

[0082] In this implementation, the request rate of the target container is limited by configuring a rate limiter, reducing the impact on other related services and preventing the spread of cascading failures caused by overload. According to real-time monitoring information, the request rate, queuing strategy and other parameters of the rate limiter are flexibly adjusted to adapt the rate limiter strategy to the ever-changing traffic demand, effectively reducing the request loss rate and service unavailability risk in high-load scenarios, and ensuring the high elasticity and responsiveness of the system.

[0083] In one embodiment, step S40 includes: if there is an actual fault in the container within the cluster, locating the faulty container according to the fault diagnosis result; if the fault diagnosis result is a transient fault, restarting the faulty container and recording the number of restarts; if the number of restarts reaches a preset restart threshold, directing the corresponding traffic of the faulty container to a new container copy.

[0084] It should be noted that a short-term fault may be an exception caused by individual data, such as an exception inside the container (thread exception caused by data format problem) that is not caught and causes an exception to be thrown. The faulty container may be a container instance that the diagnosis results indicate has a problem. The preset restart threshold may be the maximum number of restarts allowed for the container within a specific time range, which may be set according to the fault tolerance of the container and the importance of the service. For example, the preset restart threshold for high-priority services is usually lower. Exemplarily, the abnormally running container instance may be stopped and restarted, and the problem may be solved by reallocating resources, cleaning up processes, or restarting service processes. At the same time, the number of restarts of a single container instance within a certain period of time may be recorded to monitor the stability of the container. Exemplarily, the container is marked with status within the cluster, such as Running, Error, etc. Running is a stable running state, and Error is a state caused by an error inside the container. The frequency of occurrence of these states may be used to determine whether the container is stable. If a container is often in the Error state, the traffic is directed to a healthy container (other healthy copies of the same container). Therefore, when the faulty container cannot be recovered or exceeds the restart threshold, a healthy container instance is created, and user requests or service calls are redirected from the faulty container to the healthy container instance that is running normally, to take over the work of the faulty container.

[0085] In this implementation, the specific faulty container is located through the fault diagnosis results, which can quickly narrow the scope of the problem, reduce the troubleshooting time, and improve the efficiency of fault handling. Automatically restarting the faulty container provides a self-healing method for rapid recovery, reducing the recovery delay caused by manual intervention. When restarting cannot solve the problem, a healthy copy is quickly created to take over the traffic of the faulty container, improving the system's fault tolerance and enhancing system elasticity. By recording the number of restarts and setting a preset threshold, the system resources are avoided from being consumed due to continuous restart of the container, so that resources can be allocated more efficiently.

[0086] In one embodiment, after the step of locating the fault container according to the fault diagnosis result, the method further includes: detecting whether the service response time of the fault container reaches a preset response time; if the service response time of the fault container reaches the preset response time, blocking new requests by setting a circuit breaker, creating a new container copy, and directing the corresponding traffic of the fault container to the new container copy.

[0087] It should be noted that the service response time can be understood as the time interval from the user or client sending a request to the target container completing the request and returning a response. The preset response time can be a performance benchmark value set for the container service response capability, which is used to determine whether the service is normal. Exemplarily, the normal response time should be 20 milliseconds, and the response time reaches 5 seconds when overloaded (5 seconds means that the user refreshes the page and can only see the data after 5 seconds). There are many reasons for this situation, such as network delay, slow database response, excessive CPU and memory usage, etc. The circuit breaker can be a mechanism to protect services or containers. When an abnormality or overload is detected, it temporarily interrupts the inflow of new requests. The creation of a new container replica can be understood as a new container instance created based on the number of original container replicas to share traffic or replace a faulty container. Exemplarily, the new container replica can be consistent with the original container configuration (same image, resource allocation, etc.), and the user request or service call originally directed to the faulty container is transferred to the new container replica.

[0088] In this implementation, by detecting the service response time of the faulty container in real time, it is possible to quickly determine whether the service quality of the container meets the preset standard, set a circuit breaker to quickly block new requests, and prevent the faulty container from continuing to receive traffic, thereby reducing the risk of the problem spreading and saving troubleshooting time. By creating a new container replica and switching traffic to a healthy instance, the service is quickly restored.

[0089] In one embodiment, step S40 also includes: after the system version is updated, if the fault diagnosis result is a transient fault, restarting the fault container and recording the number of restarts; if the number of restarts reaches a preset restart threshold, determining that the fault diagnosis result is an update failure; if the fault diagnosis result is an update failure, rolling back to the previous version until the system returns to normal.

[0090] It should be noted that system version update can be understood as the version iteration of the container operating environment of the system or application, which may include function updates, performance optimization or vulnerability fixes. After the system version is updated, if the fault diagnosis result is a short-term fault, it is necessary to restart the faulty container and record the number of restarts. If the number of restarts at this time reaches the preset restart threshold, the fault diagnosis result is determined to be an update failure, that is, the updated container cannot run normally. Rolling back to the previous version can be understood as restoring the system or container environment from the current version to the previous stable version until the system returns to normal, that is, the container runs normally without frequent restarts.

[0091] It should be noted that the update failure may be due to the fact that the container image itself is bad or the configuration file is missing, causing the container to be in an Error state. In this case, the problem cannot be solved by restarting the system and the service must be rolled back to the last correct version.

[0092] In this implementation, after the system version is updated, temporary anomalies (such as resource competition, network jitter, etc.) can be cleared in time by restarting the faulty container, quickly restoring services, and reducing the impact on user experience. When restarting cannot solve the problem and reaches the threshold, the update failure is promptly determined and a rollback operation is triggered. Through a fast rollback operation, services can be restored in time to avoid users perceiving the failure for a long time.

[0093] Reference Figure 3 , Figure 3 This is a flow chart of the third embodiment of the automated maintenance method for containers of the present application, based on the above Figure 2 The second embodiment shown provides a third embodiment of the automated maintenance method for a container of the present application.

[0094] In the third embodiment, before step S20, the following steps are included:

[0095] Step S11, adding labels to the preprocessed indicator data to obtain labeled data, where the labels are used to distinguish between normal states, potential faults, and actual faults.

[0096] It should be noted that labels can be used to mark the category of preprocessed data to reflect the current operating status of the system. For example, the time before and after a service suddenly crashes can be marked as an abnormal state. Labeled data can be understood as a data set formed by adding labels to preprocessed indicator data, which is used for training and optimizing models.

[0097] Step S12, inputting historical data and annotated data into the initial operation and maintenance model for training, optimizing the model weights through supervised learning methods, and obtaining the target operation and maintenance model.

[0098] It should be noted that historical data can be container performance indicators and related information accumulated over a long period of time during the actual operation of the system. The initial operation and maintenance model is a model used for preliminary training, which is established based on historical data and labeled data and is used to preliminarily classify the operating status of containers. Supervised learning methods can include classification algorithms such as support vector machines (SVM) or random forests, as well as deep learning algorithms such as deep neural networks (DNN) or long short-term memory networks (LSTM). The target operation and maintenance model can be the final model generated after initial operation and maintenance model training and weight optimization, which is used to predict the operating status of containers in real time.

[0099] For example, ensemble learning can be used to quickly identify potential faults, such as using random forests, and then using SVM or DNN for further fault location and diagnosis. Different algorithms can also be used independently according to the amount of data, data dimension, computing resources, and prediction accuracy requirements. Random forests are suitable for fast, interpretable small-scale data scenarios, SVMs are suitable for situations where the feature space is complex but the amount of data is small, and DNNs are suitable for high-precision scenarios with large-scale data and complex feature mapping. The above examples are only used to understand this application and do not constitute a limitation on this embodiment.

[0100] In this embodiment, by adding labels to the preprocessed indicator data, the normal state, potential faults and actual faults are clearly distinguished, accurate training samples are provided for the model, and the model's ability to distinguish different states is improved. Combining historical data and labeled data ensures that the model can learn a variety of operating scenarios. By labeling data in different states, a variety of normal, abnormal and faulty operating conditions are covered to avoid overfitting of the model due to sample bias. The supervised learning method combines labeled data to optimize the model weights, making it more accurate in predicting new data and enhancing the model's adaptability in actual scenarios.

[0101] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the automated maintenance method of the container of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0102] The present application also provides an automated maintenance device for a container, please refer to Figure 4 , the automated maintenance device for the container comprises:

[0103] The data acquisition module 10 is used to monitor the performance indicators of containers in the cluster in real time and obtain pre-processed indicator data;

[0104] A fault classification module 20 is used to input the indicator data into a pre-trained target operation and maintenance model to calculate a status classification result of a container in the current cluster, wherein the status classification result includes potential faults and actual faults;

[0105] A fault prediction module 30 is used to issue an alarm message according to the fault prediction information if the container in the cluster has the potential fault, and perform corresponding maintenance on the target container in combination with the current running status information;

[0106] The fault diagnosis module 40 is configured to perform corresponding maintenance on a target container according to a fault diagnosis result if the container in the cluster has the actual fault.

[0107] The automated maintenance device for containers provided in the present application adopts the automated maintenance method for containers in the above-mentioned embodiments, and can solve the technical problem of low maintenance efficiency of the container system. Compared with the prior art, the beneficial effects of the automated maintenance device for containers provided in the present application are the same as the beneficial effects of the automated maintenance method for containers provided in the above-mentioned embodiments, and other technical features in the automated maintenance device for containers are the same as the features disclosed in the above-mentioned embodiment methods, and are not described in detail here.

[0108] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0109] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0110] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned automated maintenance method for a container when executed by a processor.

[0111] The computer program product provided by the present application can solve the technical problem of low maintenance efficiency of the container system. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the automated maintenance method of the container provided by the above embodiment, which will not be described in detail here.

[0112] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An automated maintenance method for a container, characterized in that: The method includes: Monitor container performance indicators in the cluster in real time and obtain pre-processed indicator data; Input the indicator data into the pre-trained target operation and maintenance model to calculate the status classification results of the containers in the current cluster, wherein the status classification results include potential faults and actual faults; If the container in the cluster has the potential fault, an alarm message is issued according to the fault prediction information, and corresponding maintenance is performed on the target container in combination with the current running status information; If the actual fault exists in the container in the cluster, corresponding maintenance is performed on the target container according to the fault diagnosis result.

2. The method according to claim 1, characterized in that The state classification result includes a normal state, a potential fault, and an actual fault. The step of inputting the indicator data into a pre-trained target operation and maintenance model to calculate the state classification result of the container in the current cluster includes: Obtain data features through indicator data, input preset weights and the data features into the target operation and maintenance model, and output the operation status information of the current cluster; Based on the operation status information, a status classification result of the container in the current cluster is generated, and the status classification result is identified as the normal state, the potential fault or the actual fault.

3. The method according to claim 2, characterized in that The current running status information includes the current number of service requests. If the container in the cluster has the potential fault, an alarm is issued according to the fault prediction information, and corresponding maintenance is performed on the target container in combination with the current running status information, including: If the container in the cluster has the potential fault, an alarm message is issued according to the fault prediction information; Detecting whether the current number of service requests reaches a preset request number threshold; If the current number of service requests reaches a preset threshold of the number of requests, the containers in the cluster are ranked according to the frequency of container use to determine a priority container and a secondary container; By reducing the number of container replicas of the secondary container, resources are allocated to the priority container.

4. The method according to claim 3, characterized in that After the step of detecting that the current number of service requests reaches a threshold of a preset number of requests, the method further includes: If the current service request quantity reaches a preset request quantity threshold, a rate limiter is configured for the target container, and real-time monitoring information of the rate limiter is obtained; The current limiting parameters are dynamically adjusted according to the real-time monitoring information.

5. The method according to claim 1, characterized in that The step of performing corresponding maintenance on the target container according to the fault diagnosis result if the actual fault exists in the container in the cluster includes: If the actual fault exists in the container in the cluster, locate the faulty container according to the fault diagnosis result; If the fault diagnosis result is a transient fault, restart the faulty container and record the number of restarts; If the number of restarts reaches the preset restart threshold, the corresponding traffic of the failed container is directed to the new container copy.

6. The method according to claim 5, characterized in that After the step of locating the fault container according to the fault diagnosis result, the method includes: Check whether the service response time of the faulty container reaches the preset response time; If the service response time of the faulty container reaches the preset response time, a circuit breaker is set to block new requests, a new container copy is created, and the corresponding traffic of the faulty container is directed to the new container copy.

7. The method according to claim 5, characterized in that The step of performing corresponding maintenance on the target container according to the fault diagnosis result if the actual fault exists in the container in the cluster includes: After the system version is updated, if the fault diagnosis result is a transient fault, restart the fault container and record the number of restarts; If the number of restarts reaches a preset restart threshold, determining the fault diagnosis result as an update failure; If the fault diagnosis result is that the update failed, roll back to the previous version until the system returns to normal.

8. The method according to any one of claims 1 to 7, characterized in that Before the step of inputting the indicator data into the pre-trained target operation and maintenance model, the method includes: Adding labels to the preprocessed indicator data to obtain labeled data, wherein the labels are used to distinguish between normal states, potential faults, and actual faults; The historical data and the labeled data are input into the initial operation and maintenance model for training, and the model weights are optimized through a supervised learning method to obtain the target operation and maintenance model.

9. An automated maintenance device for a container, characterized in that: The device comprises: The data acquisition module is used to monitor the performance indicators of containers in the cluster in real time and obtain pre-processed indicator data; A fault classification module is used to input the indicator data into a pre-trained target operation and maintenance model to calculate the status classification results of the containers in the current cluster, wherein the status classification results include potential faults and actual faults; A fault prediction module is used to issue an alarm message according to the fault prediction information if the container in the cluster has the potential fault, and perform corresponding maintenance on the target container in combination with the current running status information; The fault diagnosis module is used to perform corresponding maintenance on the target container according to the fault diagnosis result if the container in the cluster has the actual fault.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the steps of the method for automated maintenance of a container according to any one of claims 1 to 8.

Citation Information

Cited By

  • Cluster disaster recovery method and device based on hyper-converged cluster

    CN120474897A