Container cloud platform management side exception handling method and device, medium and equipment

Abstract

The invention provides a container cloud platform management side exception handling method and device, a medium and equipment. The method comprises the following steps: checking whether long connection exists in a container set or not; if the long connection exists, judging whether the long connection is influenced by retransmission times when the opposite terminal is abnormally disconnected; if the retransmission timeout is influenced by the retransmission times, performing optimization configuration on the reconnection timeout so as to shorten the duration for the local end to detect that the opposite end is abnormal; adding a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism in a routing forwarding rule in a service product provided by the container set so as to shorten the duration for a home terminal to detect that an opposite terminal is abnormal; and verifying the configuration effect. According to the invention, when the opposite terminal is abnormal, the local terminal can identify the abnormity of the opposite terminal in time, so that the response time is shortened, the access of a management side console can be quickly recovered, the high availability of the management side of the container cloud platform is improved, and reliable guarantee is provided for continuous operation of the container cloud platform.

Description

Technical Field

[0001] The present invention relates to the technical field of container cloud platforms, and in particular to a method, apparatus, medium, and equipment for handling exceptions on the management side of a container cloud platform. Background Art

[0002] The container cloud platform can provide high-performance, scalable container application management services, support the lifecycle management of enterprise-level containerized applications, and provide a one-stop cloud application empowerment platform for cloud-native applications. It supports X86, ARM and MIPS heterogeneous environments, supports multiple container runtimes, supports multiple container networks, and supports multiple container storages, creating the best containerized application operating environment for users.

[0003] The container cloud platform management console is used to centrally view and manage cloud platform products and services. It also provides multi-tenant management capabilities. Through the console, users can purchase, view, and use cloud platform products and services, manage service offerings, and monitor the performance and resource utilization of cloud service offerings. Therefore, the ability of the container cloud platform management console to quickly identify anomalies when failures occur is crucial for improving availability. Summary of the Invention

[0004] In response to at least one of the above technical problems, embodiments of the present invention provide a method, apparatus, medium, and device for handling exceptions on the management side of a container cloud platform.

[0005] According to the first aspect, the container cloud platform management side exception handling method provided by the embodiment of the present invention includes:

[0006] Check whether there are persistent connections in the container set;

[0007] If a persistent connection exists, determine whether the persistent connection will be affected by the number of retransmissions when the peer end abnormally disconnects; if so, optimize the reconnection timeout to shorten the time it takes for the local end to detect an abnormality on the peer end;

[0008] A keep-alive detection mechanism and an abnormal instance detection and elimination mechanism are added to the routing and forwarding rules of the service product provided by the container set to shorten the time it takes for the local end to detect an abnormality in the other end; wherein the keep-alive detection mechanism is to send a keep-alive detection packet to the other end once every first preset time period, and if no confirmation information returned by the other end is received after N consecutive keep-alive detection packets are sent, the connection between the local end and the other end is closed; the abnormal instance detection and elimination mechanism is to close the abnormal backend container set within a third preset time period if abnormal information appears M times consecutively within a second preset time period, so as to stop the abnormal backend container set from providing services; M and N are integers greater than 1;

[0009] After optimizing the reconnection timeout, the effect is verified by checking whether the time it takes for the local end to detect an abnormality on the other end is shortened; after adding a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism to the routing forwarding rules, the effect is verified by checking whether the time it takes for the local end to detect an abnormality on the other end is shortened.

[0010] In one embodiment, checking whether there is a persistent connection in the container set includes:

[0011] Determine the node where the container set is located;

[0012] After logging into the node where the container set is located, enter the network namespace of the container set;

[0013] It is queried in the network namespace whether there is a keep-alive type connection. If so, the long connection exists.

[0014] In one embodiment, determining whether the long connection is affected by the number of retransmissions when the peer end is abnormally disconnected includes:

[0015] Determine whether a timeout limit is configured in the custom business code of the business container corresponding to the persistent connection;

[0016] If so, it is determined that the long connection is not affected by the number of retransmissions;

[0017] Otherwise, determining whether the connection state of the long connection is a keep-alive state or a retransmission state;

[0018] If it is in the keep-alive state, it is determined that the long connection is not affected by the number of retransmissions;

[0019] If it is in the retransmission state, determine whether the number of retransmissions of the long connection has been increasing within a specific time and the number of retransmissions is always less than the upper limit of the number of retransmissions; if so, determine that the long connection is affected by the number of retransmissions; otherwise, determine that the long connection is not affected by the number of retransmissions.

[0020] In one embodiment, optimizing the configuration of the reconnection timeout includes:

[0021] An initialization container is set in the container set, and the number of retransmissions configured in the initialization container is less than the number of retransmissions configured in the service container in the same container set, so that in a retransmission state, the service container can receive service packets retransmitted by the local end according to the number of retransmissions configured in the initialization container.

[0022] In one embodiment, the optimizing configuration of the reconnection timeout further includes:

[0023] Setting detection configuration information in the keep-alive state in the initialization container; wherein the detection configuration information includes a probe detection interval, an upper limit on the number of probe detections, and a time to start the probe;

[0024] Among them, the probe detection interval configured in the initialization container is less than the probe detection interval configured in the business container, the upper limit of the probe detection times configured in the initialization container is less than the upper limit of the probe detection times configured in the business container, and the start time of the probe configured in the initialization container is earlier than the start time of the probe configured in the business container, so that the business container can receive the detection packet sent by this end according to the detection configuration information configured in the initialization container in the keep-alive state.

[0025] In one embodiment, N in the keep-alive detection mechanism is the upper limit of the probe detection times; the abnormal instance detection and elimination mechanism is also configured with a maximum ratio and a minimum healthy instance ratio; wherein the maximum ratio is the maximum ratio of the set of faulty containers that are shut down in the load balancing pool; when the set of healthy containers in the load balancing pool is less than the minimum healthy instance ratio, the abnormal instance detection and elimination mechanism is disabled.

[0026] In one embodiment, after adding the keep-alive detection mechanism and the abnormal instance detection and elimination mechanism to the routing forwarding rule, verifying whether the time taken by the local end to detect an abnormality at the other end is shortened includes:

[0027] Determine the node where the container set is located; log in to the node where the container set is located and enter the network namespace; add an IP list discard rule in the network namespace, where the IP list discard rule is to discard the container set IP of the ingress gateway;

[0028] Simulate multiple requests to the container set corresponding to the container set IP address.

[0029] Observe the log of the ingress gateway and determine, based on the log, whether the request can be forwarded to the container set if the exception information appears M times continuously within the second preset time period;

[0030] If not, the IP list discarding rule is deleted, and it is determined based on the log whether the request can be forwarded to the backend container set after the third preset time period. If so, it is determined that after adding an abnormal instance detection and elimination mechanism to the routing forwarding rule, the verification of the shortened time taken by this end to detect an abnormality on the other end is passed.

[0031] According to the second aspect, the container cloud platform management side exception handling device provided by the embodiment of the present invention includes:

[0032] A first checking module is used to check whether there is a persistent connection in the container set;

[0033] A first judgment module is configured to, if a persistent connection exists, determine whether the persistent connection will be affected by the number of retransmissions when the peer end abnormally disconnects; if so, optimize the reconnection timeout to shorten the time it takes for the local end to detect an abnormality at the peer end;

[0034] A mechanism adding module is used to add a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism to the routing and forwarding rules in the service products provided by the container set, so as to shorten the time it takes for the local end to detect an abnormality in the other end; wherein the keep-alive detection mechanism is to send a keep-alive detection packet to the other end once every first preset time period, and if no confirmation information returned by the other end is received after N consecutive keep-alive detection packets are sent, the connection between the local end and the other end is closed; the abnormal instance detection and elimination mechanism is to close the abnormal backend container set within a third preset time period if abnormal information appears M times consecutively within a second preset time period, so as to stop the abnormal backend container set from providing services; M and N are integers greater than 1;

[0035] The effect verification module is used to verify whether the time taken by the local end to detect an abnormality at the other end is shortened after the optimized configuration of the reconnection timeout is configured; after adding the keep-alive detection mechanism and the abnormal instance detection and elimination mechanism to the routing forwarding rules, the effect verification is used to verify whether the time taken by the local end to detect an abnormality at the other end is shortened.

[0036] In one embodiment, the first inspection module is specifically configured to: determine the node where the container set is located; after logging into the node where the container set is located, enter the network namespace of the container set; and query whether a keep-alive type connection exists in the network namespace. If so, the long connection exists.

[0037] In one embodiment, the process of the first judgment module judging whether the long connection will be affected by the number of retransmissions when the other end is abnormally disconnected includes: judging whether a timeout limit is configured in the custom business code of the business container corresponding to the long connection; if so, determining that the long connection is not affected by the number of retransmissions; otherwise, judging whether the connection state of the long connection is a keep-alive state or a retransmission state; if it is a keep-alive state, determining that the long connection is not affected by the number of retransmissions; if it is a retransmission state, judging whether the number of retransmissions of the long connection has been increasing within a specific time and the number of retransmissions is always less than the upper limit of the number of retransmissions; if so, determining that the long connection is affected by the number of retransmissions; otherwise, determining that the long connection is not affected by the number of retransmissions.

[0038] In one embodiment, the process of optimizing the reconnection timeout by the first judgment module includes: setting an initialization container in the container set, and the number of retransmissions configured in the initialization container is less than the number of retransmissions configured in the service container in the same container set, so that the service container can receive the service packets retransmitted by the local end according to the number of retransmissions configured in the initialization container in the retransmission state.

[0039] In one embodiment, the process of optimizing the configuration of the reconnection timeout by the first judgment module also includes: setting the detection configuration information in the keep-alive state in the initialization container; wherein the detection configuration information includes the probe detection interval, the upper limit of the probe detection times and the time to start the probe; wherein the probe detection interval configured in the initialization container is less than the probe detection interval configured in the business container, the upper limit of the probe detection times configured in the initialization container is less than the upper limit of the probe detection times configured in the business container, and the time to start the probe configured in the initialization container is earlier than the time to start the probe configured in the business container, so that the business container can receive the detection packet sent by the local end according to the detection configuration information configured in the initialization container in the keep-alive state.

[0040] In one embodiment, N in the keep-alive detection mechanism is the upper limit of the probe detection times; the abnormal instance detection and elimination mechanism is also configured with a maximum ratio and a minimum healthy instance ratio; wherein the maximum ratio is the maximum ratio of the set of faulty containers that are shut down in the load balancing pool; when the set of healthy containers in the load balancing pool is less than the minimum healthy instance ratio, the abnormal instance detection and elimination mechanism is disabled.

[0041] In one embodiment, the effect verification module is specifically used to: determine the node where the container set is located; log in to the node where the container set is located, and enter the network namespace; add an IP list discarding rule in the network namespace, and the IP list discarding rule is to discard the container set IP of the entry gateway; simulate sending requests to the container set corresponding to the container set IP multiple times; observe the log of the entry gateway, and determine according to the log whether the request can be forwarded to the container set when M times of abnormal information appears continuously within the second preset time length; if not, delete the IP list discarding rule, and judge according to the log whether the request can be forwarded to the backend container set after the third preset time length. If so, it is determined that after adding the abnormal instance detection and elimination mechanism in the routing forwarding rule, the verification of the shortened time length for the local end to detect the abnormality of the other end is passed.

[0042] According to a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method provided in the first aspect.

[0043] According to a fourth aspect, an embodiment of the present invention provides a computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method provided in the first aspect is implemented.

[0044] The container cloud platform management side exception handling method, apparatus, medium, and equipment provided by the embodiment of the present invention first checks whether there is a long connection in the container set. If so, it determines whether the long connection will be affected by the number of retransmissions when the other end is abnormally disconnected. If it is affected by the number of retransmissions, the reconnection timeout is optimized; a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism are added to the routing forwarding rules in the service product provided by the container set; finally, the effect is verified by shortening the time it takes for the local end to detect an abnormality on the other end. Since the embodiment of the present invention optimizes the reconnection timeout and adds a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism, the local end can promptly identify the abnormality of the other end when an abnormality occurs on the other end, thereby shortening the response time, helping to quickly restore access to the management side console, improving the high availability of the container cloud platform management side, and providing reliable protection for the continuous operation of the container cloud platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of a method for handling exceptions on the management side of a container cloud platform according to one embodiment of the present invention;

[0046] Figure 2 This is an overall flow chart of a method for handling exceptions on the container cloud platform management side in one embodiment of the present invention;

[0047] Figure 3 This is a structural block diagram of an exception handling device on the management side of a container cloud platform in one embodiment of the present invention. DETAILED DESCRIPTION

[0048] In the first aspect, the embodiment of the present invention provides a method for handling exceptions on the management side of a container cloud platform, see Figure 1 and Figure 2 The method includes the following steps S110 to S140:

[0049] S110. Check whether there is a persistent connection in the container set;

[0050] In one embodiment, the step of checking whether there is a persistent connection in the container set in S110 may include the following steps:

[0051] Determine the node where the container set is located;

[0052] After logging into the node where the container set is located, enter the network namespace of the container set;

[0053] It is queried in the network namespace whether there is a keep-alive type connection. If so, the long connection exists.

[0054] The container set is a pod, and the container is a container. A container set includes one or more containers.

[0055] The keepalive type is the keepalive type.

[0056] It can be seen that in the embodiment of the present invention, the keep-alive type connection is regarded as a long connection.

[0057] For long connections, you can use S120 and S130 to configure. Whether it is a long connection or a short connection, you can use S130 to configure it.

[0058] S120. If a long connection exists, determine whether the long connection will be affected by the number of retransmissions when the peer end is abnormally disconnected; if it is affected by the number of retransmissions, optimize the reconnection timeout to shorten the time it takes for the local end to detect the abnormality of the peer end;

[0059] In one embodiment, determining whether the long connection is affected by the number of retransmissions when the peer end is abnormally disconnected may include the following steps:

[0060] 1. Determine whether a timeout limit is configured in the custom business code of the business container corresponding to the persistent connection;

[0061] 2. If yes, then determine that the long connection is not affected by the number of retransmissions;

[0062] 3. Otherwise, determine whether the connection state of the long connection is a keep-alive state or a retransmission state;

[0063] 4. If it is in the keep-alive state, it is determined that the long connection is not affected by the number of retransmissions;

[0064] 5. If it is in the retransmission state, determine whether the number of retransmissions of the long connection has been increasing within a specific time and the number of retransmissions is always less than the upper limit of the number of retransmissions; if so, determine that the long connection is affected by the number of retransmissions; otherwise, determine that the long connection is not affected by the number of retransmissions.

[0065] It is understandable that if a timeout limit is configured in the custom business code of the business container, the business code will detect whether it needs to actively disconnect, which takes a very short time. If the timeout limit is not configured in the custom business code, it is necessary to determine whether the state of the long connection is the keepalive state or the retransmission state, that is, the on state. If it is in the keepalive state, it is necessary to send a keepalive detection packet to detect whether the other end is normal. The number of detection packets sent is not related to the number of retransmissions, so it is not affected by the number of retransmissions. If it is in the on state, it is determined whether the number of retransmissions of the long connection has been increasing within a certain period of time, and is less than the upper limit of the retransmission number. If so, it is considered to be affected by the number of retransmissions, otherwise it is not affected by the number of retransmissions. Among them, what is sent during retransmission is the business packet, not the detection packet.

[0066] In one embodiment, optimizing the configuration of the reconnection timeout may include:

[0067] An initialization container is set in the container set, and the number of retransmissions configured in the initialization container is less than the number of retransmissions configured in the service container in the same container set, so that in a retransmission state, the service container can receive service packets retransmitted by the local end according to the number of retransmissions configured in the initialization container.

[0068] It can be seen that for the filtered long connections that are affected by the number of retransmissions when an abnormal disconnection occurs on the other end, an initialization container is added to the container and the number of retransmissions in the configuration information of the initialization container is less than the number of retransmissions configured in the business container in the same container set. In this way, there is no need to modify the configuration information of the business container in the same container set. In the retransmission state, if the local end continuously sends business packets for the number of retransmissions configured by the initialization container to the business container of the other end without receiving any confirmation feedback, the long connection is closed.

[0069] For example, in the same container set, the number of retransmissions in the business container is 15, while the number of retransmissions in the initialization container is 8. Therefore, if the local end does not receive confirmation feedback after sending business packages for 8 consecutive times, the persistent connection will be closed, instead of closing the persistent connection after not receiving confirmation feedback after sending business packages for 15 consecutive times. This allows the local end to close abnormal persistent connections in a timely manner.

[0070] In one embodiment, the optimizing configuration of the reconnection timeout may further include:

[0071] Setting detection configuration information in the keep-alive state in the initialization container; wherein the detection configuration information includes a probe detection interval, an upper limit on the number of probe detections, and a time to start the probe;

[0072] Among them, the probe detection interval configured in the initialization container is less than the probe detection interval configured in the business container, the upper limit of the probe detection times configured in the initialization container is less than the upper limit of the probe detection times configured in the business container, and the start time of the probe configured in the initialization container is earlier than the start time of the probe configured in the business container, so that the business container can receive the detection packet sent by this end according to the detection configuration information configured in the initialization container in the keep-alive state.

[0073] In other words, the keepalive probe interval, probe detection limit, and probe startup time are configured in the initialization container. These configurations are smaller than the corresponding configurations in the service container. This allows the local end to promptly detect any anomalies in the peer end during the keepalive state and close the persistent connection promptly. This allows the initialization container configuration to affect the service container, eliminating the need to modify the configuration of each service container individually, making it more convenient and reliable.

[0074] S130. Add a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism to the routing and forwarding rules of the service product provided by the container set to shorten the time it takes for the local end to detect an abnormality in the other end; wherein the keep-alive detection mechanism is to send a keep-alive detection packet to the other end once every first preset time period, and if no confirmation information is received from the other end after sending N consecutive keep-alive detection packets, then close the connection between the local end and the other end; the abnormal instance detection and elimination mechanism is to close the abnormal backend container set within a third preset time period if abnormal information appears M times consecutively within a second preset time period, so as to stop the abnormal backend container set from providing services; M and N are integers greater than 1;

[0075] Wherein, N in the keep-alive detection mechanism may be the upper limit of the probe detection times.

[0076] For example, if no confirmation information is received from the other end after sending 8 keep-alive detection packets to the other end in succession, the connection between the local end and the other end will be closed, and the interval between sending two detection packets is 3 seconds.

[0077] The abnormal instance detection and elimination mechanism can also be configured with a maximum ratio and a minimum healthy instance ratio; wherein the maximum ratio is the maximum ratio of the set of faulty containers that are shut down in the load balancing pool; when the set of healthy containers in the load balancing pool is less than the minimum healthy instance ratio, the abnormal instance detection and elimination mechanism is disabled.

[0078] For example, in the abnormal instance detection and elimination mechanism, if an abnormality occurs three times in a row within 10 seconds, the abnormal backend container set will be shut down within the next 5 minutes. In this way, the abnormal backend container set will stop providing services within 5 minutes and can only provide services after 5 minutes.

[0079] Specifically, the exception may be a 502, 503, or 504 error.

[0080] A 502 error indicates a gateway timeout or gateway error. When a server acts as a gateway or proxy and receives an invalid response from an upstream server, it returns a 502 error. This typically indicates a malfunction on the upstream server or a network interruption, preventing it from responding to the request. Possible solutions include checking the server and proxy configuration, clearing your browser cache and cookies, and checking your network connection.

[0081] A 503 error indicates "Service Unavailable." This is usually caused by the server being temporarily unable to process the request, possibly due to server overload, maintenance, or upgrades. The client can try sending the request again later or contact the website administrator for more information. Solutions include waiting for a while and then trying again, or contacting the website administrator.

[0082] A 504 error indicates a gateway timeout. This means that a server acting as a gateway or proxy failed to receive a timely response from an upstream or secondary server when attempting to execute a request. Possible solutions include checking network connectivity, clearing browser cache and cookies, and checking website status.

[0083] S140. After optimizing the configuration of the reconnection timeout, verify whether the time taken by the local end to detect an abnormality at the other end is shortened; after adding a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism to the routing forwarding rule, verify whether the time taken by the local end to detect an abnormality at the other end is shortened.

[0084] That is to say, after optimizing the reconnection timeout configuration and adding the keep-alive detection mechanism and the abnormal instance detection and elimination mechanism to the routing forwarding rules, the effects need to be verified to verify whether the local end can close the long connection in time when an abnormality occurs on the other end.

[0085] In one embodiment, after adding the keep-alive detection mechanism and the abnormal instance detection and elimination mechanism to the routing forwarding rule, verifying whether the time taken by the local end to detect the abnormality of the other end is shortened may include S141 to S144:

[0086] S141. Determine the node where the container set is located; log in to the node where the container set is located and enter the network namespace; add an IP list discard rule in the network namespace, where the IP list discard rule is to discard the container set IP of the ingress gateway;

[0087] S142. Simulate sending multiple requests to the container set corresponding to the container set IP address.

[0088] S143. Observe the log of the ingress gateway and determine, based on the log, whether the request can be forwarded to the container set if the exception information appears M times continuously within the second preset time period.

[0089] S144. If not, the IP list discarding rule is deleted, and it is determined based on the log whether the request can be forwarded to the backend container set after the third preset time period. If so, it is determined that after adding an abnormal instance detection and elimination mechanism to the routing forwarding rule, the verification of the shortened time taken by the local end to detect an abnormality on the other end is passed.

[0090] It is understandable that after adding the IP list discarding rule, the container set corresponding to the container set IP cannot receive the request, so an exception will occur. At this time, simulate sending requests to this container set multiple times. By observing the log of the ingress gateway, it is determined whether the request can be forwarded to the container set when the exception information appears M times continuously within the second preset time period. If the request cannot be forwarded to the container set when the exception information appears M times continuously within the second preset time period, it means that this end has identified that the other end has an exception. At this time, the other end cannot provide services, so the IP list discarding rule is deleted, and then it is determined whether the request can be forwarded to the backend container set after the third preset time period. If it can, it means that the other end has recovered to normal and can provide services. At this point, the effectiveness of the abnormal instance detection and elimination mechanism has been verified.

[0091] It is understandable that the verification process for the effectiveness of the keepalive detection mechanism and the verification of the effectiveness of the optimized configuration of the reconnection timeout are similar. After the verification is passed, the above method can be used to handle the exception, ensuring that the local end can promptly identify the abnormality of the other end.

[0092] Here, the local end refers to the container cloud platform management console.

[0093] Keepalive is an HTTP persistent connection mechanism used to maintain connections during network requests to improve efficiency. The main difference between a keepalive connection and a persistent connection is that a keepalive connection is closed immediately after receiving a response, while a persistent connection remains open for extended periods of time, reducing concurrency pressure on the server. In a keepalive connection, retransmission typically refers to heartbeat messages at the TCP protocol level. These messages are used to detect whether the peer has crashed or is unreachable, thereby initiating a reconnection process.

[0094] It can be seen that currently, in scenarios such as the downtime of the node server on the cloud platform management side, the client of a service with a long connection will remain connected when the other end is abnormally disconnected. Since the long connection cannot be disconnected in time, the service will be unavailable for a long time. The embodiment of the present invention can close the long connection in a timely manner, add a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism, quickly identify the abnormality of the other end, shorten the response time, help to quickly restore access to the management side console, improve the high availability of the container cloud platform management side, and provide reliable protection for the continuous operation of the container cloud platform.

[0095] In the second aspect, the embodiment of the present invention provides a container cloud platform management side exception handling device, see Figure 3 , the apparatus 100 comprises:

[0096] A first checking module 110 is configured to check whether there is a persistent connection in the container set;

[0097] The first determination module 120 is configured to, if a persistent connection exists, determine whether the persistent connection will be affected by the number of retransmissions when the peer end abnormally disconnects; if so, optimize the reconnection timeout to shorten the time it takes for the local end to detect an abnormality at the peer end;

[0098] The mechanism adding module 130 is configured to add a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism to the routing and forwarding rules in the service product provided by the container set, so as to shorten the time it takes for the local end to detect an abnormality in the other end; wherein the keep-alive detection mechanism is to send a keep-alive detection packet to the other end once every first preset time period, and if no confirmation information returned by the other end is received after N consecutive keep-alive detection packets are sent, the connection between the local end and the other end is closed; the abnormal instance detection and elimination mechanism is to close the abnormal backend container set within a third preset time period if abnormal information appears M times consecutively within a second preset time period, so as to stop the abnormal backend container set from providing services; M and N are integers greater than 1;

[0099] The effect verification module 140 is used to verify the effect of whether the time taken by the local end to detect an abnormality at the other end is shortened after the optimized configuration of the reconnection timeout; and to verify the effect of whether the time taken by the local end to detect an abnormality at the other end is shortened after adding a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism to the routing forwarding rules.

[0100] In one embodiment, the first inspection module is specifically configured to: determine the node where the container set is located; after logging into the node where the container set is located, enter the network namespace of the container set; and query whether a keep-alive type connection exists in the network namespace. If so, the long connection exists.

[0101] In one embodiment, the process of the first judgment module judging whether the long connection will be affected by the number of retransmissions when the other end is abnormally disconnected includes: judging whether a timeout limit is configured in the custom business code of the business container corresponding to the long connection; if so, determining that the long connection is not affected by the number of retransmissions; otherwise, judging whether the connection state of the long connection is a keep-alive state or a retransmission state; if it is a keep-alive state, determining that the long connection is not affected by the number of retransmissions; if it is a retransmission state, judging whether the number of retransmissions of the long connection has been increasing within a specific time and the number of retransmissions is always less than the upper limit of the number of retransmissions; if so, determining that the long connection is affected by the number of retransmissions; otherwise, determining that the long connection is not affected by the number of retransmissions.

[0102] In one embodiment, the process of optimizing the reconnection timeout by the first judgment module includes: setting an initialization container in the container set, and the number of retransmissions configured in the initialization container is less than the number of retransmissions configured in the service container in the same container set, so that the service container can receive the service packets retransmitted by the local end according to the number of retransmissions configured in the initialization container in the retransmission state.

[0103] In one embodiment, the process of optimizing the configuration of the reconnection timeout by the first judgment module also includes: setting the detection configuration information in the keep-alive state in the initialization container; wherein the detection configuration information includes the probe detection interval, the upper limit of the probe detection times and the time to start the probe; wherein the probe detection interval configured in the initialization container is less than the probe detection interval configured in the business container, the upper limit of the probe detection times configured in the initialization container is less than the upper limit of the probe detection times configured in the business container, and the time to start the probe configured in the initialization container is earlier than the time to start the probe configured in the business container, so that the business container can receive the detection packet sent by the local end according to the detection configuration information configured in the initialization container in the keep-alive state.

[0104] In one embodiment, N in the keep-alive detection mechanism is the upper limit of the probe detection times; the abnormal instance detection and elimination mechanism is also configured with a maximum ratio and a minimum healthy instance ratio; wherein the maximum ratio is the maximum ratio of the set of faulty containers that are shut down in the load balancing pool; when the set of healthy containers in the load balancing pool is less than the minimum healthy instance ratio, the abnormal instance detection and elimination mechanism is disabled.

[0105] In one embodiment, the effect verification module is specifically used to: determine the node where the container set is located; log in to the node where the container set is located, and enter the network namespace; add an IP list discarding rule in the network namespace, and the IP list discarding rule is to discard the container set IP of the entry gateway; simulate sending requests to the container set corresponding to the container set IP multiple times; observe the log of the entry gateway, and determine according to the log whether the request can be forwarded to the container set when M times of abnormal information appears continuously within the second preset time length; if not, delete the IP list discarding rule, and judge according to the log whether the request can be forwarded to the backend container set after the third preset time length. If so, it is determined that after adding the abnormal instance detection and elimination mechanism in the routing forwarding rule, the verification of the shortened time length for the local end to detect the abnormality of the other end is passed.

[0106] It is understandable that the explanation, specific implementation, beneficial effects, examples, etc. of the relevant contents in the device provided by the embodiment of the present invention can be found in the corresponding parts of the method provided in the first aspect, and will not be repeated here.

[0107] In a third aspect, an embodiment of the present invention provides a computer-readable medium having computer instructions stored thereon. When the computer instructions are executed by a processor, the processor executes the method provided in the first aspect.

[0108] Specifically, a system or device equipped with a storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program codes stored in the storage medium.

[0109] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.

[0110] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0111] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0112] In addition, it can be understood that the program code read from the storage medium is written into a memory provided in an expansion board inserted into the computer or into a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above embodiments.

[0113] It is understandable that the explanation, specific implementation methods, beneficial effects, examples, etc. of the relevant contents in the computer-readable medium provided in the embodiment of the present invention can be found in the corresponding parts of the method provided in the first aspect, and will not be repeated here.

[0114] In a fourth aspect, an embodiment of this specification provides a computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method in any one of the embodiments in the specification.

[0115] It is understandable that the explanation, specific implementation, beneficial effects, examples, etc. of the relevant contents in the computing device provided by the embodiment of the present invention can be found in the corresponding parts of the method provided in the first aspect, and will not be repeated here.

[0116] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0117] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the present invention may be implemented using hardware, software, widgets, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0118] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A container cloud platform management side exception handling method, characterized in that: include: Check whether there is a persistent connection in the container set; If there is a long connection, determine whether the long connection will be affected by the number of retransmissions when the other end is abnormally disconnected; if it is affected by the number of retransmissions, optimize the reconnection timeout to shorten the time it takes for the local end to detect an abnormality on the other end; A keep-alive detection mechanism and an abnormal instance detection and elimination mechanism are added to the routing forwarding rules in the service product provided by the container set to shorten the time it takes for the local end to detect an abnormality in the other end; wherein the keep-alive detection mechanism is to send a keep-alive detection packet to the other end once every first preset time period, and if no confirmation information returned by the other end is received after N consecutive keep-alive detection packets are sent, the connection between the local end and the other end is closed; the abnormal instance detection and elimination mechanism is to close the abnormal backend container set within a third preset time period if abnormal information appears M times consecutively within a second preset time period, so as to stop the abnormal backend container set from providing services; M and N are integers greater than 1; After optimizing the configuration of the reconnection timeout, the effect is verified by whether the time taken by the local end to detect an abnormality at the other end is shortened; after adding a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism to the routing forwarding rules, the effect is verified by whether the time taken by the local end to detect an abnormality at the other end is shortened.

2. The method according to claim 1, characterized in that The step of checking whether there is a persistent connection in the container set includes: Determine the node where the container set is located; After logging in to the node where the container set is located, enter the network namespace of the container set; It is queried in the network namespace whether there is a keep-alive type connection, and if so, the long connection exists.

3. The method according to claim 1, characterized in that The determining whether the long connection is affected by the number of retransmissions when the other end is abnormally disconnected includes: Determine whether a timeout limit is configured in the custom service code of the service container corresponding to the long connection; If yes, it is determined that the long connection is not affected by the number of retransmissions; Otherwise, determining whether the connection state of the long connection is a keep-alive state or a retransmission state; If it is in the keep-alive state, it is determined that the long connection is not affected by the number of retransmissions; If it is in the retransmission state, determine whether the number of retransmissions of the long connection within a specific time has been increasing and the number of retransmissions is always less than the upper limit of the number of retransmissions; if so, determine that the long connection is affected by the number of retransmissions; otherwise, determine that the long connection is not affected by the number of retransmissions.

4. The method according to claim 1, characterized in that: The optimizing configuration of the reconnection timeout includes: An initialization container is set in the container set, and the number of retransmissions configured in the initialization container is less than the number of retransmissions configured in the service container in the same container set, so that in the retransmission state, the service container can receive the service packet retransmitted by the local end according to the number of retransmissions configured in the initialization container.

5. The method according to claim 4, characterized in that The optimizing configuration of the reconnection timeout also includes: Setting detection configuration information in the keep-alive state in the initialization container; wherein the detection configuration information includes a probe detection interval, an upper limit of the number of probe detections, and a time to start the probe; Among them, the probe detection interval configured in the initialization container is smaller than the probe detection interval configured in the business container, the upper limit of the probe detection times configured in the initialization container is smaller than the upper limit of the probe detection times configured in the business container, and the start time of the probe configured in the initialization container is earlier than the start time of the probe configured in the business container, so that the business container can receive the detection packet sent by the local end according to the detection configuration information configured in the initialization container in the keep-alive state.

6. The method according to claim 4, characterized in that N in the keep-alive detection mechanism is the upper limit of the probe detection times; the abnormal instance detection and elimination mechanism is also configured with a maximum ratio and a minimum healthy instance ratio; wherein the maximum ratio is the maximum ratio of the set of faulty containers that are closed in the load balancing pool; when the set of healthy containers in the load balancing pool is less than the minimum healthy instance ratio, the abnormal instance detection and elimination mechanism is disabled.

7. The method according to claim 1, characterized in that After adding the keep-alive detection mechanism and the abnormal instance detection and elimination mechanism to the routing forwarding rule, the effect of verifying whether the time taken by the local end to detect the abnormality of the other end is shortened includes: Determine the node where the container set is located; log in to the node where the container set is located, and enter the network namespace; add an IP list discard rule in the network namespace, and the IP list discard rule is to discard the container set IP of the ingress gateway; Simulate sending multiple requests to the container set corresponding to the container set IP; Observe the log of the ingress gateway, and determine, based on the log, whether the request can be forwarded to the container set if the abnormal information appears M times continuously within the second preset time period; If not, the IP list discarding rule is deleted, and it is determined based on the log whether the request can be forwarded to the backend container set after the third preset time period. If so, it is determined that after adding an abnormal instance detection and elimination mechanism to the routing forwarding rule, the verification that the time taken by this end to detect an abnormality on the other end is shortened is passed.

8. A container cloud platform management side exception handling device, characterized in that: include: A first checking module is used to check whether there is a persistent connection in the container set; The first judgment module is used to judge whether the long connection will be affected by the number of retransmissions when the other end is abnormally disconnected if there is a long connection; if it is affected by the number of retransmissions, the reconnection timeout is optimized to shorten the time it takes for the local end to detect the abnormality of the other end; A mechanism adding module, used for adding a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism to the routing forwarding rules in the service product provided by the container set, so as to shorten the time taken by the local end to detect the abnormality of the opposite end; wherein the keep-alive detection mechanism is to send a keep-alive detection packet to the opposite end once every first preset time period, and if no confirmation information returned by the opposite end is received after N consecutive keep-alive detection packets are sent, the connection between the local end and the opposite end is closed; the abnormal instance detection and elimination mechanism is to close the abnormal backend container set within a third preset time period if abnormal information appears M times consecutively within a second preset time period, so as to stop the abnormal backend container set from providing services; M and N are integers greater than 1; The effect verification module is used to verify whether the time taken by the local end to detect an abnormality at the other end is shortened after the optimization configuration of the reconnection timeout; and to verify whether the time taken by the local end to detect an abnormality at the other end is shortened after adding a keep-alive detection mechanism and an abnormal instance detection and elimination mechanism to the routing forwarding rules.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

10. A computing device, characterized in that The method comprises a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 7 is implemented.

Images