A network packet loss processing method, system and related components

Abstract

The application discloses a network packet loss processing method and system and related components, and relates to the field of network state processing. The network packet loss processing method comprises the following steps: detecting the network packet loss rate of a node; determining the network packet loss cause when the network packet loss rate exceeds a preset threshold; the network packet loss cause comprises hardware exception or system operation exception; determining that the node does not satisfy the self-recovery condition and isolating the service of the node when the network packet loss cause is the hardware exception; determining that the node satisfies the self-recovery condition and controlling the node to perform automatic repair operation when the network packet loss cause is the system operation exception. The application can make the node perform automatic repair operation when the network packet loss cause satisfies the self-recovery condition, automatically repair the network, improve the stability of the storage system, isolate the service of the node when the network packet loss cause does not satisfy the self-recovery condition, and improve the processing efficiency of the network packet loss problem.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of network state processing, in particular to a network packet loss processing method and system and related components. BACKGROUND

[0002] At present, in the field of network security, the state that can make the network run normally and can quickly recover after being impacted by the outside world is called a healthy state, and the state that is paralyzed and cannot run normally after being impacted by the outside world is called an unhealthy state. At present, the networks of many large and medium-sized enterprises are in a sub-healthy state. The network in this state can run normally at ordinary times, but the ability to resist risks is extremely low, and it is easy to be paralyzed in the case of sudden network risks and difficult to recover for a long time.

[0003] Network packet loss is one of the common network sub-healthy states. Local node packet loss can cause a bottleneck in the data path and greatly affect the performance of the distributed storage system. There are many reasons for network packet loss, which may come from external hardware such as network cables and switches, or from internal hardware such as node network cards, or from insufficient system resources (such as insufficient CPU (Central Processing Unit) resources) to cause the inability to receive and process network packets. Because there are many reasons for network packet loss, the strategy currently used to handle network packet loss is to isolate the storage service of the entire node that has network packet loss problems. This strategy is only to ensure that the storage service is not affected and cannot repair the network. In addition, when encountering network packet loss, the alarm is usually recorded first, and then the staff intervenes to analyze. For intermittent packet loss, it is extremely likely that the cause cannot be accurately located, thereby prolonging the time for analyzing and processing network packet loss problems.

[0004] Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art need to solve at present. SUMMARY

[0005] The purpose of the present application is to provide a network packet loss processing method, system and related components, which can make the node execute automatic repair operation and automatically repair the network when the network packet loss reason meets the self-recovery condition, improve the stability of the storage system, and isolate the service of the node when the network packet loss reason does not meet the self-recovery condition, thereby improving the processing efficiency of network packet loss problems.

[0006] To solve the above technical problems, the present application provides a network packet loss processing method, comprising:

[0007] detecting the network packet loss rate of a node;

[0008] When the network packet loss rate exceeds a preset threshold, a network packet loss reason is determined; the network packet loss reason includes a hardware exception or a system running exception;

[0009] When the network packet loss reason is the hardware exception, it is determined that the node does not satisfy a self-recovery condition, and a service of the node is isolated;

[0010] When the network packet loss reason is the system running exception, it is determined that the node satisfies the self-recovery condition, and the node is controlled to perform an automatic repair operation.

[0011] Optionally, the process of detecting the network packet loss rate of the node includes:

[0012] A first detection packet is constructed and sent to a server;

[0013] A second detection packet returned by the server based on the first detection packet is received;

[0014] A sending quantity of the first detection packet and a receiving quantity of the second detection packet in a first preset time period are determined;

[0015] A network packet loss rate of the node is determined based on the sending quantity and the receiving quantity.

[0016] Optionally, the process of constructing the first detection packet includes:

[0017] The first detection packet is constructed based on an Internet Control Message Protocol or a User Datagram Protocol.

[0018] Optionally, the process of determining the network packet loss rate of the node based on the sending quantity and the receiving quantity includes:

[0019] An absolute value of a difference between the receiving quantity and the sending quantity is determined;

[0020] A ratio of the absolute value to the sending quantity is determined as the network packet loss rate of the node.

[0021] Optionally, the process of determining the network packet loss reason includes:

[0022] A packet loss quantity corresponding to a network port hardware is acquired;

[0023] The network packet loss reason is determined to be the hardware exception or the system running exception based on the packet loss quantity.

[0024] Optionally, the process of determining the network packet loss reason to be the hardware exception or the system running exception based on the packet loss quantity includes:

[0025] It is determined whether the packet loss quantity is increasing;

[0026] If not, it is determined that the network packet loss reason is hardware exception.

[0027] Optionally, the process of obtaining the packet loss quantity corresponding to the network port hardware comprises:

[0028] obtaining the total packet loss quantity of the network port hardware, the first packet loss quantity of the link channel of the network port hardware, and the second packet loss quantity corresponding to the CPU in the network port hardware.

[0029] Optionally, the process of determining whether the packet loss quantity increases comprises:

[0030] determining whether the total packet loss quantity increases;

[0031] if the total packet loss quantity increases, determining whether the first packet loss quantity increases;

[0032] if the first packet loss quantity does not increase, determining whether the second packet loss quantity increases;

[0033] if the second packet loss quantity does not increase, it is determined that the packet loss quantity corresponding to the network port hardware does not increase;

[0034] if the second packet loss quantity increases, it is determined that the packet loss quantity corresponding to the network port hardware increases.

[0035] Optionally, the process of determining that the network packet loss reason is hardware exception or system running exception based on the packet loss quantity comprises:

[0036] when the packet loss quantity corresponding to the network port hardware increases, loading a kernel packet loss detection driver to detect whether there is packet loss in the protocol layer;

[0037] if yes, it is determined that the network packet loss reason is system running exception.

[0038] Optionally, after the kernel packet loss detection driver is loaded to detect whether there is packet loss in the protocol layer, the network packet loss processing method further comprises:

[0039] when there is no packet loss in the protocol layer, it is determined that the network packet loss reason is hardware exception.

[0040] Optionally, after the kernel packet loss detection driver is loaded to detect whether there is packet loss in the protocol layer, the network packet loss processing method further comprises:

[0041] unloading the kernel packet loss detection driver.

[0042] Optionally, after it is determined that the node does not meet the self-recovery condition, the network packet loss processing method further comprises:

[0043] generating an alarm prompt information.

[0044] Optionally, the alarm prompt information comprises a current network packet loss rate, a packet loss cause analysis result, and a troubleshooting direction.

[0045] Optionally, the process of controlling the node to perform the automatic repair operation comprises:

[0046] obtaining an automatic repair service configuration table, the automatic repair service configuration table comprising target service names and a parameter usage threshold value and a restart state value corresponding to each target service name;

[0047] obtaining a process list corresponding to the target parameter;

[0048] if a service name corresponding to a process in the process list is not any of the target service names, determining the process as an unnecessary process and restarting the unnecessary process.

[0049] Optionally, after obtaining the process list corresponding to the target parameter, the network packet loss processing method further comprises:

[0050] if a service name corresponding to a process in the process list is any of the target service names, determining whether a current parameter usage value of the process is greater than a parameter usage threshold value corresponding to the target service name;

[0051] if yes, determining whether a restart state value corresponding to the target service name is a first preset state value;

[0052] if the restart state value corresponding to the target service name is the first preset state value, restarting the process.

[0053] Optionally, after determining whether the restart state value corresponding to the target service name is the first preset state value, the network packet loss processing method comprises:

[0054] if the restart state value corresponding to the target service name is a second preset state value, limiting a current parameter usage value of the process to a target parameter usage threshold value corresponding to the target service name corresponding to the process; the second preset state value is different from the first preset state value.

[0055] To solve the above technical problems, the application further provides a network packet loss processing system, comprising:

[0056] a detection module configured to detect a network packet loss rate of a node;

[0057] a first determination module configured to determine a network packet loss cause when the network packet loss rate exceeds a preset threshold value; the network packet loss cause comprises a hardware abnormality or a system running abnormality;

[0058] The first processing module is configured to determine that the node satisfies the self-recovery condition and control the node to perform an automatic repair operation when the network packet loss reason is the system abnormal operation.

[0059] The second processing module is configured to isolate the service of the node when the network packet loss reason is the hardware exception and determine that the node does not satisfy the self-recovery condition.

[0060] To solve the above technical problems, the application further provides an electronic device, which comprises:

[0061] A memory configured to store a computer program.

[0062] A processor configured to execute the computer program to implement the steps of the network packet loss processing method according to any one of the preceding embodiments.

[0063] To solve the above technical problems, the application further provides a storage system comprising a plurality of nodes, each node comprising the electronic device according to the preceding embodiments.

[0064] To solve the above technical problems, the application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the network packet loss processing method according to any one of the preceding embodiments.

[0065] The application provides a network packet loss processing method, which does not directly isolate the service of a node when the node has a network packet loss problem, but first determines the network packet loss reason, makes the node perform an automatic repair operation if the network packet loss reason satisfies a self-recovery condition, and automatically repairs the network, thereby improving the stability of a storage system, and isolates the service of the node again if the network packet loss reason does not satisfy the self-recovery condition, thereby improving the processing efficiency of the network packet loss problem. The application further provides a network packet loss processing system, an electronic device, a storage system and a computer readable storage medium, which have the same beneficial effects as the network packet loss processing method. BRIEF DESCRIPTION OF DRAWINGS

[0066] To make the application embodiments clearer, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described in the following are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0067] Figure 1 A step flowchart of the network packet loss processing method provided by the application;

[0068] Figure 2 A schematic diagram of detecting the network packet loss rate of a node provided by the application;

[0069] Figure 3 A step flow chart of a packet loss rate enhanced detection method provided by the present application;

[0070] Figure 4 A step flow chart of an automatic repair operation provided by the present application;

[0071] Figure 5 A structural schematic diagram of a network packet loss processing system provided by the present application. DETAILED DESCRIPTION

[0072] The core of the present application is to provide a network packet loss processing method, system and related components, which can make the node perform an automatic repair operation when the network packet loss reason meets a self-recovery condition, automatically repair the network, improve the stability of the storage system, and isolate the service of the node when the network packet loss reason does not meet the self-recovery condition, thereby improving the processing efficiency of the network packet loss problem.

[0073] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0074] In a first aspect, refer to Figure 1 , Figure 1 A network packet loss processing method provided by the present application, the network packet loss processing method comprises:

[0075] S101: detecting a network packet loss rate of a node;

[0076] S102: determining a network packet loss reason when the network packet loss rate exceeds a preset threshold value, wherein the network packet loss reason comprises a hardware abnormality or a system running abnormality;

[0077] It can be understood that each node has a preset threshold value of the packet loss rate, and when the network packet loss rate exceeds the preset threshold value, it indicates that the node has a network packet loss problem. If the preset threshold value is not exceeded, it indicates that the network state of the node is within a normal controllable range, and at this time, the network packet loss will not affect the normal operation of the node service.

[0078] Among them, the network packet loss rate of the node can be detected according to a preset detection period, or the network packet loss rate of the node can be detected after receiving a detection instruction. The present embodiment does not specifically limit the triggering condition of detecting the network packet loss rate of the node.

[0079] When the network packet loss rate of the node exceeds the preset threshold, it is determined that the performance of the storage system where the node is located may be affected at this time, and the network packet loss reason is determined. It can be understood that there are many reasons for network packet loss, which may come from external hardware such as network cable and switch, or from internal hardware such as node network card, or from system resources (such as insufficient CPU resources) that cannot receive and process network packets.

[0080] S103: When the network packet loss reason is hardware exception, it is determined that the node does not meet the self-recovery condition, and the service of the node is isolated;

[0081] S104: When the network packet loss reason is system running exception, it is determined that the node meets the self-recovery condition, and the node is controlled to perform automatic repair operation.

[0082] After determining the network packet loss reason, it is determined whether the network packet loss reason is in the repairable range, that is, whether it meets the self-recovery condition. It can be understood that the network packet loss caused by system running exception is in the automatic repair range, and the network packet loss caused by hardware exception is not in the automatic repair range. If it is determined that the network packet loss reason meets the self-recovery condition, the node is controlled to perform automatic repair operation. If it is determined that the network packet loss reason does not meet the self-recovery condition, an alarm prompt information can be generated and reported, and the service of the node is isolated. The alarm prompt information includes but is not limited to the current network packet loss rate, the packet loss reason analysis result and the troubleshooting direction, thereby improving the efficiency of subsequent analysis of network packet loss problem.

[0083] It can be seen that in the embodiment, when the node has network packet loss problem, the service of the node is not directly isolated, but the network packet loss reason is first determined. If the network packet loss reason meets the self-recovery condition, the node is controlled to perform automatic repair operation to automatically repair the network, thereby improving the stability of the storage system. If the network packet loss reason does not meet the self-recovery condition, the service of the node is isolated, thereby improving the processing efficiency of network packet loss problem.

[0084] On the basis of the above embodiment:

[0085] As an optional embodiment, the process of detecting the network packet loss rate of the node includes:

[0086] constructing and sending a first detection packet to a server;

[0087] receiving a second detection packet returned by the server based on the first detection packet;

[0088] determining the number of sent first detection packets and the number of received second detection packets in a first preset time period;

[0089] determining the network packet loss rate of the node based on the number of sent and received.

[0090] As an optional embodiment, the process of constructing the first detection packet comprises:

[0091] The first detection packet is constructed based on an ICMP (Internet Control Message Protocol) protocol or a UDP (User Datagram Protocol) protocol.

[0092] As an optional embodiment, the process of determining the network packet loss rate of the node based on the sending quantity and the receiving quantity comprises:

[0093] Determining an absolute value of a difference between the receiving quantity and the sending quantity;

[0094] Determining a ratio of the absolute value to the sending quantity as the network packet loss rate of the node.

[0095] Referring to Figure 2 As shown in the figure, the node corresponds to a client constructing a first detection packet and sending the first detection packet to a server. The server receives the first detection packet and returns a second detection packet to the client immediately. The client receives the second detection packet, determines a sending quantity n1 of the first detection packet sent by the client and a receiving quantity n2 of the second detection packet received within a first preset time period, and calculates the network packet loss rate of the node through a first relationship formula, which is |n1-n2| / n1. The calculated network packet loss rate is compared with a preset threshold. If the network packet loss rate exceeds the preset threshold, the enhanced packet loss rate detection is performed, otherwise the network packet loss rate detection process is returned.

[0096] The enhanced packet loss rate detection is described. The enhanced packet loss rate detection mainly determines whether the network packet loss is caused by a switch, a network card hardware or a system running abnormally through newly added detection items. The network packet loss caused by the switch and the network card hardware reports a network packet loss alarm and isolates the node service. The network packet loss caused by the system running abnormally attempts to automatically repair.

[0097] As an optional embodiment, the process of determining the network packet loss cause comprises:

[0098] Obtaining a packet loss quantity corresponding to the network port hardware;

[0099] Determining the network packet loss cause as a hardware abnormality or a system running abnormality based on the packet loss quantity.

[0100] As an optional embodiment, the process of determining the network packet loss cause as a hardware abnormality or a system running abnormality based on the packet loss quantity comprises:

[0101] Determining whether the packet loss quantity is increasing;

[0102] If not, determining the network packet loss cause as a hardware abnormality.

[0103] As an optional implementation, the process of obtaining the packet loss count corresponding to the network interface hardware includes:

[0104] Obtain the total number of packet losses in the network interface hardware, the first number of packet losses in the link channel of the network interface hardware, and the second number of packet losses in the network interface hardware corresponding to the CPU.

[0105] As an optional implementation, the process of determining whether the number of lost packets has increased includes:

[0106] Determine if the total number of lost packets has increased;

[0107] If the total number of lost packets increases, determine whether the number of the first lost packets has increased;

[0108] If the number of the first packet loss does not increase, determine whether the number of the second packet loss increases;

[0109] If the number of second packet losses does not increase, it is determined that the number of packet losses corresponding to the network interface hardware has not increased.

[0110] If the number of packet losses increases, it is determined that the number of packet losses corresponding to the network interface hardware has increased.

[0111] As an optional implementation, the process of determining whether network packet loss is caused by hardware malfunction or system malfunction based on the number of lost packets includes:

[0112] When the number of packet losses corresponding to the network interface hardware increases, the kernel packet loss detection driver is loaded to detect whether there is packet loss at the protocol layer;

[0113] If so, the cause of network packet loss is determined to be abnormal system operation.

[0114] As an optional embodiment, after loading the kernel packet loss detection driver to detect whether packet loss exists at the protocol layer, the network packet loss handling method further includes:

[0115] If there is no packet loss at the protocol layer, the cause of network packet loss is determined to be a hardware malfunction.

[0116] As an optional embodiment, after loading the kernel packet loss detection driver to detect whether packet loss exists at the protocol layer, the network packet loss handling method further includes:

[0117] Uninstall the kernel packet loss detection driver.

[0118] Reference Figure 3 As shown, Figure 3 The present application provides a flowchart of a packet loss rate enhancement detection method, which includes:

[0119] S301: Check if the total number of lost packets in the network port hardware statistics has increased. If it has increased, proceed to S302; if it has not increased, proceed to S304.

[0120] S302: Check whether the first packet loss quantity corresponding to the network interface hardware link channel has growth, if not, go to S303, if yes, go to S304;

[0121] S303: Check whether the second packet loss quantity related to the network interface hardware CPU has growth, if yes, go to S304, if not, go to S305;

[0122] S304: Load the kernel packet loss detection kernel trace driver, detect whether there is packet loss in the protocol layer in the driver, after the check, unload the packet loss detection kernel trace driver, if there is no packet loss, go to S305, if there is packet loss, record the packet loss function and statistical information, trigger S306;

[0123] Specifically, in the kernel, a packet corresponds to a structure called skb (sk_buff structure), when packet loss occurs, the kernel will call kfree_skb to discard the packet, the kfree_skb function has buried a trace point, and the function address of calling kfree_skb is recorded through builtin_return_address(0) and passed to the location parameter, so that the kernel.trace can be used to track kfree_skb to obtain the packet loss function and packet loss statistics.

[0124] S305: Alarm the existence of packet loss in the switch or hardware, please check the switch or hardware, and isolate the node storage service.

[0125] S306: Perform automatic repair operation.

[0126] The process of an automatic repair operation provided by the present application will be described below.

[0127] As an optional embodiment, the process of the control node performing the automatic repair operation comprises:

[0128] Obtain an automatic repair service configuration table, the automatic repair service configuration table comprising target service names and parameter use threshold values and restart state values corresponding to each target service name;

[0129] Obtain a process list corresponding to the target parameter;

[0130] If the service name corresponding to the process in the process list is not any target service name, determine the process as an unnecessary process, and restart the unnecessary process.

[0131] As an optional embodiment, after obtaining the process list corresponding to the target parameter, the network packet loss processing method further comprises:

[0132] If the service name corresponding to the process in the process list is any target service name, it is determined whether the current parameter usage value of the process is greater than the parameter usage threshold value corresponding to the target service name;

[0133] If yes, it is determined whether the restart state value corresponding to the target service name is the first preset state value;

[0134] If yes, the process is restarted.

[0135] As an optional embodiment, after it is determined whether the restart state value corresponding to the target service name is the first preset state value, the network packet loss processing method comprises:

[0136] If the restart state value corresponding to the target service name is the second preset state value, the current parameter usage value of the process is limited to the target parameter usage threshold value, which is the parameter usage threshold value corresponding to the target service name corresponding to the process.

[0137] Specifically, first, the automatic repair configuration is read to obtain a general CPU usage threshold value, a general memory usage threshold value (these two values are mainly used as usage threshold values of unnecessary processes) and an automatic repair service configuration table. The automatic repair service configuration table is shown in Table 1.

[0138] Table 1 Automatic repair service configuration table

[0139] Service name CPU usage threshold Memory usage threshold Whether to restart (restart status value) Service 1 100％ 10G 0 Service 2 150％ 10G 1 ......

[0140] A process list corresponding to a target parameter is obtained, where the target parameter includes but is not limited to a CPU usage value, a memory usage value, and the like. The process of the automatic repair operation is described below based on the CPU usage value and the memory usage value. Figure 4

[0141] S401: A process list in which the CPU usage value is higher than the general CPU usage threshold value is obtained, and the process list is sorted in descending order;

[0142] S402: Table 1 and the process list obtained in S401 are traversed. If the service name of the process obtained in the process list is in the automatic repair service configuration table, that is, the service name of the process is any target service name in the automatic repair service configuration table, S403 is entered. If the service name of the process is not in the automatic repair service configuration table, it indicates that the process is an unnecessary process, and S405 is entered.

[0143] S403: If the CPU usage rate of the process is higher than the CPU usage threshold value in the automatic repair service configuration table, S404 is entered, otherwise S407 is entered;

[0144] ​S404: If the whether to restart configuration in the automatic repair service configuration table is yes, i.e., the restart state value is the first preset state value 1, S405 is entered, otherwise S406 is entered.

[0145] S405: Restarting the process and entering the memory usage detection process S407.

[0146] S406: Limiting the service CPU usage to the CPU usage threshold in the configuration table using cgroup, and entering S407.

[0147] S407: The memory usage detection process and the CPU usage detection process only differ in the detection values, and other aspects are consistent.

[0148] Specifically, a process list with memory usage higher than the general memory usage threshold is obtained and sorted from large to small. The automatic repair service configuration table obtained in Table 1 and the process list corresponding to the memory usage are traversed. If the process is not in the list, the process is a non-essential process, and the process is restarted. If the process obtained in the process list is in the automatic repair service configuration table, if the memory usage of the process is higher than the memory usage threshold in the configuration table, and the whether to restart configuration in the automatic repair service configuration table is yes, the process is restarted; otherwise, the service memory usage is limited to the memory usage threshold in the configuration table using cgroup.

[0149] In summary, after identifying network packet loss, the application will start a further detection mechanism to analyze whether the cause of network packet loss is within the repairable range. If it is within the repairable range, it will be automatically repaired based on the configuration. If it is not within the repairable range, the analysis conclusion will be reported, improving the efficiency of subsequent analysis of network packet loss problems. The application can finely distinguish the cause of network packet loss and automatically repair some network packet loss problems. The stability of the system is improved, and the efficiency of subsequent analysis of network packet loss is also improved.

[0150] In a second aspect, referring to Figure 5 , Figure 5 A network packet loss processing system provided by the application comprises:

[0151] A first detection module 51 is configured to detect the network packet loss rate of a node.

[0152] A first determination module 52 is configured to determine the network packet loss cause when the network packet loss rate exceeds a preset threshold. The network packet loss cause includes hardware abnormality or system running abnormality.

[0153] A first processing module 53 is configured to control the node to perform an automatic repair operation when a first trigger instruction is received.

[0154] A second processing module 54 is configured to isolate the service of the node when a second trigger instruction is received.

[0155] It can be seen that in the embodiment, when the node has a network packet loss problem, the service of the node is not directly isolated, but the network packet loss reason is first determined. If the network packet loss reason meets the self-recovery condition, the node performs an automatic repair operation to automatically repair the network, thereby improving the stability of the storage system. If the network packet loss reason does not meet the self-recovery condition, the service of the node is isolated, thereby improving the processing efficiency of the network packet loss problem.

[0156] As an optional embodiment, the process of detecting the network packet loss rate of the node includes:

[0157] constructing and sending a first detection packet to the server;

[0158] receiving a second detection packet returned by the server based on the first detection packet;

[0159] determining the number of sent first detection packets and the number of received second detection packets within a first preset time period;

[0160] determining the network packet loss rate of the node based on the number of sent and the number of received.

[0161] As an optional embodiment, the process of constructing the first detection packet includes:

[0162] constructing the first detection packet based on the ICMP protocol or the UDP protocol.

[0163] As an optional embodiment, the process of determining the network packet loss rate of the node based on the number of sent and the number of received includes:

[0164] determining the absolute value of the difference between the number of received and the number of sent;

[0165] determining the ratio of the absolute value to the number of sent as the network packet loss rate of the node.

[0166] As an optional embodiment, the process of determining the network packet loss reason includes:

[0167] obtaining the number of lost packets corresponding to the network interface hardware;

[0168] determining the network packet loss reason to be hardware abnormality or system running abnormality based on the number of lost packets.

[0169] As an optional embodiment, the process of determining whether the node meets the self-recovery condition based on the network packet loss reason includes:

[0170] when the network packet loss reason is hardware abnormality, determining that the node does not meet the self-recovery condition;

[0171] when the network packet loss reason is system running abnormality, determining that the node meets the self-recovery condition. As an optional embodiment, the process of determining whether the node meets the self-recovery condition based on the network packet loss reason includes:

[0170] when the network packet loss reason is hardware abnormality, determining that the node does not meet the self-recovery condition;

[0171] when the network packet loss reason is system running abnormality, determining that the node meets the self-recovery condition.

[0172] As an optional embodiment, the process of determining the network packet loss cause as a hardware exception or a system operation exception based on the packet loss quantity comprises:

[0173] determining whether the packet loss quantity increases;

[0174] if not, determining the network packet loss cause as a hardware exception.

[0175] As an optional embodiment, the process of obtaining the packet loss quantity corresponding to the network port hardware comprises:

[0176] obtaining the total packet loss quantity of the network port hardware, the first packet loss quantity of the link channel of the network port hardware, and the second packet loss quantity corresponding to the CPU in the network port hardware.

[0177] As an optional embodiment, the process of determining whether the packet loss quantity increases comprises:

[0178] determining whether the total packet loss quantity increases;

[0179] if the total packet loss quantity increases, determining whether the first packet loss quantity increases;

[0180] if the first packet loss quantity does not increase, determining whether the second packet loss quantity increases;

[0181] if the second packet loss quantity does not increase, determining that the packet loss quantity corresponding to the network port hardware does not increase;

[0182] if the second packet loss quantity increases, determining that the packet loss quantity corresponding to the network port hardware increases.

[0183] As an optional embodiment, the process of determining the network packet loss cause as a hardware exception or a system operation exception based on the packet loss quantity comprises:

[0184] when the packet loss quantity corresponding to the network port hardware increases, loading a kernel packet loss detection driver to detect whether there is packet loss in the protocol layer;

[0185] if yes, determining the network packet loss cause as a system operation exception.

[0186] As an optional embodiment, the network packet loss processing system further comprises:

[0187] a first determination module 53, specifically configured to determine the network packet loss cause as a hardware exception when there is no packet loss in the protocol layer.

[0188] As an optional embodiment, after loading the kernel packet loss detection driver to detect whether there is packet loss in the protocol layer, the network packet loss processing system further comprises:

[0189] an uninstallation module, configured to uninstall the kernel packet loss detection driver.

[0190] As an optional embodiment, the network packet loss processing system further comprises:

[0191] an alarm module configured to generate an alarm prompt if the decision node does not satisfy the self-recovery condition.

[0192] As an optional embodiment, the alarm prompt comprises a current network packet loss rate, a packet loss cause analysis result, and an investigation direction.

[0193] As an optional embodiment, the first processing module 53 is configured to:

[0194] obtain an automatic repair service configuration table, the automatic repair service configuration table comprising target service names and parameter usage threshold values and restart state values corresponding to each target service name;

[0195] obtain a process list corresponding to the target parameter;

[0196] if a service name corresponding to a process in the process list is not any target service name, the process is determined as a non-essential process, and the non-essential process is restarted.

[0197] As an optional embodiment, the first processing module 53 is further configured to:

[0198] if a service name corresponding to a process in the process list is any target service name, it is determined whether a current parameter usage value of the process is greater than a parameter usage threshold value corresponding to the target service name; if yes, it is determined whether a restart state value corresponding to the target service name is a first preset state value; if yes, the process is restarted.

[0199] As an optional embodiment, the first processing module 53 is further configured to:

[0200] if the restart state value corresponding to the target service name is a second preset state value, the current parameter usage value of the process is limited to a target parameter usage threshold value, the target parameter usage threshold value being a parameter usage threshold value corresponding to a target service name corresponding to the process.

[0201] In a third aspect, the present application further provides an electronic device comprising:

[0202] a memory configured to store a computer program;

[0203] a processor configured to execute the computer program to implement the steps of the network packet loss processing method according to any one of the above aspects.

[0204] Specifically, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer readable instructions, and the internal memory provides an environment for running the operating system and computer readable instructions in the non-volatile storage medium. The processor provides computing and control capabilities for the electronic device, and when executing the computer program stored in the memory, the following steps can be implemented: detecting a network packet loss rate of the node; when the network packet loss rate exceeds a preset threshold, determining a network packet loss reason; judging whether the node meets a self-recovery condition based on the network packet loss reason; if yes, controlling the node to perform an automatic repair operation; and if no, isolating the service of the node.

[0205] As can be seen, in the embodiment, when the node has a network packet loss problem, the service of the node is not directly isolated, but the network packet loss reason is first determined. If the network packet loss reason meets the self-recovery condition, the node is caused to perform an automatic repair operation to automatically repair the network, thereby improving the stability of the storage system. If the network packet loss reason does not meet the self-recovery condition, the service of the node is isolated, thereby improving the processing efficiency of the network packet loss problem.

[0206] As an optional embodiment, when the processor executes the computer subprogram stored in the memory, the following steps can be implemented: constructing and sending a first detection packet to a server; receiving a second detection packet returned by the server based on the first detection packet; determining a sending quantity of the first detection packet and a receiving quantity of the second detection packet within a first preset time period; and determining a network packet loss rate of the node based on the sending quantity and the receiving quantity.

[0207] As an optional embodiment, when the processor executes the computer subprogram stored in the memory, the following steps can be implemented: constructing the first detection packet based on an ICMP protocol or a UDP protocol.

[0208] As an optional embodiment, when the processor executes the computer subprogram stored in the memory, the following steps can be implemented: determining an absolute value of a difference between the receiving quantity and the sending quantity; and determining a ratio of the absolute value to the sending quantity as the network packet loss rate of the node.

[0209] As an optional embodiment, when the processor executes the computer subprogram stored in the memory, the following steps can be implemented: obtaining a packet loss quantity corresponding to a network port hardware; and determining that the network packet loss reason is a hardware exception or a system running exception based on the packet loss quantity.

[0210] As an optional embodiment, when the processor executes the computer subprogram stored in the memory, the following steps can be implemented: when the network packet loss reason is the hardware exception, determining that the node does not meet the self-recovery condition; and when the network packet loss reason is the system running exception, determining that the node meets the self-recovery condition.

[0211] As an optional embodiment, the processor executes the computer program stored in the memory, and the following steps can be implemented: judging whether the number of lost packets increases; if not, determining that the network packet loss reason is hardware exception.

[0212] As an optional embodiment, the processor executes the computer program stored in the memory, and the following steps can be implemented: obtaining the total number of lost packets of the network port hardware, the first number of lost packets of the link channel of the network port hardware, and the second number of lost packets of the network port hardware corresponding to the CPU.

[0213] As an optional embodiment, the processor executes the computer program stored in the memory, and the following steps can be implemented: judging whether the total number of lost packets increases; if the total number of lost packets increases, judging whether the first number of lost packets increases; if the first number of lost packets does not increase, judging whether the second number of lost packets increases; if the second number of lost packets does not increase, determining that the number of lost packets corresponding to the network port hardware does not increase; if the second number of lost packets increases, determining that the number of lost packets corresponding to the network port hardware increases.

[0214] As an optional embodiment, the processor executes the computer program stored in the memory, and the following steps can be implemented: when the number of lost packets corresponding to the network port hardware increases, loading the kernel packet loss detection driver to detect whether there is packet loss in the protocol layer; if yes, determining that the network packet loss reason is system running exception.

[0215] As an optional embodiment, the processor executes the computer program stored in the memory, and the following steps can be implemented: when there is no packet loss in the protocol layer, determining that the network packet loss reason is hardware exception.

[0216] As an optional embodiment, the processor executes the computer program stored in the memory, and the following steps can be implemented: uninstalling the kernel packet loss detection driver.

[0217] As an optional embodiment, the processor executes the computer program stored in the memory, and the following steps can be implemented: if it is determined that the node does not satisfy the self-recovery condition, generating an alarm prompt information.

[0218] As an optional embodiment, the processor executes the computer program stored in the memory, and the following steps can be implemented: if it is determined that the node does not satisfy the self-recovery condition, generating an alarm prompt information, and the alarm prompt information includes the current network packet loss rate, the packet loss reason analysis result, and the troubleshooting direction.

[0219] As an optional embodiment, the processor executes the computer program stored in the memory, and the following steps can be implemented: obtaining an automatic repair service configuration table, the automatic repair service configuration table including target service names, and a parameter usage threshold and a restart state value corresponding to each target service name; obtaining a process list corresponding to the target parameter; if a service name corresponding to a process in the process list is not any target service name, determining the process as an unnecessary process, and restarting the unnecessary process.

[0220] As an optional embodiment, the processor executes the computer program stored in the memory, and the following steps can be implemented: if a service name corresponding to a process in the process list is any target service name, determining whether a current parameter usage value of the process is greater than a parameter usage threshold corresponding to the target service name; if yes, determining whether a restart state value corresponding to the target service name is a first preset state value; if yes, restarting the process.

[0221] As an optional embodiment, the processor executes the computer program stored in the memory, and the following steps can be implemented: if the restart state value corresponding to the target service name is a second preset state value, limiting the current parameter usage value of the process to a target parameter usage threshold, the target parameter usage threshold being the parameter usage threshold corresponding to the target service name corresponding to the process.

[0222] In the above embodiment, as a preferred embodiment, the electronic device further comprises:

[0223] The input interface is connected to the processor, and is configured to obtain a computer program, parameters and instructions imported from outside, and save the computer program, parameters and instructions into the memory under control of the processor. The input interface can be connected to an input device, and receive parameters or instructions manually input by a user. The input device can be a touch layer overlaid on a display screen, or a key, trackball or touchpad arranged on a terminal shell.

[0224] The display unit is connected to the processor, and is configured to display data sent by the processor. The display unit can be a liquid crystal display screen or an electronic ink display screen.

[0225] The network port is connected to the processor, and is configured to be communicatively connected to external terminal devices. The communication technology used in the communication connection can be wired communication technology or wireless communication technology, such as Mobile High-definition Link (MHL), Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), Wireless Fidelity (WiFi), Bluetooth communication technology, Bluetooth Low Energy (BLE) communication technology, and IEEE 802.11s-based communication technology.

[0226] In a fourth aspect, the present application further provides a storage system, including a plurality of nodes, each node including the electronic device as described above.

[0227] For the introduction of a storage system provided by the present application, please refer to the above embodiments, which will not be repeated here.

[0228] The storage system provided by the present application has the same beneficial effects as the above electronic device, which will not be repeated here.

[0229] In the fifth aspect, the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the network packet loss processing method according to any one of the above embodiments are implemented.

[0230] The storage medium can include a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage medium that can store program codes. The storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented: detecting the network packet loss rate of a node; when the network packet loss rate exceeds a preset threshold, determining the network packet loss reason; judging whether the node meets the self-recovery condition based on the network packet loss reason; if yes, controlling the node to perform an automatic repair operation; and if no, isolating the service of the node.

[0231] As can be seen, in the embodiment, when the node has a network packet loss problem, the service of the node is not directly isolated, but the network packet loss reason is first determined. If the network packet loss reason meets the self-recovery condition, the node performs an automatic repair operation to automatically repair the network, thereby improving the stability of the storage system. If the network packet loss reason does not meet the self-recovery condition, the service of the node is isolated, thereby improving the processing efficiency of the network packet loss problem.

[0232] As an optional embodiment, when the computer program stored in the computer readable storage medium is executed by the processor, the following steps can be specifically implemented: constructing and sending a first detection packet to a server; receiving a second detection packet returned by the server based on the first detection packet; determining the number of sent first detection packets and the number of received second detection packets within a first preset time period; and determining the network packet loss rate of the node based on the number of sent packets and the number of received packets.

[0233] As an optional embodiment, when the computer program stored in the computer readable storage medium is executed by the processor, the following steps can be specifically implemented: constructing the first detection packet based on the ICMP protocol or the UDP protocol.

[0234] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: determining an absolute value of a difference between the receiving quantity and the sending quantity; determining a ratio of the absolute value to the sending quantity as the network packet loss rate of the node.

[0235] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: obtaining the packet loss quantity corresponding to the network port hardware; determining the network packet loss reason as a hardware exception or a system running exception based on the packet loss quantity.

[0236] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: when the network packet loss reason is the hardware exception, determining that the node does not satisfy the self-recovery condition; when the network packet loss reason is the system running exception, determining that the node satisfies the self-recovery condition.

[0237] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: determining whether the packet loss quantity increases; if not, determining that the network packet loss reason is the hardware exception.

[0238] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: obtaining a total packet loss quantity of the network port hardware, a first packet loss quantity of a link channel of the network port hardware, and a second packet loss quantity corresponding to the CPU in the network port hardware.

[0239] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: determining whether the total packet loss quantity increases; if the total packet loss quantity increases, determining whether the first packet loss quantity increases; if the first packet loss quantity does not increase, determining whether the second packet loss quantity increases; if the second packet loss quantity does not increase, determining that the packet loss quantity corresponding to the network port hardware does not increase; if the second packet loss quantity increases, determining that the packet loss quantity corresponding to the network port hardware increases.

[0240] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: when the packet loss quantity corresponding to the network port hardware increases, loading a kernel packet loss detection driver to detect whether there is packet loss in a protocol layer; if yes, determining that the network packet loss reason is the system running exception.

[0241] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: when there is no packet loss in the protocol layer, determining that the network packet loss reason is the hardware exception.

[0242] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: uninstalling the kernel packet loss detection driver.

[0243] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: if the node does not satisfy the self-recovery condition, generating an alarm prompt information.

[0244] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: if the node does not satisfy the self-recovery condition, generating an alarm prompt information, and the alarm prompt information includes the current network packet loss rate, the packet loss reason analysis result and the troubleshooting direction.

[0245] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: obtaining an automatic repair service configuration table, the automatic repair service configuration table including target service names and parameter usage threshold values and restart state values corresponding to each target service name; obtaining a process list corresponding to the target parameter; if the service name corresponding to the process in the process list is not any target service name, determining the process as a non-essential process, and restarting the non-essential process.

[0246] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: if the service name corresponding to the process in the process list is any target service name, determining whether the current parameter usage value of the process is greater than the parameter usage threshold value corresponding to the target service name; if yes, determining whether the restart state value corresponding to the target service name is a first preset state value; if yes, restarting the process.

[0247] As an optional embodiment, the computer program stored in the computer readable storage medium is executed by the processor, and the following steps can be specifically implemented: if the restart state value corresponding to the target service name is a second preset state value, limiting the current parameter usage value of the process to the target parameter usage threshold value, and the target parameter usage threshold value is the parameter usage threshold value corresponding to the target service name corresponding to the process.

[0248] It is also noted that, in this disclosure, relational terms such as first and second, and the like, can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0249] The above description of disclosed embodiments provides enabling concepts for practicing or using the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for handling network packet loss, characterized in that, include: Detect the network packet loss rate of the node; When the network packet loss rate exceeds a preset threshold, the cause of the network packet loss is determined. The reasons for network packet loss include hardware malfunctions or system malfunctions. When the network packet loss is caused by hardware failure, it is determined that the node does not meet the self-recovery conditions, and the service of the node is isolated. When the network packet loss is caused by abnormal system operation, it is determined that the node meets the self-recovery condition, and the node is controlled to perform automatic repair operation; The process of determining the cause of network packet loss includes: Get the number of packets lost corresponding to the network port hardware; Based on the number of lost packets, the cause of network packet loss is determined to be either hardware malfunction or system malfunction. The process of controlling the node to perform automatic repair operations includes: Read the automatic repair configuration to obtain the automatic repair service configuration table, the general CPU usage threshold, and the general memory usage threshold. The automatic repair service configuration table includes the target service name and the parameter usage threshold and restart status value corresponding to each target service name. The parameter usage threshold includes the CPU usage threshold and the memory usage threshold. Obtain the process list corresponding to the target parameters; the target parameters include CPU usage values ​​and memory usage values. If the service name corresponding to a process in the process list is not any of the target service names, the process is determined to be an unnecessary process and the unnecessary process is restarted. The process of obtaining the list of processes corresponding to the target parameters includes: Obtain a list of processes whose CPU usage value is higher than the general CPU usage threshold; Obtain a list of processes whose memory usage is higher than the general memory usage threshold.

2. The network packet loss processing method according to claim 1, characterized in that, The process of detecting the network packet loss rate of the node includes: Construct and send the first detection message to the server; Receive the second detection message returned by the server based on the first detection message; Determine the number of the first detection messages sent and the number of the second detection messages received within a first preset time period; The network packet loss rate of a node is determined based on the number of packets sent and the number of packets received.

3. The network packet loss processing method according to claim 2, characterized in that, The process of constructing the first detection message includes: The first detection message is constructed based on the Internet Control Message Protocol or the User Datagram Protocol.

4. The network packet loss processing method according to claim 2, characterized in that, The process of determining the network packet loss rate of a node based on the number of packets sent and the number of packets received includes: Determine the absolute value of the difference between the number of received items and the number of sent items; The ratio of the absolute value to the number of packets sent is determined as the network packet loss rate of the node.

5. The network packet loss processing method according to claim 1, characterized in that, The process of determining whether network packet loss is due to hardware malfunction or system malfunction based on the number of lost packets includes: Determine whether the number of lost packets has increased; If not, the cause of the network packet loss is determined to be a hardware malfunction.

6. The network packet loss processing method according to claim 5, characterized in that, The process of obtaining the packet loss count corresponding to the network port hardware includes: Obtain the total number of packet losses in the network interface hardware, the first number of packet losses in the link channel of the network interface hardware, and the second number of packet losses in the network interface hardware corresponding to the CPU.

7. The network packet loss processing method according to claim 6, characterized in that, The process of determining whether the number of lost packets has increased includes: Determine whether the total number of lost packets has increased; If the total number of lost packets increases, determine whether the number of lost packets has increased; If the first number of lost packets does not increase, determine whether the second number of lost packets increases; If the second packet loss number does not increase, it is determined that the packet loss number corresponding to the network port hardware has not increased; If the second packet loss count increases, it is determined that the packet loss count corresponding to the network port hardware has increased.

8. The network packet loss processing method according to claim 7, characterized in that, The process of determining whether network packet loss is due to hardware malfunction or system malfunction based on the number of lost packets includes: When the number of packet losses corresponding to the network port hardware increases, the kernel packet loss detection driver is loaded to detect whether there is packet loss at the protocol layer; If so, the cause of the network packet loss is determined to be a system malfunction.

9. The network packet loss processing method according to claim 8, characterized in that, After loading the kernel packet loss detection driver to detect whether packet loss exists at the protocol layer, the network packet loss handling method further includes: If there is no packet loss at the protocol layer, the cause of the network packet loss is determined to be a hardware malfunction.

10. The network packet loss processing method according to claim 8, characterized in that, After loading the kernel packet loss detection driver to detect whether packet loss exists at the protocol layer, the network packet loss processing method further includes: Uninstall the kernel packet loss detection driver.

11. The network packet loss processing method according to claim 1, characterized in that, After determining that the node does not meet the self-recovery condition, the network packet loss handling method further includes: Generate alarm notification messages.

12. The network packet loss processing method according to claim 11, characterized in that, The alarm information includes the current network packet loss rate, the results of packet loss cause analysis, and the direction of investigation.

13. The network packet loss processing method according to claim 1, characterized in that, After obtaining the process list corresponding to the target parameters, this network packet loss handling method also includes: If the service name corresponding to a process in the process list is any of the target service names, determine whether the current parameter usage value of the process is greater than the parameter usage threshold corresponding to the target service name; If so, determine whether the restart status value corresponding to the target service name is the first preset status value; If the restart status value corresponding to the target service name is the first preset status value, the process is restarted.

14. The network packet loss processing method according to claim 13, characterized in that, After determining whether the restart status value corresponding to the target service name is a first preset status value, the network packet loss handling method includes: If the restart status value corresponding to the target service name is the second preset status value, the current parameter usage value of the process is restricted to the target parameter usage threshold, which is the parameter usage threshold corresponding to the target service name of the process; the second preset status value is different from the first preset status value.

15. A network packet loss processing system, characterized in that, include: The detection module is used to detect the network packet loss rate of the nodes; The first determining module is used to determine the cause of network packet loss when the network packet loss rate exceeds a preset threshold. The reasons for network packet loss include hardware malfunctions or system malfunctions. The first processing module is used to determine that the node meets the self-recovery condition when the network packet loss is caused by abnormal system operation, and to control the node to perform automatic repair operation; The second processing module is used to determine that the node does not meet the self-recovery conditions and isolate the service of the node when the network packet loss is caused by hardware failure. The process of determining the cause of network packet loss includes: Get the number of packets lost corresponding to the network port hardware; Based on the number of lost packets, the cause of network packet loss is determined to be either hardware malfunction or system malfunction. The process of controlling the node to perform automatic repair operations includes: Read the automatic repair configuration to obtain the automatic repair service configuration table, the general CPU usage threshold, and the general memory usage threshold. The automatic repair service configuration table includes the target service name and the parameter usage threshold and restart status value corresponding to each target service name. The parameter usage threshold includes the CPU usage threshold and the memory usage threshold. Obtain the process list corresponding to the target parameters; the target parameters include CPU usage values ​​and memory usage values. If the service name corresponding to a process in the process list is not any of the target service names, the process is determined to be an unnecessary process and the unnecessary process is restarted. The process of obtaining the list of processes corresponding to the target parameters includes: Obtain a list of processes whose CPU usage value is higher than the general CPU usage threshold; Obtain a list of processes whose memory usage is higher than the general memory usage threshold.

16. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the network packet loss processing method as described in any one of claims 1-14 when executing the computer program.

17. A storage system, characterized in that, It includes multiple nodes, each of which includes the electronic device as described in claim 16.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the network packet loss processing method as described in any one of claims 1-14.

Images