Abnormality detection method and device, computer equipment, storage medium and program product

Abstract

The invention relates to an anomaly detection method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: acquiring an interaction instruction set between a client and each data node; performing statistical analysis on the interaction instruction set, determining potential abnormal data nodes, and sending a self-inspection instruction to the potential abnormal data nodes; and receiving a self-inspection result returned by the potential abnormal data node, and determining a detection result of the potential abnormal data node according to the self-inspection result. According to the method, through two times of anomaly detection, the client carries out statistical analysis on the interaction instruction set, the potential abnormal data node is detected for the first time, and the potential abnormal data node is self-detected again so as to accurately judge whether the potential abnormal data node is abnormal or not; compared with an existing mode of realizing anomaly detection by directly judging whether the message connection with the data node is normal or not through the client, the method greatly improves the detection accuracy of the abnormal data node.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to an anomaly detection method, apparatus, computer equipment, storage medium and program product. Background Art

[0002] Storage server clusters usually have data nodes in a suspended state, which reduces the stability of the storage server cluster. Therefore, many methods have emerged to detect whether a data node is in a suspended state and the reasons for triggering the suspended state. In related technologies, the method for detecting data node suspended state is mainly to determine whether the message connection between the client and the data node is normal.

[0003] However, the above-mentioned data node pseudo-death detection method has the problem of low accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide an anomaly detection method, device, computer equipment, storage medium and program product that can improve the accuracy of node pseudo-death detection in response to the above technical problems.

[0005] In a first aspect, the present application provides an anomaly detection method, which is applied to a client, comprising:

[0006] Obtain the interaction instruction set between the client and each data node;

[0007] Perform statistical analysis on the interactive instruction set, identify potential abnormal data nodes, and send self-check instructions to the potential abnormal data nodes;

[0008] Receive the self-test result returned by the potential abnormal data node, and determine the detection result of the potential abnormal data node according to the self-test result.

[0009] The above method uses two anomaly detections, including the client performing statistical analysis on the interaction instruction set, initially detecting potential abnormal data nodes, and the potential abnormal data nodes performing self-inspection again to accurately determine whether the potential abnormal data nodes are abnormal. Compared with the existing method of directly implementing anomaly detection by judging whether the message connection between the client and the data node is normal, the above method greatly improves the detection accuracy of abnormal data nodes.

[0010] In one embodiment, the statistical analysis of the interactive instruction set to determine potential abnormal data nodes includes:

[0011] Count the number of interactive instructions completed in the interactive instruction set, the average delay of the interactive instructions completed by each data node, and the number of retryable error failures of each data node within a preset time period;

[0012] Identify potential abnormal data nodes based on the number, average latency, and number of retryable error failures.

[0013] The method for determining potential abnormal data nodes provided in the embodiment of the present application analyzes whether each data node is a potential abnormal data node through the completion of the interaction instructions between the client and each data node, thereby providing a data basis for subsequently determining whether the potential abnormal data node is indeed an abnormal data node.

[0014] In one embodiment, determining the potential abnormal data node based on the quantity, average latency, and number of retryable error failures includes:

[0015] If the number is the first value, the corresponding data node is determined as a potential abnormal data node;

[0016] If the number is not the first value, the potential abnormal data node is determined according to the maximum average delay in the average delay and the number of retryable error failures.

[0017] The method for determining potential abnormal data nodes provided in the embodiments of the present application is implemented by counting the number of interaction instructions completed on each data node and the average delay of the completed interaction instructions, thereby providing a data basis for subsequent determination of whether the potential abnormal data node is indeed an abnormal data node.

[0018] In one embodiment, determining the potential abnormal data node according to the maximum average delay and the number of retryable error failures in the average delay includes:

[0019] If the maximum average delay is greater than the preset delay threshold, the data node corresponding to the maximum average delay is determined as a potential abnormal data node;

[0020] If the maximum average delay is not greater than the preset delay threshold, the potential abnormal data node is determined according to the maximum number of retryable error failures among the retryable error failures.

[0021] The method for determining potential abnormal data nodes provided in the embodiments of the present application is achieved by counting the average delay and the number of retryable error failures of the interactive instructions completed on each data node, thereby providing a data basis for subsequent determination of whether the potential abnormal data nodes are indeed abnormal data nodes.

[0022] In one embodiment, determining the potential abnormal data node according to the maximum number of retryable error failures among the number of retryable error failures includes:

[0023] If the maximum number of retryable error failures is greater than a preset number threshold, the data node corresponding to the maximum number of retryable error failures is determined as a potential abnormal data node;

[0024] If the maximum number of retryable error failures is not greater than a preset number threshold, the data node corresponding to the maximum number of retryable error failures is determined as a normal data node.

[0025] The method for determining potential abnormal data nodes provided in the embodiments of the present application is implemented by counting the number of retryable error failures on each data node, thereby providing a data basis for subsequent determination of whether the potential abnormal data node is indeed an abnormal data node.

[0026] In one embodiment, the method further comprises:

[0027] Sending an exception handling request to a management node; the exception handling request includes the detection result; the exception handling request is used to instruct the management node to perform exception handling on the abnormal data node determined by the detection result;

[0028] Receive the exception handling result returned by the management node.

[0029] The method for receiving exception processing results provided in the embodiment of the present application feeds back the exception processing results of the management node to the client, so that the client can refer to the exception processing results to determine the status of the data node when sending instructions to the data node, and then determine the sending object of the instruction, thereby avoiding data congestion caused by sending instructions to data nodes with abnormalities.

[0030] In a second aspect, the present application also provides an anomaly detection method, which is applied to a target data node in a data node cluster, comprising:

[0031] Upon receiving the self-check instruction sent by the client, a self-check operation is performed to obtain a self-check result; the self-check result is used to indicate whether the target data node is an abnormal data node and / or a node abnormality type;

[0032] The self-check result is sent to the client.

[0033] The anomaly detection method provided in the embodiment of the present application performs two anomaly detections, including statistical analysis of the interaction instruction set by the client, initial detection of potential abnormal data nodes, and self-inspection of the potential abnormal data nodes again to accurately determine whether the potential abnormal data nodes are abnormal. Compared with the existing method of directly implementing anomaly detection by determining whether the message connection between the client and the data node is normal, the above method greatly improves the detection accuracy of abnormal data nodes.

[0034] In one embodiment, the method further comprises:

[0035] When receiving the processing instruction sent by the management node, a power-off or restart operation is performed according to the processing method indicated by the processing instruction.

[0036] The exception handling method provided in the embodiment of the present application, when receiving a processing instruction sent by the management node, performs exception handling on itself according to the processing method indicated by the processing instruction, so as to ensure that the data communication between the client and the data node can proceed normally, and avoid the situation where the client waits for a long time and the data node does not respond.

[0037] In one of the embodiments, when receiving the self-check instruction sent by the client, performing a self-check operation to obtain a self-check result includes:

[0038] Upon receiving the self-check instruction sent by the client, triggering the task processing self-check item to perform self-check;

[0039] Determine whether there is a timeout task during the self-check process of the task processing self-check item. If so, obtain a self-check result including a task processing timeout exception; if not, trigger the abnormal self-check item to perform self-check and obtain the self-check result; the abnormal self-check item includes at least one of a node abnormal self-check item, a network abnormal self-check item, an interaction abnormal self-check item, a freeze abnormal self-check item, a disk abnormal self-check item, and a disk interaction abnormal self-check item.

[0040] The self-check method provided in the embodiment of the present application determines whether a data node has a task processing timeout exception by triggering a task processing self-check item in the data node, thereby achieving the effect of accurately judging the type of data node exception.

[0041] In a third aspect, the present application also provides an anomaly detection method, which is applied to a management data node in a data node cluster, comprising:

[0042] receiving an exception handling request sent by a client; the exception handling request includes the detection result; the exception handling request is used to instruct the management node to perform exception handling on the abnormal data node determined by the detection result;

[0043] Perform exception processing on the abnormal data nodes determined by the detection results.

[0044] The above method improves the detection accuracy of abnormal data nodes through two abnormality detections, including the client performing statistical analysis on the interactive instruction set, the initial detection of potential abnormal data nodes, and the potential abnormal data nodes performing self-inspection again to accurately determine whether the potential abnormal data nodes are abnormal; in addition, a method for processing abnormal data nodes is also provided, which can efficiently process abnormal data nodes and avoid the client sending instructions to abnormal data nodes, causing data congestion.

[0045] In one embodiment, the above-mentioned abnormal processing of the abnormal data node determined by the detection result includes:

[0046] Counting the number of times the client reports an exception processing request within a preset time period; the number indicates the number of times each of the abnormal data node requests is processed;

[0047] Power off the abnormal data nodes corresponding to the number of times greater than the preset number threshold;

[0048] According to the redundancy of the abnormal data nodes, abnormal processing is performed on the abnormal data nodes corresponding to the number of times not greater than the preset number threshold.

[0049] The exception handling method provided in the embodiment of the present application performs exception handling on different types of abnormal data nodes, thereby preventing the client from sending instructions to the abnormal data nodes, causing data congestion.

[0050] In one embodiment, the above-mentioned performing abnormal processing on the abnormal data nodes corresponding to the number of times not greater than the preset number threshold according to the redundancy of the abnormal data nodes includes:

[0051] Determining whether the redundancy is greater than a preset redundancy threshold;

[0052] If the redundancy is greater than a preset redundancy threshold, then after a preset time period, returning to the step of determining whether the redundancy is greater than the preset redundancy threshold;

[0053] If the redundancy is not greater than a preset redundancy threshold, the corresponding abnormal data node is restarted, and the identification of the restarted abnormal data node is updated to normal.

[0054] The method for processing abnormal data nodes provided above can efficiently process abnormal data nodes and avoid the situation where the client sends instructions to the abnormal data nodes, causing data congestion.

[0055] In a fourth aspect, the present application further provides an anomaly detection device, applied to a client, comprising:

[0056] An acquisition module, used to acquire an interaction instruction set between the client and each data node;

[0057] A determination module, configured to perform statistical analysis on the interactive instruction set, determine potential abnormal data nodes, and send self-check instructions to the potential abnormal data nodes;

[0058] The receiving module is used to receive the self-test result returned by the potential abnormal data node, and determine the detection result of the potential abnormal data node according to the self-test result.

[0059] In a fifth aspect, the present application further provides an anomaly detection device, which is applied to a target data node in a data node cluster, comprising:

[0060] A self-check module, configured to perform a self-check operation upon receiving a self-check instruction sent by a client, and obtain a self-check result; the self-check result is used to indicate whether the target data node is an abnormal data node and / or a node abnormality type;

[0061] A sending module is used to send the self-test result to the client.

[0062] In a sixth aspect, the present application further provides an anomaly detection device, which is applied to a management data node in a data node cluster, comprising:

[0063] A receiving module, configured to receive an exception handling request sent by a client; the exception handling request includes the detection result; the exception handling request is used to instruct the management node to perform exception handling on the abnormal data node determined by the detection result;

[0064] A processing module is used to perform exception processing on the abnormal data nodes determined by the detection results.

[0065] In a seventh aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0066] Obtain the interaction instruction set between the client and each data node;

[0067] Performing statistical analysis on the interactive instruction set, determining potential abnormal data nodes, and sending self-check instructions to the potential abnormal data nodes;

[0068] A self-test result returned by the potential abnormal data node is received, and a detection result of the potential abnormal data node is determined according to the self-test result.

[0069] In an eighth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0070] Upon receiving the self-check instruction sent by the client, a self-check operation is performed to obtain a self-check result; the self-check result is used to indicate whether the target data node is an abnormal data node and / or a node abnormality type;

[0071] The self-check result is sent to the client.

[0072] In a ninth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0073] receiving an exception handling request sent by a client; the exception handling request includes the detection result; the exception handling request is used to instruct the management node to perform exception handling on the abnormal data node determined by the detection result;

[0074] Perform exception processing on the abnormal data nodes determined by the detection results.

[0075] In a tenth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0076] Obtain the interaction instruction set between the client and each data node;

[0077] Performing statistical analysis on the interactive instruction set, determining potential abnormal data nodes, and sending self-check instructions to the potential abnormal data nodes;

[0078] A self-test result returned by the potential abnormal data node is received, and a detection result of the potential abnormal data node is determined according to the self-test result.

[0079] In an eleventh aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0080] Upon receiving the self-check instruction sent by the client, a self-check operation is performed to obtain a self-check result; the self-check result is used to indicate whether the target data node is an abnormal data node and / or a node abnormality type;

[0081] The self-check result is sent to the client.

[0082] In a twelfth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0083] receiving an exception handling request sent by a client; the exception handling request includes the detection result; the exception handling request is used to instruct the management node to perform exception handling on the abnormal data node determined by the detection result;

[0084] Perform exception processing on the abnormal data nodes determined by the detection results.

[0085] In a thirteenth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0086] Obtain the interaction instruction set between the client and each data node;

[0087] Performing statistical analysis on the interactive instruction set, determining potential abnormal data nodes, and sending self-check instructions to the potential abnormal data nodes;

[0088] A self-test result returned by the potential abnormal data node is received, and a detection result of the potential abnormal data node is determined according to the self-test result.

[0089] In a fourteenth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0090] Upon receiving the self-check instruction sent by the client, a self-check operation is performed to obtain a self-check result; the self-check result is used to indicate whether the target data node is an abnormal data node and / or a node abnormality type;

[0091] The self-check result is sent to the client.

[0092] In a fifteenth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0093] receiving an exception handling request sent by a client; the exception handling request includes the detection result; the exception handling request is used to instruct the management node to perform exception handling on the abnormal data node determined by the detection result;

[0094] Perform exception processing on the abnormal data nodes determined by the detection results.

[0095] The above-mentioned anomaly detection method, device, computer equipment, storage medium and program product, the above-mentioned method performs two anomaly detections, including the client performing statistical analysis on the interactive instruction set, initially detecting potential abnormal data nodes, and the potential abnormal data nodes self-checking again to accurately determine whether the potential abnormal data nodes are abnormal. Compared with the existing method of directly realizing anomaly detection by judging whether the message connection between the client and the data node is normal, the above-mentioned method greatly improves the detection accuracy of abnormal data nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0097] FIG1 is a diagram of an application environment of an anomaly detection method in one embodiment;

[0098] Figure 2 1 is a flow chart of an abnormality detection method in one embodiment;

[0099] Figure 3 A schematic diagram of an interactive instruction set in an embodiment;

[0100] Figure 4 is a flow chart of an abnormality detection method in another embodiment;

[0101] FIG5 is a schematic diagram of a classification process of interactive instructions in one embodiment;

[0102] Figure 6 A statistical flow chart of an interactive instruction set in one embodiment;

[0103] Figure 7 is a flow chart of an abnormality detection method in another embodiment;

[0104] Figure 8 is a flow chart of an abnormality detection method in another embodiment;

[0105] Fig. 9 is a flow chart of an abnormality detection method in another embodiment;

[0106] Fig.10 is a flow chart of an abnormality detection method in another embodiment;

[0107] Fig.11 is a flow chart of an abnormality detection method in another embodiment;

[0108] Fig.12 is a flow chart of an abnormality detection method in another embodiment;

[0109] Fig.13 is a flow chart of an abnormality detection method in another embodiment;

[0110] Fig.14 Schematic diagram of a self-check process of a target data node in one embodiment;

[0111] Fig.15 is a flow chart of an abnormality detection method in another embodiment;

[0112] Fig.16 is a flow chart of an abnormality detection method in another embodiment;

[0113] Fig.17 is a flow chart of an abnormality detection method in another embodiment;

[0114] Fig.18 is a flow chart of an abnormality detection method in another embodiment;

[0115] Fig.19 is a flow chart of an abnormality detection method in another embodiment;

[0116] Fig. 20 is a flow chart of an abnormality detection method in another embodiment;

[0117] Fig.21 is a flow chart of an abnormality detection method in another embodiment;

[0118] Fig. 22 is a flow chart of an abnormality detection method in another embodiment;

[0119] Fig.23 is a flow chart of an abnormality detection method in another embodiment;

[0120] Fig.24 The following is a schematic diagram of a process flow of a management node processing an abnormal data node in one embodiment;

[0121] FIG25 is a flow chart of an abnormality detection method in another embodiment;

[0122] FIG26 is a block diagram of an abnormality detection device in one embodiment;

[0123] FIG27 is a block diagram of an abnormality detection device in another embodiment;

[0124] FIG28 is a block diagram of an abnormality detection device in another embodiment;

[0125] FIG. 29 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0126] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0127] Storage server clusters usually have data nodes in a suspended state, which will reduce the stability of the storage server cluster. Therefore, many methods have emerged to detect whether a data node is in a suspended state and the reasons for triggering the suspended state. In the related art, the method for detecting the suspended state of a data node is mainly to determine whether the message connection between the client and the data node is normal. However, the above-mentioned method for detecting the suspended state of a data node has the problem of low accuracy. The present application aims to solve this problem.

[0128] After the above introduction to the background technology of the anomaly detection method provided by the embodiment of the present application, the implementation environment involved in the anomaly detection method provided by the embodiment of the present application will be briefly described below. The anomaly detection method provided by the embodiment of the present application can be applied in the application environment shown in Figure 1. Referring to Figure 1, it includes a client 102, multiple data nodes 104 in a server cluster, and a management node 106 in a server cluster. The client 102 and each data node in the server cluster communicate data through a communication network, and each data node in the server cluster communicates with each other through a communication network. The client 102 is used to send a request message to each data node in the server cluster, and instructs the data node that receives the request message to complete the transmission of data between the client 102 and the data node according to the information indicated by the request message; the data node is used to transmit corresponding data to the client 102 or other data nodes according to the information indicated by the request message when receiving the request message sent by the client 102 or other data nodes; the management node 106 is used to associate each data node, for example, to modify the identification information of each data node, and the identification information is used to characterize whether the data node is an abnormal data node. The user terminal 102 may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and other devices.

[0129] It should be noted that the data node status includes the following three states:

[0130] (1) Normal state: Data nodes in normal state can process IO requests normally;

[0131] (2) Abnormal state: The IO of many disks on the data node is slow and blocks user services. At this time, the data node is marked as an abnormal data node. In addition, when the failure on the data node is not super redundant, the abnormal data node does not process IO requests, write operations are directly reported, and read operations are supplemented from other objects. However, when the failure on the data node is super redundant, the abnormal data node can process IO requests.

[0132] (3) High-pressure state: The data node itself is normal, but due to high business pressure, the request queue time is long and the processing is slow. Common reasons for the high-pressure state are uneven disk selection and background tasks. The data management node in the high-pressure state tries not to process read requests, performs supplementary reading within the redundant range, processes write requests normally, and does not report write requests.

[0133] The various states of the data node can be converted to each other, for example, converting the data node from a normal state to an abnormal state, converting the data node from an abnormal state to a normal state, converting the data node from a normal state to a high pressure state, and converting the data node from a high pressure state to a normal state.

[0134] Those skilled in the art will understand that the structure shown in FIG. 1 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. A specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0135] After introducing the application scenarios of the anomaly detection method provided in the embodiments of the present application, the following focuses on introducing the anomaly detection method described in the present application.

[0136] In one embodiment, Figure 2 As shown, an anomaly detection method is provided, which is described by taking the method applied to the client 102 in FIG. 1 as an example, and includes the following steps:

[0137] S201. Obtain an interaction instruction set between the client and each data node.

[0138] Among them, the interactive instruction set refers to the instruction set being processed between the client and each data node. The client will send read instructions, write instructions, truncation instructions, etc. to each data node. Each interactive instruction in the interactive instruction set carries the type of interactive instruction, the duration of the interactive instruction, and the object to which the interactive instruction is sent; the instruction being processed refers to the information that has not been returned by the data node after the client sends a request instruction (for example, a read instruction, a write instruction, a truncation instruction, etc.) to the data node.

[0139] In the embodiment of the present application, the client can check the completion status of the interactive instructions sent by itself to each data node, and determine which nodes are normal nodes and which nodes are abnormal nodes according to the completion status of the interactive instructions. Optionally, the client can call the data input / output (IO) module to obtain the read instructions, write instructions, truncate instructions and other instructions being processed between the client and each data node according to the issuance time of the instructions, and form an interactive instruction set based on the obtained read instructions, write instructions, truncate instructions and other instructions being processed.

[0140] like Figure 3As shown, a part of the interaction instruction set between the client and each data node is shown. The information contained in the interaction instruction 1 (cmd 1) includes: the type of the instruction is a read instruction, the duration of the instruction is 15s, and the instruction is an instruction sent by the client to the data management (data) node 1; the information contained in the interaction instruction 2 (cmd 2) includes: the type of the instruction is a read instruction, the duration of the instruction is 14s, and the instruction is an instruction sent by the client to the data management (data) node 1 and the data management (data) node 2; the information contained in the interaction instruction 3 (cmd 3) includes: the type of the instruction is a write instruction, the duration of the instruction is 13s, and the instruction is an instruction sent by the client to the data management (data) node 1; the information contained in the interaction instruction 4 (cmd 4) includes: the type of the instruction is a write instruction, the duration of the instruction is 12s, and the instruction is an instruction sent by the client to the data management (data) node 1.

[0141] S202: Perform statistical analysis on the interactive instruction set, determine potential abnormal data nodes, and send self-check instructions to the potential abnormal data nodes.

[0142] Among them, a potential abnormal data node refers to a data node that may have an abnormality. Whether a potential abnormal data node is an abnormal data node requires the potential abnormal data node to complete its own self-check operation. The self-check instruction is used to instruct the potential abnormal data node to complete the self-check operation.

[0143] In an embodiment of the present application, after the interaction instruction set between the client and each data node is obtained as described above, a statistical analysis can be performed on each interaction instruction in the interaction instruction set according to the completion status of each interaction instruction, and whether the data node is a potential abnormal data node can be determined based on the statistical analysis results. If the data node is a potential abnormal data node, a self-test instruction is sent to the potential abnormal data node to instruct the potential abnormal data node to complete the self-test operation. If the data node is a normal data node, interaction instructions continue to be sent to the data node to perform data interaction.

[0144] S203: Receive a self-test result returned by the potential abnormal data node, and determine a detection result of the potential abnormal data node according to the self-test result.

[0145] The self-check result includes whether the potential abnormal data node is normal, whether the potential abnormal data node is abnormal, and the type of the potential abnormal data node abnormality.

[0146] In the embodiment of the present application, after the self-test instruction is sent to the potential abnormal data node, the potential abnormal data node performs a self-test operation according to the self-test instruction, generates a self-test result, and returns the self-test result to the client. The client receives the self-test result returned by the potential abnormal data node, and determines the detection result of the potential abnormal data node according to the self-test result. For example, if the self-test result indicates that the potential abnormal data node is normal, the client determines that the potential abnormal data node is a normal data node; if the self-test result indicates that the potential abnormal data node is abnormal, and the type of abnormality, the client determines that the potential abnormal data node is an abnormal data node, and the abnormal type of the data node.

[0147] The anomaly detection method provided in the embodiment of the present application is applied to the client, obtains the interactive instruction set between the client and each data node, performs statistical analysis on the interactive instruction set, determines the potential abnormal data node, and sends a self-test instruction to the potential abnormal data node; receives the self-test result returned by the potential abnormal data node, and determines the detection result of the potential abnormal data node according to the self-test result. The above method performs two anomaly detections, including the client performing statistical analysis on the interactive instruction set, initially detecting the potential abnormal data node, and the potential abnormal data node self-testing again to accurately determine whether the potential abnormal data node is abnormal. Compared with the existing method of directly implementing anomaly detection by judging whether the message connection between the client and the data node is normal, the above method greatly improves the detection accuracy of abnormal data nodes.

[0148] In one embodiment, Figure 2 Based on the embodiment shown, the process of determining potential abnormal data nodes can be described as follows: Figure 4 As shown, the above S202 "performs statistical analysis on the interactive instruction set to determine potential abnormal data nodes" includes:

[0149] S301. Count the number of completed interaction instructions in the interaction instruction set, the average delay of the interaction instructions completed by each data node, and the number of retryable error failures of each data node within a preset time period.

[0150] The interactive instruction set includes completed interactive instructions and uncompleted interactive instructions.

[0151] The preset time period may be a predetermined time period when performing abnormality detection on a data node, such as 1 minute or 2 minutes.

[0152] In an embodiment of the present application, at each moment, the client may send an interaction instruction set generated by an interaction instruction to each data node, count the interaction instruction sets between the client and each data node at multiple moments within a preset time period, analyze the completion status of each interaction instruction in the entire interaction instruction set within the preset time period, obtain the number of interaction instructions completed in the interaction instruction set, the average delay of the interaction instructions completed by each data node, analyze the data nodes sending each interaction instruction, and determine the number of retryable errors of each data node within the preset time period.

[0153] Optionally, the data node that sent the interactive instruction can directly obtain the information carried in the interactive instruction (such as Figure 3 As shown), the method for determining the completion status of the interaction instruction includes: comparing the duration of the interaction instruction with a preset time threshold; if the duration of the interaction instruction is greater than the preset time threshold, the task interaction instruction is not completed; if the duration of the interaction instruction is not greater than the preset time threshold, the interaction instruction is considered to be completed.

[0154] Optionally, at each moment, the client may send interaction instructions to each data node to form an interaction instruction set, and count the interaction instruction sets at multiple moments within a preset time period. Only any 10 interaction instructions in the interaction instruction set at each moment may be counted. For example, an interaction instruction set is obtained once every 1s. If the interaction instruction set within 1 minute is to be counted, a total of 600 interaction instructions are obtained to form the interaction instruction set, so as to avoid the problem of heavy statistical analysis workload caused by too many interaction instructions.

[0155] Optionally, as shown in Figure 5, when determining the potential abnormal data node in the interaction instruction set, each interaction instruction in the interaction instruction set can be traversed to determine whether the duration of each interaction instruction is greater than a preset time threshold. If it is not greater than the preset time threshold, it means that the interaction instruction is a completed interaction instruction, that is, proceed to detect whether the next interaction instruction is greater than the preset time threshold. If it is greater than the preset time threshold, continue to determine whether the data node corresponding to the interaction instruction has a temporary fault super-redundancy situation. If there is no temporary fault super-redundancy situation, it is determined that the interaction instruction is an uncompleted interaction instruction. If there is a temporary fault super-redundancy situation, it means that the interaction instruction is a completed interaction instruction, that is, proceed to detect the next interaction instruction, ..., until all interaction instructions in the interaction instruction set are detected, and the number of completed interaction instructions in the interaction instruction set is obtained, and then the delay of the interaction instructions completed by each data node is counted, the average delay of each data node to complete the interaction instruction, and the number of retryable error failures of each data node within the preset time period is counted.

[0156] For example, Figure 6As shown in the figure, it shows the process of counting the unfinished interaction instructions in the interaction instruction set at multiple times to obtain the completion status of the corresponding interaction instructions of each data node, see Figure 6 , count the completion status of read instructions and write instructions at each moment in time1-time60 and the data nodes that sent them, and obtain the statistical information of read instructions on each data node, including: the number of successful completions of read instructions, the average delay of successful read instructions, and the number of retryable error failures, as well as the statistical information of write instructions on each data node, including: the number of successful completions of write instructions, the average delay of successful write instructions, and the number of retryable error failures.

[0157] S302: Determine potential abnormal data nodes according to the quantity, average latency, and number of retryable error failures.

[0158] In the embodiment of the present application, when the number of interactive instructions completed in the interactive instruction set, the average delay of the completed interactive instructions, and the number of retryable error failures of each data node within a preset time period are obtained, the number of interactive instructions completed in the interactive instruction set, the average delay of the completed interactive instructions, and the number of retryable error failures of each data node within a preset time period can be statistically analyzed to determine potential abnormal data nodes. For example, the ratio between the interactive instructions completed on the data node and the total interactive instructions on the data node is compared with a preset threshold value. If the ratio is greater than the preset threshold value, the data node is considered to be a normal node. If the ratio is less than the preset threshold value, the data node is considered to be a potential abnormal data node.

[0159] Optionally, when the number of completed interaction instructions in the interaction instruction set is obtained as mentioned above, the number of completed interaction instructions corresponding to each data node can be further determined, and then the proportion of the number of completed interaction instructions corresponding to the data node to the total number of interaction instructions in the data node can be determined to determine the potential abnormal data node; or, when the average delay of the completed interaction instructions is obtained as mentioned above, the average delay of the completed interaction instructions corresponding to each data node can be further determined to determine whether the average delay of the completed interaction instructions corresponding to each data node is greater than the average delay of the completed interaction instructions. If so, the data node is determined to be a potential abnormal data node; or, after the number of retryable error failures of each data node within a preset time period is obtained as mentioned above, it is determined whether the number of retryable error failures of each data node is greater than a preset retryable error failure threshold. If so, the data node is determined to be a potential abnormal data node.

[0160] The method for determining potential abnormal data nodes provided in the embodiment of the present application analyzes whether each data node is a potential abnormal data node by acquiring the number of interaction instructions completed in the interaction instruction set, the average latency of the completed interaction instructions, and the number of retryable error failures of each data node within a preset time period, thereby providing a data basis for subsequently determining whether the potential abnormal data node is indeed an abnormal data node.

[0161] In one embodiment, Figure 4 Based on the embodiment shown, the process of determining potential abnormal data nodes can be described as follows: Figure 7 As shown, the above S302 "determine potential abnormal data nodes according to the quantity, average latency, and number of retryable error failures" includes:

[0162] S401: If the quantity is a first value, determining the corresponding data node as a potential abnormal data node.

[0163] S402: If the number is not the first value, determine potential abnormal data nodes according to the maximum average delay in the average delay and the number of retryable error failures.

[0164] The first value may be 0.

[0165] In an embodiment of the present application, after the number of completed interaction instructions in the interaction instruction set is obtained as mentioned above, if the number of interaction instructions completed by the data node within the preset time period is 0, it means that the data node does not have a completed interaction instruction within the preset time period; if the number of interaction instructions completed by the data node within the preset time period is not 0, it means that the data node has completed interaction instructions within the preset time period, then the potential abnormal data node is determined based on the maximum average delay of the average delay of the interaction instructions completed by each data node within the preset time period and the number of retryable error failures.

[0166] In an embodiment of the present application, if the number of interactive instructions completed by a data node within a preset time period is not 0, the average delay of the interactive instructions completed by each data node within the preset time period is counted, and the maximum average delay among the average delays of the interactive instructions completed by each data node within the preset time period is determined, and after obtaining the number of retryable error failures of each data node within the preset time period, the potential abnormal data node is determined based on the maximum average delay and the number of retryable error failures in the average delay. Optionally, the maximum average delay among the average delays of the interactive instructions completed by each data node within the preset time period can be compared with a preset delay threshold. If the maximum average delay is greater than the preset time threshold, it means that the average delay of the interactive instructions completed on the data node is longer. At this time, it means that the data node is a potential abnormal data node.

[0167] The method for determining potential abnormal data nodes provided in the embodiments of the present application is achieved by counting the number of interactive instructions completed on each data node, the maximum average delay in the average delay, and the number of retryable error failures, thereby providing a data basis for subsequent determination of whether the potential abnormal data node is indeed an abnormal data node.

[0168] In one embodiment, Figure 7 Based on the embodiment shown, the process of determining potential abnormal data nodes can be further described, such as Figure 8 As shown, the above S402 "determine the potential abnormal data node according to the maximum average delay and the number of retryable error failures in the average delay" includes:

[0169] S501: If the maximum average delay is greater than a preset delay threshold, a data node corresponding to the maximum average delay is determined as a potential abnormal data node.

[0170] Among them, the preset delay threshold can be a fixed value, and the preset delay threshold can also be first determining other data nodes except the data node corresponding to the maximum average delay, and then determining 5 times the maximum average delay among the average delays of the completed interaction instructions of other data nodes.

[0171] In an embodiment of the present application, the maximum average delay among the average delays of the interaction instructions completed by each data node within a preset time period is obtained, and it is determined whether the maximum average delay is greater than the preset delay threshold. If the maximum average delay is greater than the preset delay threshold, it means that the average delay for the data node to complete the interaction instruction is large, further indicating that the data node is a potential abnormal data node.

[0172] S502: If the maximum average delay is not greater than the preset delay threshold, determine a potential abnormal data node according to the maximum number of retryable error failures among the retryable error failures.

[0173] In an embodiment of the present application, if the maximum average delay is not greater than the preset delay threshold, it means that the average delay of the data node to complete the interaction instruction is not much different from the average delay of other data nodes to complete the interaction instruction. It is necessary to further determine the potential abnormal data node based on the maximum number of retryable error failures among the retryable error failures.

[0174] Optionally, the number of retryable error failures of each data node within a preset time period can be compared with a preset failure number threshold. If the number of retryable error failures of a data node within the preset time period is greater than the preset failure number threshold, it means that the number of retryable error failures on the data node is relatively large. In this case, the data node is a potential abnormal data node.

[0175] The method for determining potential abnormal data nodes provided in the embodiments of the present application is achieved by counting the average delay and the number of retryable error failures of the interactive instructions completed on each data node, thereby providing a data basis for subsequent determination of whether the potential abnormal data nodes are indeed abnormal data nodes.

[0176] In one embodiment, Figure 8 Based on the illustrated embodiment, the process of determining a potential abnormal data node can be further described, such as: Fig. 9 As shown, the above S502 "determining a potential abnormal data node according to the maximum number of retryable error failures among the number of retryable error failures" includes:

[0177] S601: If the maximum number of retryable error failures is greater than a preset number threshold, a data node corresponding to the maximum number of retryable error failures is determined as a potential abnormal data node.

[0178] Among them, the preset number threshold can be a fixed value, and the preset number threshold can also be first determining other data nodes except the data node corresponding to the maximum average delay, and then determining 5 times the maximum average delay among the average delays of the completed interaction instructions of other data nodes.

[0179] In an embodiment of the present application, after the maximum number of retryable error failures among the retryable error failures of each data node within a preset time period is obtained, it is determined whether the maximum number of retryable error failures is greater than a preset number threshold. If the maximum number of retryable error failures is greater than the preset number threshold, it means that the number of retryable error failures on the data node is relatively large. At this time, it means that the data node is a potential abnormal data node.

[0180] S602: If the maximum number of retryable error failures is not greater than a preset number threshold, determine the data node corresponding to the maximum number of retryable error failures as a normal data node.

[0181] In the embodiment of the present application, if the maximum number of retryable error failures is not greater than the preset number threshold, it means that the data node is a normal data node, and the corresponding data node is determined to be a normal data node.

[0182] The method for determining potential abnormal data nodes provided in the embodiments of the present application is implemented by counting the number of retryable error failures on each data node, thereby providing a data basis for subsequent determination of whether the potential abnormal data node is indeed an abnormal data node.

[0183] In one embodiment, Figure 2-Figure 9 Based on the embodiment shown, Fig.10 As shown, the above method also includes:

[0184] S204: Send the exception handling request to the management node.

[0185] The exception handling request includes the detection result, and the exception handling request is used to instruct the management node to perform exception handling on the abnormal data node determined by the detection result.

[0186] In an embodiment of the present application, after the client receives the self-test results returned by the potential abnormal data node and determines the detection results of the potential abnormal data node based on the self-test results, an exception handling request can be generated based on the detection results, and the exception handling request can be sent to the management node to instruct the management node to perform exception handling on the abnormal data node determined by the detection results.

[0187] S205: Receive the exception handling result returned by the management node.

[0188] The abnormal processing result includes a processing result of powering off the abnormal data node, a processing result of restarting the abnormal data node, and updating the identification of the abnormal data node to "abnormal data node".

[0189] In an embodiment of the present application, after the above-mentioned client sends the exception handling request to the management node, the management node will perform exception handling on the abnormal data node determined according to the detection result in the exception management request, and after the exception handling, generate an exception handling result, and send the exception handling result to the client, and the client receives the exception handling result returned by the management node.

[0190] The method for receiving exception processing results provided in the embodiment of the present application feeds back the exception processing results of the management node to the client, so that the client can refer to the exception processing results to determine the status of the data node when sending instructions to the data node, and then determine the sending object of the instruction, thereby avoiding data congestion caused by sending instructions to data nodes with abnormalities.

[0191] In one embodiment, slow disks of data nodes causing slow request processing, pseudo-death of nodes causing request jams, and abnormalities of other data storage nodes causing interaction jams can all lead to node anomalies. Fig.11 As shown, an anomaly detection method is provided, which is described by taking the method applied to the target data node 104 in the data node cluster in FIG. 1 as an example, and includes the following steps:

[0192] S701. Upon receiving a self-check instruction sent by a client, perform a self-check operation to obtain a self-check result.

[0193] The self-check result is used to indicate whether the target data node is an abnormal data node and / or a node abnormality type.

[0194] In the embodiment of the present application, after the client determines the potential abnormal data node by statistically analyzing the interactive instruction set, it can send a self-test instruction to the potential abnormal data node. After the potential abnormal data node receives the self-test instruction sent by the client, it starts the data node self-test program, performs a self-test operation on itself, and obtains the self-test result after the self-test operation is completed. It should be noted that the target data node is any one of the potential abnormal data nodes.

[0195] S702: Send the self-test result to the client.

[0196] In an embodiment of the present application, after the target data node starts a self-test program and obtains the self-test results, the self-test results can be sent to the client so that the client can clearly understand which data nodes are normal data nodes, which data nodes are abnormal data nodes, and the abnormal types of the abnormal data nodes.

[0197] The anomaly detection method provided in the embodiment of the present application performs two anomaly detections, including statistical analysis of the interaction instruction set by the client, initial detection of potential abnormal data nodes, and self-inspection of the potential abnormal data nodes again to accurately determine whether the potential abnormal data nodes are abnormal. Compared with the existing method of directly implementing anomaly detection by determining whether the message connection between the client and the data node is normal, the above method greatly improves the detection accuracy of abnormal data nodes.

[0198] In one embodiment, Fig.11 Based on the embodiment shown, Fig.12 As shown, the above method also includes:

[0199] S703: When receiving a processing instruction sent by the management node, perform a power-off or restart operation according to the processing method indicated by the processing instruction.

[0200] In an embodiment of the present application, after the above-mentioned target data node sends the self-test result to the client, the client will generate an exception handling request based on the self-test result, and send the exception handling request to the management node. After receiving the exception handling request, the management node will generate a processing instruction for the abnormal data node based on the identifier of the abnormal data node and the abnormal type of the abnormal data node indicated in the exception handling request, and send the processing instruction to the target data node. After receiving the processing instruction sent by the management node, the target data node can power off or restart the abnormal data node according to the processing method indicated by the processing instruction.

[0201] For example, if data redundancy will not be generated after removing the abnormal data node, the abnormal data node can be restarted. Also, if the abnormal type of the abnormal data node is frequent reporting by the client within a preset time period, the processing instruction for generating the abnormal data node is to perform a node offline operation on the abnormal data node.

[0202] The exception handling method provided in the embodiment of the present application, when receiving a processing instruction sent by the management node, performs exception handling on itself according to the processing method indicated by the processing instruction, so as to ensure that the data communication between the client and the data node can proceed normally, and avoid the situation where the client waits for a long time and the data node does not respond.

[0203] In one embodiment, Fig.11 Based on the embodiment shown, the process of performing a self-check operation on the target data node to obtain a self-check result can be described as follows: Fig.13 As shown, the above S701 "upon receiving the self-test instruction sent by the client, performing a self-test operation to obtain a self-test result" includes:

[0204] S801. When a self-check instruction is received from a client, a task processing self-check item is triggered to perform a self-check.

[0205] Among them, the task processing self-check item is used to detect whether the data node has a dog feeding timeout.

[0206] In the embodiment of the present application, after the target data node receives the self-check instruction sent by the client, it triggers its own task processing self-check item to perform a self-check on whether a dog feeding timeout occurs.

[0207] Optional, such as Fig.14 As shown, the target data node can also determine whether there is a self-check task of the target data node in progress at the current moment when receiving the self-check instruction sent by the client. If there is an ongoing self-check task, it is necessary to wait for the ongoing self-check task to end. If there is no ongoing self-check task, it is determined whether there is a self-check result of the target data node within 5 seconds. If so, directly check the self-check result of the target data node within 5 seconds to determine whether there is an abnormality in the target data node and the type of abnormality. If not, trigger its own task processing self-check item to perform a self-check on whether the dog feeding timeout has occurred, and determine whether there is an abnormality in the target data node based on the self-check result of the task processing self-check item.

[0208] S802, determine whether there is a timeout task in the self-check process of the task processing self-check item, if yes, execute the following S803, if not, execute the following S804.

[0209] In an embodiment of the present application, after the above-mentioned triggered task processing self-check item performs self-check, it is possible to check whether there is a timeout task in the self-check record generated by the task processing self-check item during the self-check process. If there is a timeout task, execute the following S803; if there is no timeout task, execute the following S804.

[0210] S803: Obtain a self-test result including a task processing timeout exception.

[0211] In an embodiment of the present application, if there is a timed task in the self-check record generated during the self-check process of checking the task processing self-check item, it is determined that there is a task processing timeout exception in the target data node, and based on the existence of the task processing timeout exception in the target data node, a self-check result including the task processing timeout exception is generated.

[0212] S804: Trigger an abnormal self-check item to perform a self-check and obtain a self-check result.

[0213] Among them, the abnormal self-check item includes at least one of the node abnormal self-check item, the network abnormal self-check item, the interaction abnormal self-check item, the jam abnormal self-check item, the disk abnormal self-check item, and the disk interaction abnormal self-check item.

[0214] In an embodiment of the present application, if there is no timeout task in the self-check record generated during the self-check process of checking the task processing self-check item, it is determined that there is no task processing timeout exception on the target data node, and then the abnormal self-check item is triggered to perform self-check to obtain the self-check result of whether the data node has a node abnormality.

[0215] In one embodiment, Fig.13 Based on the embodiment shown, the process of triggering the abnormal self-check item to perform self-check can be described as follows: Fig.15 As shown, the above S804 "triggering an abnormal self-checking item to perform self-checking and obtain a self-checking result" includes:

[0216] S901, triggering the node abnormality self-checking item to perform self-checking, and determining whether there is a node abnormality during the self-checking process of the node abnormality self-checking item. If there is, execute the following S902; if not, execute the following S903.

[0217] In an embodiment of the present application, after triggering the node abnormality self-check item for self-check, it is possible to check whether there is a node abnormality in the self-check record generated by the node abnormality self-check item during the self-check process. If there is a node abnormality, execute the following S902; if there is no node abnormality, execute the following S903.

[0218] S902: Obtain self-check results including node abnormalities.

[0219] In an embodiment of the present application, if there is a node anomaly in the self-inspection record generated during the self-inspection process of the above-mentioned node anomaly self-inspection item, it is determined that there is a node anomaly in the target data node, and based on the existence of the node anomaly in the target data node, a self-inspection result including the node anomaly is generated.

[0220] S903: Trigger the network abnormality self-check item to perform self-check and obtain the self-check result.

[0221] Among them, the network anomaly self-check item is used to detect whether there is a network anomaly in the data node.

[0222] In an embodiment of the present application, when there is no node abnormality in the self-inspection record generated during the self-inspection process of the above-mentioned node abnormality self-inspection item, it is determined that there is no node abnormality in the target data node, and then the network abnormality self-inspection item is triggered to perform self-inspection to obtain the self-inspection result of whether there is a network abnormality in the data node.

[0223] The self-check method provided in the embodiment of the present application determines whether a data node has a node abnormality by triggering a node abnormality self-check item in the data node, thereby achieving the effect of accurately judging the type of data node abnormality.

[0224] In one embodiment, Fig.15 Based on the embodiment shown, the process of triggering the network abnormality self-check item to perform self-check can be described as follows: Fig.16 As shown, the above S903 "triggering a network abnormality self-checking item to perform a self-check and obtain a self-checking result" includes:

[0225] S1001. Determine whether there is a network anomaly during the self-checking process of the network anomaly self-checking item. If yes, execute the following S1002. If not, execute the following S1003.

[0226] In an embodiment of the present application, after the above-mentioned network abnormality self-check item is triggered for self-check, it is possible to check whether there is a network abnormality in the self-check record generated by the node abnormality self-check item during the self-check process. If there is a network abnormality, execute the following S1002; if there is no network abnormality, execute the following S1003.

[0227] S1002: Obtain self-check results including network anomalies.

[0228] In an embodiment of the present application, if there is a network anomaly in the self-inspection record generated during the self-inspection process of checking the network anomaly self-inspection item, it is determined that there is a network anomaly in the target data node, and based on the existence of the network anomaly in the target data node, a self-inspection result including the network anomaly is generated.

[0229] It should be noted that network anomalies include diagnostic message network timeout anomalies and other network anomalies of diagnostic messages. For diagnostic message network timeout anomalies, the data node will resend the diagnostic message when it receives the first returned network timeout error. If the diagnostic message timeout error is returned twice in a row, the node is considered abnormal. For other network anomalies of diagnostic messages, if other network anomalies of diagnostic messages are returned five times in a row, the node is considered abnormal.

[0230] S1003: Trigger the interaction abnormality self-check item to perform self-check and obtain the self-check result.

[0231] The interaction anomaly self-check item is used to detect whether there is an interaction anomaly between the data node and other data nodes.

[0232] In an embodiment of the present application, when there is no network anomaly in the self-inspection record generated during the self-inspection process of checking the network anomaly self-inspection item, it is determined that there is no network anomaly in the target data node, and then the interaction anomaly self-inspection item is triggered to perform self-inspection to obtain a self-inspection result of whether there is an interaction anomaly in the data node.

[0233] The self-check method provided in the embodiment of the present application determines whether there is a network anomaly in the data node by triggering the network anomaly self-check item in the data node, thereby achieving the effect of accurately judging the type of data node anomaly.

[0234] In one embodiment, Fig.16 Based on the embodiment shown, the process of triggering the interaction abnormality self-check item to perform self-check can be described as follows: Fig.17 As shown, the above S1003 "triggering the interaction abnormality self-checking item to perform self-checking and obtain the self-checking result" includes:

[0235] S1101. Determine whether there is an interaction anomaly during the self-checking process of the interaction anomaly self-checking item. If so, execute the following S1102; if not, execute the following S1103.

[0236] In an embodiment of the present application, after the above-mentioned interaction abnormality self-check item is triggered for self-check, it is possible to check whether there is an interaction abnormality in the self-check record generated by the interaction abnormality self-check item during the self-check process. If there is an interaction abnormality, execute the following S1102; if there is no interaction abnormality, execute the following S1103.

[0237] S1102: Obtain self-test results including interaction anomalies.

[0238] In an embodiment of the present application, when there is an interaction anomaly in the self-inspection record generated during the self-inspection process of checking the interaction anomaly self-inspection item, it is determined that there is an interaction anomaly in the target data node, and based on the existence of the interaction anomaly in the target data node, a self-inspection result including the interaction anomaly is generated.

[0239] It should be noted that if there is an interaction anomaly with the target data node, the client needs to monitor the data node that interacts with the target data node for anomalies. If it is detected that the data node that interacts with the target data node has an abnormality, the client reports the abnormality of the data node that interacts with the target data node to the management node. If it is detected that there is no abnormality in the data node that interacts with the target data node, the client reports the abnormality of the target data node to the management node.

[0240] S1103, trigger the jamming abnormality self-check item to perform self-check and obtain the self-check result.

[0241] Among them, the jam abnormality self-check item is used to detect whether there is a jam abnormality in the data node.

[0242] In an embodiment of the present application, when there is no interaction anomaly in the self-test record generated during the self-test process of checking the interaction anomaly self-test item, it is determined that there is no interaction anomaly in the target data node, and then the jamming anomaly self-test item is triggered to perform self-test to obtain a self-test result of whether the data node has a jamming anomaly.

[0243] The self-check method provided in the embodiment of the present application determines whether there is an interaction anomaly in the data node by triggering the interaction anomaly self-check item in the data node, thereby achieving the effect of accurately judging the type of data node anomaly.

[0244] In one embodiment, Fig.17 Based on the embodiment shown, the process of self-checking the self-check item that triggers the jamming abnormality can be described as follows: Fig.18 As shown, the above S1103 "triggering the jamming abnormality self-checking item to perform self-checking and obtain the self-checking result" includes:

[0245] S1201. Determine whether there is a jamming exception during the self-check process of the jamming exception self-check item. If so, execute the following S1202; if not, execute the following S1203.

[0246] In an embodiment of the present application, after the above-mentioned self-check item that triggers the jamming exception is self-checked, it can be checked whether there is a jamming exception in the self-check record generated by the jamming self-check item during the self-check process. If there is a jamming exception, execute the following S1202; if there is no jamming exception, execute the following S1203.

[0247] S1202. Obtain self-test results including freeze abnormalities.

[0248] In an embodiment of the present application, when there is a jamming abnormality in the self-test record generated during the self-test process of checking the jamming abnormality self-test item, it is determined that there is a jamming abnormality in the target data node, and based on the existence of the jamming abnormality in the target data node, a self-test result including the jamming abnormality is generated.

[0249] S1203, triggering the disk abnormality self-check item to perform a self-check and obtain a self-check result.

[0250] Among them, the disk abnormality self-check item is used to detect whether there is a disk abnormality in the data node.

[0251] In an embodiment of the present application, when there is no jamming abnormality in the self-test record generated during the self-test process of checking the jamming abnormality self-test item, it is determined that there is no jamming abnormality in the target data node, and then the disk abnormality self-test item is triggered to perform self-test to obtain the self-test result of whether the data node has a disk abnormality.

[0252] The self-check method provided in the embodiment of the present application determines whether a data node has a jamming abnormality by triggering a jamming abnormality self-check item in the data node, thereby achieving the effect of accurately judging the abnormality type of the data node.

[0253] In one embodiment, Fig.18 Based on the embodiment shown, the process of triggering the self-check item of the disk abnormality self-check to perform the self-check can be described as follows: Fig.19 As shown, the above S1203 "triggering a disk abnormality self-check item to perform a self-check and obtain a self-check result" includes:

[0254] S1301. Determine whether there is a disk abnormality during the self-check process of the disk abnormality self-check item. If yes, execute the following S1302; if not, execute the following S1303.

[0255] In an embodiment of the present application, after the above-mentioned disk abnormality self-test item is triggered for self-test, it is possible to check whether there is a disk abnormality in the self-test record generated by the disk self-test item during the self-test process. If there is a disk abnormality, execute the following S1302; if there is no disk abnormality, execute the following S1303.

[0256] S1302: Obtain self-test results including disk abnormalities.

[0257] In an embodiment of the present application, when there is a disk anomaly in the self-test record generated during the self-test process of checking the disk anomaly self-test item, it is determined that there is a disk anomaly in the target data node, and based on the existence of the disk anomaly in the target data node, a self-test result including the disk anomaly is generated.

[0258] S1303: Trigger the disk interaction abnormality self-check item to perform a self-check and obtain a self-check result.

[0259] The disk interaction anomaly self-check item is used to detect whether there is an interaction anomaly between the data node and other disks.

[0260] In an embodiment of the present application, when there is no disk abnormality in the self-test record generated during the self-test process of checking the disk abnormality self-test item, it is determined that there is no disk abnormality in the target data node, and then the disk interaction abnormality self-test item is triggered to perform self-test to obtain the self-test result of whether the data node has a disk interaction abnormality.

[0261] The self-check method provided in the embodiment of the present application determines whether a data node has a disk abnormality by triggering a disk abnormality self-check item in the data node, thereby achieving the effect of accurately judging the abnormality type of the data node.

[0262] In one embodiment, Fig.19Based on the embodiment shown, the process of triggering the self-check item of disk interaction abnormality to perform self-check can be described as follows: Fig. 20 As shown, the above S1303 "triggering the disk interaction abnormality self-check item to perform self-check and obtain the self-check result" includes:

[0263] S1401. Determine whether there is a disk interaction abnormality during the self-check process of the disk interaction abnormality self-check item. If yes, execute the following S1402; if not, execute the following S1403.

[0264] In an embodiment of the present application, after the above-mentioned self-test item of triggering disk interaction abnormality is performed, it is possible to check whether there is a disk interaction abnormality in the self-test record generated by the disk interaction self-test item during the self-test process. If there is a disk interaction abnormality, execute the following S1302; if there is no disk interaction abnormality, execute the following S1303.

[0265] S1402: Obtain self-test results including disk interaction abnormalities.

[0266] In an embodiment of the present application, when there is a disk interaction anomaly in the self-test record generated during the self-test process of checking the disk interaction anomaly self-test item, it is determined that there is a disk interaction anomaly in the target data node, and based on the existence of the disk interaction anomaly in the target data node, a self-test result including the disk interaction anomaly is generated.

[0267] S1403: Return a self-check result indicating that the business pressure exceeds a preset threshold.

[0268] In an embodiment of the present application, when there is no disk interaction anomaly in the self-test record generated during the self-test process of checking the disk interaction anomaly self-test item, it is determined that there is no disk interaction anomaly in the target data node, and a self-test result that the business pressure exceeds the preset threshold is returned.

[0269] It should be noted that when the target data node returns the self-test result that the business pressure exceeds the preset threshold to the client, the client does not need to report the self-test result that the business pressure exceeds the preset threshold to the management node. The client only needs to maintain the high pressure state by itself. For example, for read operations that are not excessively redundant, avoid disks or data nodes whose business pressure exceeds the preset threshold, and regularly check whether the data nodes or disks whose business pressure exceeds the preset threshold are still in a high pressure state. If recovered, the read operation can be processed normally on the data node that has recovered to normal.

[0270] The self-check method provided in the embodiment of the present application determines whether a data node has a disk interaction anomaly by triggering a disk interaction anomaly self-check item in the data node, thereby achieving the effect of accurately judging the type of data node anomaly.

[0271] The self-check method provided in the embodiment of the present application determines whether a data node has a task processing timeout exception by triggering a task processing self-check item in the data node, thereby achieving the effect of accurately judging the type of data node exception.

[0272] In one embodiment, see Fig.14 , and also provides a complete anomaly detection method, including: when the target data node receives the self-check instruction sent by the client, it determines whether there is a self-check task of the target data node in progress at the current moment. If there is an ongoing self-check task, it is necessary to wait for the ongoing self-check task to end. If there is no ongoing self-check task, it is determined whether there is a self-check result of the target data node within 5s. If so, directly check the self-check result of the target data node within 5s to determine whether there is an abnormality in the target data node and the type of abnormality. If not, trigger its own task processing self-check item to self-check whether it has a dog feeding timeout. If so, obtain a self-check result including a task processing timeout exception. If not, determine whether there is a node abnormality in the self-check process of the node abnormality self-check item (detection thread, log, etc.). If so, obtain a self-check result including a node abnormality. If not, determine the network abnormality self-check item (network nrq If not, determine whether there is a network anomaly in the self-test process of the interaction anomaly self-test item (whether there is an interaction request with a long period of freeze). If so, obtain a self-test result including the interaction anomaly. If not, determine whether there is a freeze anomaly in the self-test process of the freeze anomaly self-test item (whether there is a freeze service thread). If so, obtain a self-test result including the freeze anomaly. If not, determine whether there is a disk anomaly in the self-test process of the disk anomaly self-test item. If so, obtain a self-test result including the disk anomaly. If not, determine whether there is a disk interaction anomaly in the self-test process of the disk interaction anomaly self-test item (whether there is a command that is executed for more than 10s and is stuck in the disk interaction stage). If so, obtain a self-test result including the disk interaction anomaly. If not, return the self-test result that the business pressure exceeds the preset threshold.

[0273] It should be noted that if the client detects that a data node is abnormal three times in a row, but the data node considers itself normal after self-checking, and there is no abnormality in the interaction between the client and other data nodes during this period, the client needs to report the data node abnormality to the management node.

[0274] The exception handling method provided in the embodiment of the present application, when receiving a processing instruction sent by the management node, performs exception handling on itself according to the processing method indicated by the processing instruction, so as to ensure that the data communication between the client and the data node can proceed normally, and avoid the situation where the client waits for a long time and the data node does not respond.

[0275] In one embodiment, Fig.21 As shown, an anomaly detection method is provided, which is described by taking the method applied to the management data node 106 in the data node cluster in FIG. 1 as an example, and includes the following steps:

[0276] S1501. Receive an exception handling request sent by a client.

[0277] The exception handling request includes the detection result; the exception handling request is used to instruct the management node to perform exception handling on the abnormal data node determined by the detection result.

[0278] In an embodiment of the present application, after the client determines the potential abnormal data node based on the interaction instruction set between the client and each data node, the client will send a self-test instruction to the potential abnormal data node. After the potential abnormal data node receives the self-test instruction sent by the client, it performs a self-test operation, obtains and sends the test result to the client, and the client generates an exception handling request based on the self-test result, and sends the exception handling request to the management data node. The management data node receives the exception handling request sent by the client.

[0279] S1502: Perform exception processing on the abnormal data nodes determined by the detection results.

[0280] Among them, exception handling includes power-off processing, restart processing, and modifying the identification of abnormal data nodes.

[0281] In an embodiment of the present application, after the management node obtains the exception handling request sent by the client, the management node performs exception handling on the abnormal data node determined according to the detection result in the exception handling request.

[0282] The above method improves the detection accuracy of abnormal data nodes through two abnormality detections, including the client performing statistical analysis on the interaction instruction set, initially detecting potential abnormal data nodes, and the potential abnormal data nodes performing self-inspection again to accurately determine whether the potential abnormal data nodes are abnormal; in addition, a method for processing abnormal data nodes is also provided, which can efficiently process abnormal data nodes and avoid the client sending instructions to abnormal data nodes, causing data congestion.

[0283] In one embodiment, Fig.21 Based on the embodiment shown, the process of triggering the self-check item of disk interaction abnormality to perform self-check can be described as follows: Fig. 22 As shown, the above S1502 "performing abnormal processing on abnormal data nodes determined by the detection results" includes:

[0284] S1601. Count the number of times the client reports an exception handling request within a preset time period.

[0285] The number of times indicates the number of times each abnormal data node request is processed.

[0286] The preset duration may be 1 hour.

[0287] In an embodiment of the present application, the management data node may analyze the exception handling requests reported by the client within a preset time period, and determine the number of times each data node is required to perform exception handling in the exception handling request.

[0288] S1602: Power off the abnormal data nodes corresponding to the number of times greater than the preset number threshold.

[0289] In an embodiment of the present application, after obtaining the number of times each data node requires exception processing within the preset time period, it is determined whether the number of times for each data node is greater than the preset number threshold, and if the number of times for the data node is greater than the preset number threshold, the abnormal data node corresponding to the number greater than the preset number threshold is powered off.

[0290] S1603: According to the redundancy of the abnormal data nodes, perform abnormal processing on the abnormal data nodes corresponding to the number of times not greater than the preset number threshold.

[0291] In an embodiment of the present application, after the number of times each data node requires exception processing within the preset time length is obtained as mentioned above, it is determined whether the number of times for each data node is greater than the preset number threshold, and when the number of times for the data node is not greater than the preset number threshold, the redundancy of the abnormal data node is detected, and the abnormal data node whose redundancy exceeds the redundancy requirement is indicated, and the redundancy detection is performed again after a preset time period. In addition, if the redundancy of the abnormal data node does not exceed the redundancy requirement of the abnormal data node, the abnormal data node is restarted, and after the abnormal data node is restarted, the node status identifier of the abnormal data node is updated to a normal data node.

[0292] The exception handling method provided in the embodiment of the present application performs exception handling on different types of abnormal data nodes, thereby preventing the client from sending instructions to the abnormal data nodes, causing data congestion.

[0293] In one embodiment, Fig. 22Based on the embodiment shown, the process of performing abnormal processing on abnormal data nodes corresponding to the number of abnormal data nodes not greater than the preset number threshold according to the redundancy of the abnormal data nodes can be described as follows: Fig.23 As shown, the above S1603 "redundancy of abnormal data nodes, abnormal processing of abnormal data nodes corresponding to the number of times not greater than the preset number threshold" includes:

[0294] S1701. Determine whether the redundancy is greater than a preset redundancy threshold. If the redundancy is greater than the preset redundancy threshold, then after a preset time, return to determine whether the redundancy is greater than the preset redundancy threshold. If the redundancy is not greater than the preset redundancy threshold, execute the following S1702.

[0295] In an embodiment of the present application, after the number of processing requests of the data node within the preset time length is determined as above, if the number of processing requests of the data node within the preset time length is not greater than the preset number threshold, the redundancy of the data node is detected to obtain the redundancy of the data node, and it is determined whether the redundancy of the data node is greater than the preset redundancy threshold. If the redundancy of the data node is greater than the preset redundancy threshold, the data node is instructed to wait for a preset time length, and the redundancy of the data node is continued to be detected after the preset time length to obtain a new redundancy of the data node, and it is determined again whether the new redundancy of the data node is greater than the preset redundancy threshold. When the new redundancy of the new data node is greater than the preset redundancy threshold, the data node is continued to be instructed to wait for the preset time length, and the redundancy of the data node is continued to be detected after the preset time length until the redundancy of the data node is no greater than the preset redundancy threshold.

[0296] It should be noted that during the waiting process, each data node is processing the corresponding data, so the redundancy of each node will be reduced. Therefore, the redundancy of the data node after the preset time must be lower than the redundancy of the data node before the preset time.

[0297] S1702: Restart the corresponding abnormal data node, and update the identification of the restarted abnormal data node to normal.

[0298] In the embodiment of the present application, when the redundancy of the data node is determined to be not greater than the preset redundancy threshold, the corresponding abnormal data node is restarted, and the identification of the abnormal data node after the restart is updated to normal.

[0299] The exception handling method provided in the embodiment of the present application performs exception handling on different types of abnormal data nodes to avoid the client sending instructions to the abnormal data nodes, causing data congestion.

[0300] In one embodiment, Fig.24As shown, a complete anomaly detection method is also provided, including: after the management node receives the anomaly handling request sent by the client, it determines the abnormal data node. If the anomaly handling request includes multiple abnormal data nodes, it selects any abnormal data node as the target abnormal data node, sets the identifier of the target abnormal data node as abnormal, generates an abnormal handling report, and feeds back the abnormal handling report to the client. In addition, the management node counts the number of times the client reports the abnormal handling request within a preset time period, and powers off the abnormal data nodes corresponding to the number of times greater than the preset number threshold. For the abnormal data nodes corresponding to the number of times not greater than the preset number threshold, it determines whether the redundancy of the abnormal data node is greater than the preset redundancy threshold. If the redundancy is greater than the preset redundancy threshold, after the preset time period, it returns to re-determine whether the redundancy is greater than the preset redundancy threshold. If the redundancy is not greater than the preset redundancy threshold, the corresponding abnormal data node is restarted, and the identifier of the restarted abnormal data node is updated to normal.

[0301] The method for processing abnormal data nodes provided above can efficiently process abnormal data nodes and avoid the situation where the client sends instructions to the abnormal data nodes, causing data congestion.

[0302] In one embodiment, as shown in FIG. 25 , a complete client-data node-management node, three-end interaction anomaly detection schematic diagram is also provided, including:

[0303] S10, obtaining an interaction instruction set between the client and each data node;

[0304] S11, performing statistical analysis on the interactive instruction set to determine potential abnormal data nodes;

[0305] S12, sending a self-check instruction to a potential target data node;

[0306] S13, upon receiving the self-test instruction sent by the client, performing a self-test operation to obtain a self-test result; S14, sending the self-test result to the client;

[0307] S15, receiving the self-test result returned by the potential abnormal data node, and determining the detection result of the potential abnormal data node according to the self-test result;

[0308] S16. Generate an exception handling request according to the detection result;

[0309] S17, sending the exception handling request to the management node;

[0310] S18, receiving an exception handling request sent by the client;

[0311] S19, performing exception processing on the abnormal data node determined by the detection result in the exception processing request;

[0312] S20. Send the exception handling result to the client.

[0313] The above method uses two anomaly detections, including the client performing statistical analysis on the interaction instruction set, initially detecting potential abnormal data nodes, and the potential abnormal data nodes performing self-inspection again to accurately determine whether the potential abnormal data nodes are abnormal. Compared with the existing method of directly implementing anomaly detection by judging whether the message connection between the client and the data node is normal, the above method greatly improves the detection accuracy of abnormal data nodes.

[0314] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0315] Based on the same inventive concept, the embodiment of the present application also provides an anomaly detection device for implementing the anomaly detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more anomaly detection device embodiments provided below can refer to the limitations on the anomaly detection method above, and will not be repeated here.

[0316] In an exemplary embodiment, as shown in FIG26 , an abnormality detection device is provided, including: an acquisition module 10, a determination module 11 and a receiving module 12, wherein:

[0317] The acquisition module 10 is used to acquire the interaction instruction set between the client and each data node.

[0318] The determination module 11 is used to perform statistical analysis on the interactive instruction set, determine potential abnormal data nodes, and send self-check instructions to the potential abnormal data nodes.

[0319] The receiving module 12 is used to receive the self-test result returned by the potential abnormal data node, and determine the detection result of the potential abnormal data node according to the self-test result.

[0320] In an exemplary embodiment, the receiving module 12 includes: a statistical unit and a determination unit, wherein:

[0321] A statistical unit, specifically used to count the number of interactive instructions completed in the interactive instruction set, the average delay of the interactive instructions completed by each data node, and the number of retryable error failures of each data node within a preset time period;

[0322] The determination unit is specifically used to determine potential abnormal data nodes according to the quantity, average latency, and number of retryable error failures.

[0323] In an exemplary embodiment, the above-mentioned determination unit is specifically used to determine the corresponding data node as a potential abnormal data node when the quantity is a first value; when the quantity is not the first value, the potential abnormal data node is determined based on the maximum average delay in the average delay and the number of retryable error failures.

[0324] In an exemplary embodiment, the above-mentioned determination unit is specifically used to determine the data node corresponding to the maximum average delay as a potential abnormal data node when the maximum average delay is greater than a preset delay threshold; when the maximum average delay is not greater than the preset delay threshold, determine the potential abnormal data node according to the maximum number of retryable error failures among the retryable error failures.

[0325] In an exemplary embodiment, the above-mentioned determination unit is specifically used to determine the data node corresponding to the maximum number of retryable error failures as a potential abnormal data node when the maximum number of retryable error failures is greater than a preset number threshold; and to determine the data node corresponding to the maximum number of retryable error failures as a normal data node when the maximum number of retryable error failures is not greater than the preset number threshold.

[0326] In an exemplary embodiment, the above device further includes: a sending module and a receiving module, wherein:

[0327] A sending module, used to send an exception handling request to a management node; the exception handling request includes a detection result; the exception handling request is used to instruct the management node to perform exception handling on an abnormal data node determined by the detection result;

[0328] The receiving module is used to receive the exception handling result returned by the management node.

[0329] In an exemplary embodiment, as shown in FIG. 27 , an abnormality detection device is provided, including: a self-checking module 20 and a sending module 22, wherein:

[0330] The self-check module 20 is used to perform a self-check operation upon receiving a self-check instruction sent by the client, and obtain a self-check result; the self-check result is used to indicate whether the target data node is an abnormal data node and / or a node abnormality type;

[0331] The sending module 22 is used to send the self-test result to the client.

[0332] In an exemplary embodiment, the above-mentioned device further includes: a receiving module, which is used to perform a power-off or restart operation according to a processing method indicated by the processing instruction when receiving a processing instruction sent by the management node.

[0333] In an exemplary embodiment, the self-check module 20 includes: a trigger unit and a determination unit, wherein:

[0334] The trigger unit is specifically used to trigger the task processing self-check item to perform self-check upon receiving the self-check instruction sent by the client;

[0335] The determination unit is specifically used to determine whether there is a timeout task in the self-check process of the task processing self-check item. If so, a self-check result including a task processing timeout exception is obtained; if not, the abnormal self-check item is triggered to perform self-check and obtain a self-check result; the abnormal self-check item includes at least one of a node abnormal self-check item, a network abnormal self-check item, an interaction abnormal self-check item, a freeze abnormal self-check item, a disk abnormal self-check item, and a disk interaction abnormal self-check item.

[0336] In an exemplary embodiment, as shown in FIG. 28 , an abnormality detection device is provided, including: a receiving module 30 and a processing module 31, wherein:

[0337] The receiving module 30 is used to receive an exception handling request sent by the client; the exception handling request includes the detection result; the exception handling request is used to instruct the management node to perform exception handling on the abnormal data node determined by the detection result;

[0338] The processing module 31 is used to perform abnormal processing on the abnormal data nodes determined by the detection results.

[0339] In an exemplary embodiment, the processing module 31 includes: a statistical unit, a processing unit and an exception handling unit, wherein:

[0340] A statistical unit, specifically used to count the number of times the client reports an exception processing request within a preset time period; the number indicates the number of times each abnormal data node request is processed;

[0341] A processing unit, specifically configured to power off abnormal data nodes corresponding to a number of times greater than a preset number threshold;

[0342] The exception processing unit is specifically used to perform exception processing on the abnormal data nodes corresponding to the number of times not greater than a preset number threshold according to the redundancy of the abnormal data nodes.

[0343] In an exemplary embodiment, the above-mentioned exception processing unit is specifically used to determine whether the redundancy is greater than a preset redundancy threshold; if the redundancy is greater than the preset redundancy threshold, then after a preset time period, return to execute the step of determining whether the redundancy is greater than the preset redundancy threshold; if the redundancy is not greater than the preset redundancy threshold, then the corresponding abnormal data node is restarted, and the identification of the restarted abnormal data node is updated to normal.

[0344] Each module in the above-mentioned abnormality detection device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0345] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG29. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store detection result data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an abnormality detection method is implemented.

[0346] Those skilled in the art will understand that the structure shown in FIG. 29 is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0347] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0348] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0349] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0350] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0351] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0352] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0353] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An anomaly detection method, characterized in that: Applied to a client, the method comprises: Obtaining an interaction instruction set between the client and each data node; Performing statistical analysis on the interactive instruction set, determining potential abnormal data nodes, and sending self-check instructions to the potential abnormal data nodes; A self-test result returned by the potential abnormal data node is received, and a detection result of the potential abnormal data node is determined according to the self-test result.

2. The method according to claim 1, characterized in that The performing statistical analysis on the interactive instruction set to determine potential abnormal data nodes includes: Counting the number of completed interaction instructions in the interaction instruction set, the average delay of the interaction instructions completed by each data node, and the number of retryable error failures of each data node within a preset time period; The potential abnormal data node is determined according to the quantity, the average delay, and the number of retryable error failures.

3. The method according to claim 2, characterized in that The determining the potential abnormal data node according to the quantity, the average delay, and the number of retryable error failures includes: If the number is a first value, determining the corresponding data node as the potential abnormal data node; If the number is not the first value, the potential abnormal data node is determined according to the maximum average delay in the average delays and the number of retryable error failures.

4. The method according to claim 3, characterized in that The determining the potential abnormal data node according to the maximum average delay in the average delays and the number of retryable error failures includes: If the maximum average delay is greater than the preset delay threshold, determining the data node corresponding to the maximum average delay as the potential abnormal data node; If the maximum average delay is not greater than the preset delay threshold, the potential abnormal data node is determined according to the maximum number of retryable error failures among the retryable error failures.

5. The method according to claim 4, characterized in that The determining the potential abnormal data node according to the maximum number of retryable error failures among the number of retryable error failures includes: If the maximum number of retryable error failures is greater than a preset number threshold, the data node corresponding to the maximum number of retryable error failures is determined as the potential abnormal data node; If the maximum number of retryable error failures is not greater than the preset number threshold, the data node corresponding to the maximum number of retryable error failures is determined as a normal data node.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Sending an exception handling request to a management node; the exception handling request includes the detection result; the exception handling request is used to instruct the management node to perform exception handling on the abnormal data node determined by the detection result; Receive the exception handling result returned by the management node.

7. An anomaly detection method, characterized in that: Applied to a target data node in a data node cluster, the method comprises: Upon receiving the self-check instruction sent by the client, a self-check operation is performed to obtain a self-check result; the self-check result is used to indicate whether the target data node is an abnormal data node and / or a node abnormality type; The self-check result is sent to the client.

8. The method according to claim 7, characterized in that The method further comprises: When receiving the processing instruction sent by the management node, a power-off or restart operation is performed according to the processing method indicated by the processing instruction.

9. The method according to claim 7, characterized in that: When receiving the self-check instruction sent by the client, performing a self-check operation and obtaining a self-check result includes: Upon receiving the self-check instruction sent by the client, triggering the task processing self-check item to perform self-check; Determine whether there is a timeout task during the self-check process of the task processing self-check item. If so, obtain a self-check result including a task processing timeout exception; if not, trigger the abnormal self-check item to perform self-check and obtain the self-check result; the abnormal self-check item includes at least one of a node abnormal self-check item, a network abnormal self-check item, an interaction abnormal self-check item, a freeze abnormal self-check item, a disk abnormal self-check item, and a disk interaction abnormal self-check item.

10. An anomaly detection method, characterized in that: Applied to managing data nodes in a data node cluster, the method comprises: Receive an exception handling request sent by a client; the exception handling request includes a detection result; the exception handling request is used to instruct the management node to perform exception handling on the abnormal data node determined by the detection result; Perform exception processing on the abnormal data nodes determined by the detection results.

11. The method according to claim 10, characterized in that The performing abnormal processing on the abnormal data node determined by the detection result includes: Counting the number of times the client reports an exception processing request within a preset time period; the number indicates the number of times each of the abnormal data node requests is processed; Power off the abnormal data nodes corresponding to the number of times greater than the preset number threshold; According to the redundancy of the abnormal data nodes, abnormal processing is performed on the abnormal data nodes corresponding to the number of times not greater than the preset number threshold.

12. The method according to claim 11, characterized in that The performing abnormal processing on the abnormal data nodes corresponding to the number of times not greater than the preset number threshold according to the redundancy of the abnormal data nodes includes: Determining whether the redundancy is greater than a preset redundancy threshold; If the redundancy is greater than a preset redundancy threshold, then after a preset time period, returning to the step of determining whether the redundancy is greater than the preset redundancy threshold; If the redundancy is not greater than a preset redundancy threshold, the corresponding abnormal data node is restarted, and the identification of the abnormal data node after the restart is updated to normal.

13. An abnormality detection device, characterized in that: Applied to a client, the device comprises: An acquisition module, used to acquire an interaction instruction set between the client and each data node; A determination module, configured to perform statistical analysis on the interactive instruction set, determine potential abnormal data nodes, and send self-check instructions to the potential abnormal data nodes; The receiving module is used to receive the self-test result returned by the potential abnormal data node, and determine the detection result of the potential abnormal data node according to the self-test result.

14. An abnormality detection device, characterized in that: Applied to a target data node in a data node cluster, the device comprises: A self-check module, configured to perform a self-check operation upon receiving a self-check instruction sent by a client, and obtain a self-check result; the self-check result is used to indicate whether the target data node is an abnormal data node and / or a node abnormality type; A sending module is used to send the self-test result to the client.

15. An abnormality detection device, characterized in that: Applicable to managing data nodes in a data node cluster, the device comprises: A receiving module, configured to receive an exception handling request sent by a client; the exception handling request includes a detection result; the exception handling request is used to instruct the management node to perform exception handling on the abnormal data node determined by the detection result; A processing module is used to perform exception processing on the abnormal data nodes determined by the detection results.

16. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

18. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images