Anomaly detection method and device for microservice system

Abstract

The present invention discloses an abnormality detection method and device for a microservice system, wherein the method obtains a detection result by training a target node data feature vector corresponding to the data features of each node of a target node call link or a target node message queue storing the target node data feature vector based on a node abnormality detection model, and can quickly determine the target node call link where an abnormal fault occurs, without the need to manually label a massive amount of node data, thereby reducing labor costs, saving time and effort, and improving detection efficiency. At the same time, the real-time nature of the data of each node is guaranteed, and the response speed of the microservice system is significantly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of microservice system fault detection, and in particular to an anomaly detection method and device for a microservice system. Background Art

[0002] Microservice system is a software architecture that treats a single application as a set of software service suites. Each service runs in an independent process and communicates with each other through lightweight protocols. With the rapid development of computer technology and network communication technology, microservice system architecture has become a widely used server architecture. Due to the complex and changeable service relationships in the microservice system architecture, thousands of server interface nodes often cause abnormalities during work due to the excessive burden of additional hardware and software caused by fine granularity. Therefore, it is necessary to perform abnormality detection on the microservice system architecture.

[0003] At present, the anomaly detection method of the microservice system in the prior art is mainly based on a single server, that is, by monitoring the business indicators of each server separately, specifically by long-term detection of whether the resource indicators and performance indicators of the business contained in each server exceed the preset threshold, and then determine the server interface node where the microservice system has an abnormality, and then use manual troubleshooting to check the abnormal node. On the one hand, this detection method is only for a single server, so it is difficult to know the relationship between the mutual call information between different servers, which makes it difficult to detect abnormal phenomena between different server nodes. On the other hand, the real-time performance of long-term detection of the business information of the microserver is also poor, and manual troubleshooting of abnormal nodes is also required, which is also time-consuming and labor-intensive, and ultimately leads to high costs. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides an anomaly detection method for a microservice system to solve the problem in the prior art that manual troubleshooting of abnormal nodes is time-consuming and labor-intensive, ultimately resulting in high costs and poor real-time performance.

[0005] According to a first aspect, an embodiment of the present invention provides an anomaly detection method for a microservice system, characterized in that it includes the following steps:

[0006] Build a node network diagram for the microservice architecture;

[0007] According to the node network diagram, a target node call link corresponding to the service call request instruction is obtained, wherein the nodes in the target node call link are arranged in sequence according to the node call order;

[0008] Acquire data features of each node in the target node call link, and compose a target node data feature vector from the data features of each node according to the node call sequence;

[0009] Constructing a node anomaly detection model, and inputting the target node data feature vector into the abnormal node detection model for training to obtain a detection result;

[0010] According to the detection result, the target node calling link where the abnormal failure occurs is confirmed.

[0011] In combination with the first aspect, in the first implementation of the first aspect, a plurality of the target node data feature vectors are sequentially stored in a node message queue with a preset length as a target node message queue to be detected simultaneously.

[0012] In combination with the first implementation of the first aspect, in the second implementation of the first aspect, the target node message queue is input into a plurality of sequentially arranged node anomaly detection models for training to obtain detection results, and the number of the plurality of node anomaly detection models is equal to the number of target node data feature vectors stored in the target node message queue.

[0013] In combination with the second implementation manner of the first aspect, in the third implementation manner of the first aspect, the step of constructing a node anomaly detection model and inputting the target node data feature vector into the abnormal node detection model for training to obtain a detection result includes:

[0014] The node anomaly detection model is constructed using the isolation forest algorithm, and the target node data feature vector or the target node message queue is input into the abnormal node monitoring model for training to obtain the detection result.

[0015] In combination with the first aspect or any implementation scheme of the first aspect, in a fourth implementation scheme of the first aspect, the data characteristics of each node include node service delay and / or data resource occupancy and / or network transmission speed.

[0016] According to a second aspect, an embodiment of the present invention provides an abnormality detection device for a microservice system, comprising:

[0017] The first building module is used to build a node network diagram of the microservice architecture;

[0018] A first acquisition module is used to acquire a target node call link corresponding to the service call request instruction according to the node network diagram, wherein the nodes in the target node call link are arranged in sequence according to the node call order;

[0019] A second acquisition module is used to acquire data features of each node in the target node call link, and compose a target node data feature vector from the data features of each node according to the node call sequence;

[0020] The second construction module is used to construct a node anomaly detection model, and input the target node data feature vector into the abnormal node detection model for training to obtain a detection result;

[0021] A confirmation module is used to confirm the target node call link where an abnormal failure occurs according to the detection result.

[0022] In combination with the second aspect, in the first implementation of the second aspect, the plurality of target node data feature vectors are sequentially stored in a node message queue with a preset length as a target node message queue to be detected simultaneously.

[0023] In combination with the first implementation of the second aspect, in the second implementation of the second aspect, the target node message queue is input into a plurality of sequentially arranged node anomaly detection models for training to obtain detection results, and the number of the plurality of node anomaly detection models is equal to the number of target node data feature vectors stored in the target node message queue.

[0024] In combination with the second implementation manner of the second aspect, in a third implementation manner of the second aspect, the second building block includes:

[0025] The construction submodule is used to construct the node anomaly detection model using the isolation forest algorithm, and input the target node data feature vector or the target node message queue into the abnormal node monitoring model for training to obtain the detection result.

[0026] According to a third aspect, an embodiment of the present invention provides a storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the anomaly detection method for a microservice system described in the first aspect or any embodiment of the first aspect.

[0027] According to the fourth aspect, an embodiment of the present invention provides a microservice system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the anomaly detection method for the microservice system described in the first aspect or any embodiment of the first aspect are implemented.

[0028] The technical solution of the embodiment of the present invention has the following advantages:

[0029] An embodiment of the present invention provides an abnormality detection method and device for a microservice system, wherein the method obtains a detection result by training a target node data feature vector corresponding to the data features of each node of a target node call link or a target node message queue storing the target node data feature vector based on a node abnormality detection model, and can quickly determine the target node call link where an abnormal failure occurs, without the need to manually label a large amount of node data, thereby reducing labor costs, saving time and effort, and improving detection efficiency. At the same time, the real-time nature of the data of each node is ensured, and the response speed of the microservice system is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 This is a flowchart of an abnormality detection method for a microservice system in an embodiment of the present invention;

[0032] Figure 2 A node network diagram of a microservice architecture in an embodiment of the present invention;

[0033] Figure 3 A schematic diagram of a call link trajectory corresponding to a service call request in an embodiment of the present invention is specified;

[0034] Figure 4 Schematic diagram of a target node data feature vector in an embodiment of the present invention;

[0035] Figure 5 A schematic diagram of a model training for a target node calling a link in an embodiment of the present invention;

[0036] Figure 6 This is a schematic diagram of a target node message queue in an embodiment of the present invention;

[0037] Figure 7 A schematic diagram of a model training of a target node message queue in an embodiment of the present invention;

[0038] Figure 8 This is a structural block diagram of an abnormality detection device for a microservice system in an embodiment of the present invention;

[0039] Fig. 9 Schematic diagram of the hardware structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0041] Example 1

[0042] The embodiment of the present invention provides a microservice system abnormality detection method for detecting abnormal failures of each microservice node in the microservice system. Figure 1 As shown, the steps include:

[0043] Step S1: Construct a node network diagram of the microservice architecture. The node network diagram here can be formed into a network node diagram by embedding points or installing agent programs during the operation of the microservice system through the Instramentation mechanism, such as Figure 2 The figure shows a schematic diagram of the node network of the microservice architecture, which is distributed in a star-shaped pattern to form a network structure diagram. Figure 2 In the node network diagram, each node can be a component of the microservice system, which can be a machine or a technical component (message middleware, network server) or a database. Each network node in the node network diagram stores a microservice log, which is equivalent to a data tag of each network node, and the data information of each network node is stored on the data tag.

[0044] Step S2: According to the node network diagram, obtain the target node call link corresponding to the business call request instruction, and the nodes in the target node call link are arranged in sequence according to the node call order. The business call request instruction here is the user instruction to be operated by the user. For example, when a user purchases goods online on the Taobao client, the user's shopping on the Taobao client can be a business call request instruction. The user logs in to the Taobao client in advance - determines the target product - pays online. The user's operation sequence is the target node call link when the user uses Taobao shopping. The target node call link is the node call link to be detected. For example: the business call request instruction is M1, and its corresponding target call link can be ACD; the business call request instruction is M2, and its corresponding target call link can be ABE. The nodes in the target node call link here are paths formed in sequence. As Figure 3 As shown in the figure, different services correspond to different entry call requests, and the target node call links corresponding to two different call requests are:

[0045] Business call request instruction 1: A->C->F;

[0046] Business call request instruction 2: D->F->E;

[0047] The target node call link can also be regarded as the user operation running track corresponding to the business call request instruction, and the running track can be regarded as the ID number of the microservice node.

[0048] The data characteristics of each node mentioned above include node service delay and / or data resource occupancy and / or network transmission speed. The data characteristics of each node may also include other types of data characteristics, but are not limited thereto.

[0049] Step S3: Obtain the data features of each node in the target node call link, and combine the data features of each node into a target node data feature vector according to the node call sequence.

[0050] For example, taking service latency as an example, the target node calls the link through three microservice nodes A, C, and F. Figure 4 As shown in the figure, the spans of microservice nodes A, C, and F on the time axis are Span1, Span2, and Span3 respectively. In these three time spans, the data feature vectors of microservice nodes A, C, and F in the same call are extracted. The extractable data features generally include node service delay and / or data resource occupancy and / or network transmission speed, such as CPU percentage, memory occupancy, network IO byte number, etc. The extracted data features are concatenated into a feature vector according to the call order. Taking the service delay feature as an example, the final trajectory feature vector is expressed as (Span1, Span2, Span3). In order to distinguish the independent occupancy of each node for time, the trajectory feature vector of the target node call link can also be expressed as (Span1–Span2, Span2-Span3, Span3).

[0051] Step S4: construct a node anomaly detection model, and input the target node data feature vector into the abnormal node detection model for training to obtain the detection result.

[0052] In a specific embodiment, the above step S4 may specifically include the following steps during execution:

[0053] The node anomaly detection model is constructed using the isolation forest algorithm, and the target node data feature vector is input into the abnormal node monitoring model for training to obtain the detection result.

[0054] like Figure 5 As shown, it is a structural block diagram of the target node data feature vector for training the node anomaly detection model.

[0055] exist Figure 5Among them, the number of eigenvectors of the target node data is the same as the number of training models, that is, there are as many training models as there are multiple trajectory operation vectors. The training model is a node anomaly detection model constructed based on the isolation forest algorithm.

[0056] The above isolation forest algorithm is an unsupervised anomaly detection algorithm applicable to continuous data. This algorithm defines the isolation tree and the path length h(x) of the sample point x in the isolation tree.

[0057] For the isolation tree: If T is a node of the isolation tree, T has two cases: an external node without child nodes, and an internal node with two child nodes (Tl, Tr) and a test. The test in T consists of an attribute q and a split point p. Points with q < p belong to Tl, and vice versa belong to Tr. The path length h(x) of the sample point x in the isolation tree: The number of edges that the sample point x passes through from the root node of the iTree to the leaf node.

[0058] The isolation forest algorithm can be roughly divided into two stages. In the first stage, t isolation trees need to be trained to form an isolation forest. Subsequently, each sample point is brought into each isolation tree in the forest, the average height is calculated, and then the anomaly score of each sample point is calculated.

[0059] The first stage:

[0060] Step S111:

[0061] X = {x1, …… x n}, is a given data set, Randomly draw ψ sample points from X to form a subset X ′ and put them into the root node.

[0062] Step S112: Randomly specify a dimension q from d, and randomly generate a cut point p in the current data,

[0063] min(x ij , j = q, x ij ∈X ′ ) < p < max(x ij , j = q, x ij ∈X ′ ).

[0064] Step S113: This cut point p generates a hyperplane that divides the current data space into two subspaces: Sample points with the specified dimension less than p are put into the left child node, and those greater than or equal to p are put into the right child node.

[0065] Step S114: Recursively execute Step S112 and Step S113 until all leaf nodes have only one sample point or the isolation tree (iTree) has reached the specified height.

[0066] Step S115: Loop step S112 to step S114 until t isolated trees (iTree) are generated.

[0067] Phase 2:

[0068] Step S211: For each data point x i , let it traverse each isolated tree (iTree) and calculate the point x i The average height in the forest h(x i ), normalize the average height of all points. The calculation formula of the anomaly score is as follows:

[0069] in,

[0070]

[0071] A node anomaly detection model is constructed by using the isolation forest algorithm, and the target node data feature vector input into the model is trained to obtain the target node data features where abnormal failures occur.

[0072] Of course, the embodiment of the present invention may also utilize other unsupervised detection algorithms to detect whether an abnormal fault occurs in the target node data feature vector.

[0073] Step S5: According to the detection result, confirm that the target node call link has an abnormal fault. The detection result is that the value between [0,1] is considered normal data, and the value greater than 1 is considered abnormal data. Therefore, according to the detection result, it can be confirmed that the target node call link has an abnormal fault.

[0074] The anomaly detection method for the microservice system in the embodiment of the present invention obtains the detection result by training the target node data feature vector corresponding to the data features of each node of the target node call link based on the node anomaly detection model, and can quickly determine the target node call link where the abnormal failure occurs. There is no need to manually label the massive amount of node data, which reduces labor costs, saves time and effort, improves detection efficiency, and at the same time, ensures the real-time nature of the data of each node.

[0075] Example 2

[0076] An embodiment of the present invention provides an anomaly detection method for a microservice system. In order to improve the response speed and real-time requirements of anomaly detection in the microservice system, as a preferred implementation, multiple target node data feature vectors can be sequentially stored in a node message queue with a preset team leader as a target node message queue to be detected at the same time.

[0077] like Figure 6As shown in the figure, it is the target node message queue with a preset leader. The length of the preset leader can be set according to different needs. Figure 6 In the target node message queue of the preset captain, N target node data feature vectors can be stored, and the N target node data feature vectors are stored in sequence. The target node message queue here is equivalent to a data storage container. Multiple target node data features are stored in sequence in the node message queue of the preset captain as the target node message queue to be detected at the same time, so as to ensure that multiple target node data features are detected at the same time, thereby improving the response speed of abnormal detection of the microservice system.

[0078] During the execution of the above step S4 in Embodiment 1, the following steps may be specifically included:

[0079] The node anomaly detection model is constructed using the isolation forest algorithm, and the target node message queue is input into the abnormal node monitoring model training to obtain the detection result. The target node message queue here can be multiple queues, and the target node message queue stores multiple target node data features. The target node message queue is detected simultaneously in the node anomaly detection model to improve the response speed of the microservice system detection. The process of constructing the node anomaly detection model using the isolation forest algorithm is the same as above, and will not be repeated here.

[0080] Specifically, Figure 7 As shown in the figure, in order to facilitate the horizontal expansion of anomaly detection of trajectory operation vectors, multiple node anomaly detection models can be loaded into one process, and multiple identical detection processes can be deployed in parallel. The producer process collects and splices trajectory operation feature vectors, and publishes the trajectory operation feature vector data to the target node message queue. The anomaly detection process, as a consumer, obtains trajectory feature data and performs anomaly detection alone. After anomalies are found, they are reported to the anomaly analysis, alarm and other modules through the message queue for subsequent processing.

[0081] exist Figure 7 In the example, the number of multiple node anomaly detection models is equal to the number of target node data features stored in the target node message queue.

[0082] The anomaly detection method for a microservice system in an embodiment of the present invention obtains a detection result by training a target node message queue storing multiple target node data feature vectors based on a node anomaly detection model, and can quickly determine the target node message queue where an abnormal failure occurs. There is no need to manually label a large amount of node data, which reduces labor costs, saves time and effort, and improves detection efficiency. At the same time, it also ensures the real-time nature of the data of each node, and significantly improves the response speed of anomaly detection in the microservice system.

[0083] Example 3

[0084] The embodiment of the present invention provides an abnormality detection device for a microservice system, such as Figure 8 As shown, including:

[0085] The first construction module 81 is used to construct a node network diagram of the microservice architecture.

[0086] The first acquisition module 82 is used to acquire a target node call link corresponding to the service call request instruction according to the node network diagram, and each node in the target node call link is arranged in sequence according to the node call order.

[0087] The second acquisition module 83 is used to acquire the data features of each node in the target node call link, and compose the target node data feature vector from the data features of each node according to the node call sequence.

[0088] The second construction module 84 is used to construct a node anomaly detection model, and input the target node data feature vector into the abnormal node detection model for training to obtain a detection result.

[0089] The confirmation module 85 is used to confirm the abnormal fault target node call link according to the detection result.

[0090] In the anomaly detection device of the microservice system in the embodiment of the present invention, multiple target node data features are sequentially stored in a node message queue with a preset captain as a target node message queue to be detected at the same time.

[0091] The anomaly detection device of the microservice system in the embodiment of the present invention inputs the target node message queue into multiple sequentially arranged node anomaly detection model training to obtain the detection result, and the number of the multiple node anomaly detection models is equal to the number of target node data features stored in the target node message queue.

[0092] In the abnormality detection device of the microservice system in the embodiment of the present invention, the second building module 84 includes:

[0093] The construction submodule is used to build a node anomaly detection model using the isolation forest algorithm, and input the target node data features or the target node message queue into the abnormal node monitoring model training to obtain the detection results.

[0094] In the anomaly detection device of the microservice system in the embodiment of the present invention, the data characteristics of each node include node service delay and / or data resource occupancy and / or network transmission speed.

[0095] The anomaly detection device of the microservice system in the embodiment of the present invention can quickly determine the target node call link where the abnormal failure occurs by training the target node data feature vector corresponding to the data features of each node of the target node call link or the target node message queue storing the target node data feature vector based on the node anomaly detection model to obtain the detection result. There is no need to manually label the massive amount of node data, which reduces labor costs, saves time and effort, and improves detection efficiency. At the same time, it also ensures the real-time nature of the data of each node, and significantly improves the response speed of the microservice system.

[0096] Example 4

[0097] The embodiment of the present invention provides a storage medium on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method in embodiment 1 or embodiment 2 are implemented. The storage medium also stores a node network diagram, a target node call link, a service call request, a target node data feature vector, a detection result, etc. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0098] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0099] Example 5

[0100] An embodiment of the present invention provides a computer device, such as Fig. 9 As shown, the computer device includes a memory 920, a processor 910, and a computer program stored in the memory 920 and executable on the processor 910. When the processor 910 executes the program, the steps of the method in Example 1 or Example 2 are implemented.

[0101] Fig. 9 is a schematic diagram of the hardware structure of a computer device for executing a list item operation processing method provided by an embodiment of the present invention, such as Fig. 9 As shown, the computer device includes one or more processors 910 and a memory 920. Fig. 9A processor 910 is taken as an example.

[0102] The computer device for executing the method for processing list item operations may further include: an input device 930 and an output device 940 .

[0103] The processor 910, the memory 920, the input device 930 and the output device 940 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0104] The processor 910 may be a central processing unit (CPU). The processor 910 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0105] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A method for detecting anomalies in a microservice system, characterized in that: The steps include: Build a node network diagram for the microservice architecture; According to the node network diagram, a target node call link corresponding to the service call request instruction is obtained, wherein the nodes in the target node call link are arranged in sequence according to the node call order; Acquire data features of each node in the target node call link, and compose a target node data feature vector from the data features of each node according to the node call sequence; Constructing a node anomaly detection model, and inputting the target node data feature vector into the node anomaly detection model for training to obtain a detection result; According to the detection result, confirm that the target node where the abnormal failure occurs calls the link; The plurality of target node data feature vectors are sequentially stored in a node message queue of a preset leader as a target node message queue to be detected simultaneously; Inputting the target node message queue into a plurality of sequentially arranged node anomaly detection models for training to obtain a detection result, wherein the number of the plurality of node anomaly detection models is equal to the number of the target node data feature vectors stored in the target node message queue; Loading a plurality of the node anomaly detection models into one process, and deploying a plurality of identical detection processes in parallel.

2. The method for detecting anomalies in a microservice system according to claim 1, characterized in that: The step of constructing a node anomaly detection model and inputting the target node data feature vector into the node anomaly detection model for training to obtain a detection result comprises: The node anomaly detection model is constructed using the isolation forest algorithm, and the target node data feature vector or the target node message queue is input into the node anomaly detection model for training to obtain a detection result.

3. The method for detecting anomalies in a microservice system according to any one of claims 1 to 2, characterized in that: The data characteristics of each node include node service delay and / or data resource occupancy and / or network transmission speed.

4. An abnormality detection device for a microservice system, comprising: The first building module is used to build a node network diagram of the microservice architecture; A first acquisition module is used to acquire a target node call link corresponding to the service call request instruction according to the node network diagram, wherein the nodes in the target node call link are arranged in sequence according to the node call order; A second acquisition module is used to acquire data features of each node in the target node call link, and compose a target node data feature vector from the data features of each node according to the node call sequence; The second construction module is used to construct a node anomaly detection model, and input the target node data feature vector into the node anomaly detection model for training to obtain a detection result; A confirmation module, used to confirm, according to the detection result, that the target node calling link has an abnormal failure; The plurality of target node data feature vectors are sequentially stored in a node message queue of a preset leader as a target node message queue to be detected simultaneously; Inputting the target node message queue into a plurality of sequentially arranged node anomaly detection models for training to obtain a detection result, wherein the number of the plurality of node anomaly detection models is equal to the number of the target node data feature vectors stored in the target node message queue; Loading a plurality of the node anomaly detection models into one process, and deploying a plurality of identical detection processes in parallel.

5. The abnormality detection device for a microservice system according to claim 4, characterized in that: The second building block comprises: The construction submodule is used to construct the node anomaly detection model using the isolation forest algorithm, and input the target node data feature vector or the target node message queue into the node anomaly detection model for training to obtain the detection result.

6. A storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by the processor, the steps of the anomaly detection method for the microservice system described in any one of claims 1 to 3 are implemented.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the anomaly detection method for the microservice system described in any one of claims 1 to 3 are implemented.

Images