A fault prediction method and terminal of a microservice

Abstract

The application discloses a kind of microservice fault prediction method and terminal, collect the history fault of each instance on microservice, according to the classification type and historical running information of history fault constructs fault type library. According to the cycle time, the real-time running information of each instance on microservice is collected in real time, if real-time running information and historical running information in fault type library match successfully, then the fault type corresponding to real-time running information is obtained. In this way, possible fault conditions can be predicted before the real fault occurs, facilitating subsequent execution of related remedial action, and the probability of system failure can also be reduced through prediction, improving user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fault detection, and in particular to a micro-service fault prediction method and a terminal. BACKGROUND

[0002] In the current Internet system, in order to ensure the high concurrency and high throughput function of the server system, a distributed deployment method is usually used, that is, multiple nodes of the same system are deployed, and the functions of the entire server system are completed by the nodes on multiple different machines.

[0003] However, under the distributed deployment method, there are many instances of each system or component, and each instance is usually deployed on a different machine, which aims to reduce the risk of downtime. However, under this method, the increase in the number of instances increases the risk of failure of each instance. Therefore, under each micro-service architecture, in order to monitor numerous micro-service instances, a distributed monitoring system is essential. However, the monitoring system gives an alarm prompt message when a fault occurs, which has already caused a fault and brought a bad experience to the user. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a micro-service fault prediction method and terminal, which can reduce the probability of micro-service failure and improve user experience.

[0005] In order to solve the above technical problems, the technical scheme adopted by the present application is:

[0006] A micro-service fault prediction method, comprising the steps of:

[0007] Collecting historical running information of each instance on the micro-service within a period of time before each historical fault, and classifying the historical faults, obtaining a fault type library according to different types of historical faults and their historical running information, wherein the historical running information includes log information and distributed link information;

[0008] Real-time collecting real-time running information of each instance on the micro-service according to the period of time, and if the real-time running information matches the historical running information in the fault type library successfully, obtaining the fault type corresponding to the real-time running information.

[0009] In order to solve the above technical problems, another technical scheme adopted by the present application is:

[0010] A micro-service fault prediction terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements each step of the above micro-service fault prediction method when executing the computer program.

[0011] The application has the beneficial effect that the historical faults of each instance on the microservice are collected, a fault type library is constructed according to the classified types of the historical faults and the historical running information, the real-time running information of each instance on the microservice is collected in real time according to a period of time, and if the real-time running information matches the historical running information in the fault type library successfully, the fault type corresponding to the real-time running information is obtained. In this way, the possible fault condition can be predicted before the real fault occurs, the subsequent related remedial action can be facilitated, and the probability of system failure can be reduced through the prediction manner, and the user experience is improved. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 A flowchart of a microservice fault prediction method according to an embodiment of the application;

[0013] Figure 2 A schematic diagram of a microservice fault prediction terminal according to an embodiment of the application;

[0014] Label explanation:

[0015] 1. A microservice fault prediction terminal; 2. A memory; 3. A processor. DETAILED DESCRIPTION

[0016] To explain the technical content, the achieved purposes and effects of the application in detail, the following will be described in combination with the embodiments and the accompanying drawings.

[0017] Please refer to Figure 1 The embodiment of the application provides a microservice fault prediction method, which comprises the following steps:

[0018] The historical running information of each instance on the microservice within a period of time before each historical fault is collected, the historical faults are classified, a fault type library is obtained according to different types of historical faults and the historical running information thereof, and the historical running information includes log information and distributed link information;

[0019] The real-time running information of each instance on the microservice is collected in real time according to a period of time, and if the real-time running information matches the historical running information in the fault type library successfully, the fault type corresponding to the real-time running information is obtained.

[0020] From the above description, the beneficial effects of the present application are that the historical faults of each instance on the micro service are collected, and a fault type library is constructed according to the classified types of the historical faults and historical running information. Real-time running information of each instance on the micro service is collected in a period of time, and if the real-time running information matches the historical running information in the fault type library successfully, the fault type corresponding to the real-time running information is obtained. In this way, the possible fault situation can be predicted before the real fault occurs, which facilitates subsequent execution of related remedial actions, and can also reduce the probability of system failure through prediction, thereby improving user experience.

[0021] Further, the historical faults are classified, including:

[0022] If the garbage collection time interval of a single instance in the historical running information corresponding to the historical fault is less than a first time length and the interface delay time of the single instance is greater than a second time length, it is determined that the historical fault is a slow response fault type.

[0023] If the memory occupation and thread occupation of a single instance under a preset time length in the historical running information corresponding to the historical fault both reach a first proportion, it is determined that the historical fault is a memory fault type.

[0024] From the above description, by classifying faults in a preset fault type manner, the more common fault types can be quickly divided.

[0025] Further, the historical faults are classified, and further including:

[0026] For each historical fault, an overall historical trajectory graph of the garbage collection time interval, the interface delay time, the memory occupation, and the thread occupation of a single instance in the historical running information corresponding to the historical fault is drawn, and clustering analysis is performed on the historical trajectory graphs of all historical faults to obtain the classification of the historical faults.

[0027] From the above description, by establishing a historical trajectory graph based on the historical running information of the historical faults and classifying the types of the historical faults by trajectory graph clustering, the historical faults can be objectively and reasonably classified.

[0028] Further, the historical faults are classified, and further including:

[0029] A preset repair method corresponding to the historical fault is configured.

[0030] After obtaining the fault type corresponding to the real-time running information, further including:

[0031] A preset repair method is found according to the fault type of the real-time running information to repair the fault.

[0032] From the above description, the corresponding preset repair mode is equipped for the historical fault, so that the subsequent related remedial repair action is facilitated, and the probability of system failure is reduced.

[0033] Further, if the real-time running information does not match the historical running information in the fault type library, a real-time trajectory graph corresponding to the real-time running information is generated, a historical trajectory graph with the highest similarity to the real-time trajectory graph is searched, and a preset repair mode of a historical fault corresponding to the historical trajectory graph is used for fault repair.

[0034] From the above description, if no fault type is matched and the real-time running information exceeds the preset running standard, that is, there is an anomaly but it is not detected, a solution to a certain type of anomaly close to the data trajectory graph can be tried to handle, and if the handling is successful, the new anomaly is bound to the solution, and a relatively feasible repair scheme can be provided for the new fault in advance.

[0035] Please refer to Figure 2 Another embodiment of the present application provides a microservice fault prediction terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements each step of the above-mentioned microservice fault prediction method when executing the computer program.

[0036] The above-mentioned microservice fault prediction method and terminal of the present application are suitable for performing related remedial repair actions before the microservice actually fails, reducing the probability of system failure, and improving user experience, which will be described in detail in the following specific embodiments.

[0037] Embodiment one

[0038] Please refer to Figure 1 A microservice fault prediction method comprises the following steps:

[0039] S1, collect historical running information of each instance on the microservice within a period of time before each historical fault, and classify the historical faults, obtain a fault type library according to different types of historical faults and historical running information, and the historical running information includes log information and distributed link information.

[0040] In this embodiment, there is a microservice component, which has 100 instances, and each instance exists on a separate machine. Through the fault prediction method of this embodiment, related information on each instance is collected, including logs, distributed link information and other various monitoring data, and the possibility of instance failure is analyzed according to these information.

[0041] Specifically, step S1 comprises the following steps:

[0042] S11, set a cycle time as 1 hour, collect the historical running information of each instance on the micro service in each historical failure before a cycle time as a failure template.

[0043] S12, classify the historical failures.

[0044] In some embodiments, the failures can be classified manually:

[0045] S121, if the garbage collection time interval of a single instance in the historical running information corresponding to the historical failure is less than a first time length and the interface delay time of the single instance is greater than a second time length, it is determined that the historical failure is a slow response failure type.

[0046] In this embodiment, the slow response failure type mainly manifests that the garbage collection time interval of a single instance is less than 10 minutes, and the normal time interval is generally half an hour or more; the interface delay of a single instance increases by more than 1 second, and the normal query interface response time is less than 1 second.

[0047] S122, if the memory occupation and thread occupation of a single instance under a preset time length in the historical running information corresponding to the historical failure both reach a first proportion, it is determined that the historical failure is a memory failure type.

[0048] In this embodiment, the memory OOM failure mainly manifests that the memory of a single instance is high and long time reaches more than 80%, and the normal time period is below 80%; the thread number proportion of a single instance reaches more than 80% for a long time, and the normal time period is below 80%.

[0049] In some embodiments, the failures can be automatically classified by a program:

[0050] S123, for each historical failure, draw the overall historical trajectory graph of the garbage collection time interval, interface delay time, memory occupation, and thread occupation of a single instance in the historical running information corresponding to the historical failure, and perform clustering analysis on the historical trajectory graphs of all historical failures to obtain the classification of the historical failures.

[0051] Specifically, each type of failure has consistent overall data (including garbage collection time interval, interface delay time, memory occupation, and thread occupation) trajectory graph or a certain range of difference, and is classified as one type of abnormal data, and the unique identification code and name of the failure type are automatically set.

[0052] In some embodiments, the overall data of the failure can also include manually input data, that is, not only the data of the system itself is used for trajectory graph drawing, but also the data of other systems, so as to improve the accuracy of the failure type classification.

[0053] Further, for each clustered feature data, a certain range value is set, that is, the data of the features within the certain range value conforms to the cluster, so that more fault data ranges can be compatible.

[0054] S13, obtaining a fault type library according to different types of historical faults and historical running information thereof.

[0055] S14, configuring a corresponding preset repair mode for the historical fault.

[0056] When the slow response fault type is met, it may be a local fault or a resource limitation problem, and the preset repair mode is specifically: preferentially performing flow cutting processing, cutting the flow of the local machine to other instances, and deploying a new instance on another machine, the instance is consistent with the instance configuration of the local machine, and finally the newly deployed instance is connected to the cluster to restore the flow.

[0057] When the memory OOM fault type is met, it may be a local instance memory shortage, and the preset repair mode is specifically: preferentially performing flow cutting processing, cutting the flow of the local machine to other instances. Expand the memory of the local instance, and restart to cut back the flow to the local machine.

[0058] For the clustered fault type, a corresponding preset repair mode is manually configured.

[0059] S2, collecting real-time running information of each instance on the microservice in real time according to a period of time, if the real-time running information matches the historical running information in the fault type library successfully, a fault type corresponding to the real-time running information is obtained.

[0060] Specifically, if the real-time running information matches the historical running information in the fault type library successfully, a fault type corresponding to the real-time running information is obtained, and a preset repair mode is searched for the fault type of the real-time running information to repair the fault.

[0061] If the real-time running information does not match the historical running information in the fault type library successfully, and the real-time running information exceeds the preset running standard, a real-time trajectory graph corresponding to the real-time running information is generated, a historical trajectory graph with the highest similarity to the real-time trajectory graph is searched, and a preset repair mode of the historical fault corresponding to the historical trajectory graph is used to repair the fault.

[0062] In the embodiment, if no fault type is matched and the real-time running information exceeds the preset running standard, that is, there is an exception but it is not detected, a solution of a certain type of exception close to the data trajectory graph of the exception can be tried to find and process preferentially, if the processing is successful, the new exception is bound to the solution, otherwise, manual intervention is needed for manual repair, after the repair is completed, the exception is automatically bound to the solution.

[0063] Embodiment two

[0064] Please refer to Figure 2 A microservice fault prediction terminal 1, comprising a memory 2, a processor 3, and a computer program stored in the memory 2 and executable on the processor 3, wherein the processor 3 implements each step of the microservice fault prediction method of embodiment one when executing the computer program.

[0065] In summary, the microservice fault prediction method and terminal provided by the present application collect historical faults of each instance on the microservice, construct a fault type library according to the classification type and historical running information of the historical faults, collect real-time running information of each instance on the microservice in real time according to a cycle time, if the real-time running information matches the historical running information in the fault type library successfully, a fault type corresponding to the real-time running information is obtained. In this way, possible fault conditions can be predicted before a real fault occurs, which facilitates subsequent execution of related remedial actions, and the probability of system failure can be reduced through prediction, thereby improving user experience.

[0066] The above description is only an embodiment of the present application, and does not limit the patent range of the present application, any equivalent transformation or direct or indirect application in related technical fields based on the content of the specification and drawings of the present application are also included in the patent protection range of the present application.

Claims

1. A method for failure prediction of microservices, characterized in that, The method comprises the steps of: collecting historical running information of each instance on the microservice within a period of time before each historical fault, classifying the historical faults, and obtaining a fault type library according to different types of historical faults and the historical running information, wherein the historical running information comprises log information and distributed link information; collecting real-time running information of each instance on the microservice in real time according to a period of time, and obtaining a fault type corresponding to the real-time running information if the real-time running information matches the historical running information in the fault type library successfully; the classifying the historical faults further comprises: for each historical fault, drawing an overall historical trajectory graph of a garbage collection time interval, an interface delay time, a memory occupation condition and a thread occupation condition of a single instance in the historical running information corresponding to the historical fault, and performing clustering analysis on the historical trajectory graphs of all historical faults to obtain a classification of the historical faults; the classifying the historical faults further comprises: configuring a preset repair mode corresponding to the historical fault; the obtaining the fault type corresponding to the real-time running information further comprises: looking up the preset repair mode for the fault type of the real-time running information to repair the fault; if the real-time running information does not match the historical running information in the fault type library successfully, generating a real-time trajectory graph corresponding to the real-time running information, looking up a historical trajectory graph with the highest similarity to the real-time trajectory graph, and repairing the fault according to a preset repair mode of a historical fault corresponding to the historical trajectory graph. 2.The method of claim 1, wherein, the classifying the historical faults comprises: if a garbage collection time interval of a single instance in the historical running information corresponding to the historical fault is less than a first time length and an interface delay time of the single instance is greater than a second time length, determining that the historical fault is a slow response fault type; if a memory occupation and a thread occupation of a single instance under a preset time length in the historical running information corresponding to the historical fault both reach a first proportion, determining that the historical fault is a memory fault type. 3.A failure prediction terminal of microservices, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the following steps: collecting historical running information of each instance on the microservice within a period of time before each historical fault, classifying the historical faults, and obtaining a fault type library according to different types of historical faults and the historical running information, wherein the historical running information comprises log information and distributed link information; collecting real-time running information of each instance on the microservice in real time according to a period of time, and obtaining a fault type corresponding to the real-time running information if the real-time running information matches the historical running information in the fault type library successfully; the classifying the historical faults further comprises: for each historical fault, drawing an overall historical trajectory graph of a garbage collection time interval, an interface delay time, a memory occupation condition and a thread occupation condition of a single instance in the historical running information corresponding to the historical fault, and performing clustering analysis on the historical trajectory graphs of all historical faults to obtain a classification of the historical faults; the classifying the historical faults further comprises: configuring a preset repair mode corresponding to the historical fault; Obtaining a fault type corresponding to the real-time running information, and then including: Finding a preset repair method for the fault type of the real-time running information to repair the fault; If the real-time running information does not match the historical running information in the fault type library, a real-time trajectory graph corresponding to the real-time running information is generated, a historical trajectory graph with the highest similarity to the real-time trajectory graph is found, and a preset repair method of a historical fault corresponding to the historical trajectory graph is used to repair the fault.

4. The fault prediction terminal of a microservice according to claim 3, characterized in that, Classifying the historical faults, including: If a garbage collection time interval of a single instance in historical running information corresponding to the historical fault is less than a first time length and an interface delay time of the single instance is greater than a second time length, the historical fault is determined to be a slow response fault type; If memory occupation and thread occupation of a single instance under a preset time length in the historical running information corresponding to the historical fault both reach a first proportion, the historical fault is determined to be a memory fault type.

Images