Method for automatically analyzing transaction logs of a distributed computing system

Abstract

An aspect of the invention relates to a method for automatically analysing a transaction log of a distributed computing system, comprising a plurality of lines, the method comprising the following steps:For each line:Cutting the line into words;Constructing a comparison vector by comparing the line with the other lines of the same size as the line;Constructing a pattern from the comparison vector;Creating an event per pattern;Constructing at least one prediction model by training an artificial neural network on a group of training events, the prediction model being configured to predict the next event in the transaction log;For at least one event:Using the prediction model to predict the event, from a group of prediction events;Generating from the prediction model, a causal graph of the event comprising a causal relation for each event of the group of prediction events responding to a relevance condition.

Description

BACKGROUND OF THE INVENTIONField of the Invention[0001]The technical field of the invention is that of distributed computing systems and in particular that of analysing transaction logs of distributed computing systems.[0002]The present invention relates to a method for analysing at least one transaction log of a distributed computing system and in particular a method for automatically analysing at least one transaction log of a distributed computing system.Description of the Related Art[0003]The transaction logs of distributed computing systems are very useful sources of information in many cases such as for example, discovering the root case of an error, assistance with installation or automatic “debugging”.[0004]However, fully analysing these transaction logs by a user, in particular identifying the events that have occurred and the causal relations between the different events identified, is extremely difficult and even impossible in certain cases, even for an experienced user.[...

AI Technical Summary

Benefits of technology

The invention offers a way to analyze the logs of a distributed computing system without user intervention and in less than an hour. It automatically identifies events from the logs and generates a causal graph for each event, providing a view of the overall transaction. The method reduces computing time and memory resources by using descriptors that represent groups of events and a prediction model that takes into account the similarity of events. This increases the precision of predicting the next event. Overall, the invention improves the efficiency and accuracy of analyzing log data.

Problems solved by technology

However, fully analysing these transaction logs by a user, in particular identifying the events that have occurred and the causal relations between the different events identified, is extremely difficult and even impossible in certain cases, even for an experienced user.

A first problem encountered by a user is the difficulty in analysing a file of the size of a transaction log, which, in the case of large-scale distributed computing systems and in particular supercomputers such as HPC (for “High-Performance Computing”), can reach several hundred million lines.

A second problem encountered by a user is the difficulty in establishing causal links between several events each associated with several lines, sometimes separated between them by several lines associated with other events.

Given that each source sends messages independently of the other sources, the order of the messages in the transaction log is random, which makes it difficult for a user to determine causal relations between the different messages.

However, none of these analysers is capable of analysing hundreds of millions of lines in a reasonable amount of time, the fastest taking 48 hours, which is much too long in light of the filling speed of a transaction log.

In addition, the automatic analysers of the prior art used to establish causalities focus solely on one type of events, for example detecting HTTP traffic, and on transaction logs considered to be abnormal, and therefore do not conduct an analysis on all of the events that occurred in the transaction logs.

Images