Method for automatically analyzing transaction logs of a distributed computing system

a distributed computing system and transaction log technology, applied in the field of automatic analysis of transaction logs of distributed computing systems, can solve the problems of inability to fully analyse the transaction logs of users, in particular causal relations between, and is extremely difficult and even impossible in certain cases, so as to increase the precision of its prediction

Pending Publication Date: 2021-11-04
BULL SA
View PDF1 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Benefits of technology

[0034]Thus, the artificial neural network that uses the numeric vectors of a group of events to predict the next event, makes its prediction by taking account of the proximity in meaning between the words of the events of the group of events, which increases the precision of its prediction.

Problems solved by technology

However, fully analysing these transaction logs by a user, in particular identifying the events that have occurred and the causal relations between the different events identified, is extremely difficult and even impossible in certain cases, even for an experienced user.
A first problem encountered by a user is the difficulty in analysing a file of the size of a transaction log, which, in the case of large-scale distributed computing systems and in particular supercomputers such as HPC (for “High-Performance Computing”), can reach several hundred million lines.
A second problem encountered by a user is the difficulty in establishing causal links between several events each associated with several lines, sometimes separated between them by several lines associated with other events.
Given that each source sends messages independently of the other sources, the order of the messages in the transaction log is random, which makes it difficult for a user to determine causal relations between the different messages.
However, none of these analysers is capable of analysing hundreds of millions of lines in a reasonable amount of time, the fastest taking 48 hours, which is much too long in light of the filling speed of a transaction log.
In addition, the automatic analysers of the prior art used to establish causalities focus solely on one type of events, for example detecting HTTP traffic, and on transaction logs considered to be abnormal, and therefore do not conduct an analysis on all of the events that occurred in the transaction logs.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for automatically analyzing transaction logs of a distributed computing system
  • Method for automatically analyzing transaction logs of a distributed computing system
  • Method for automatically analyzing transaction logs of a distributed computing system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048]Unless mentioned otherwise, the same element appearing in different figures has a unique reference.

[0049]A first aspect of the invention relates to a method for automatically analysing at least one transaction log of a distributed computing system.

[0050]“Automatic analysis” means an analysis implemented by computer and conducted without any configuration of the user being required.

[0051]The method according to the invention is implemented by a computer comprising at least one processor and a memory.

[0052]The distributed computing system is for example a large-scale distributed computing system, such as for example a high-performance supercomputer also called HPC (for “High-Performance Computing”) supercomputer.

[0053]FIG. 2 shows a diagrammatical representation of a transaction log 200.

[0054]A transaction log 200 comprises a plurality of lines 201.

[0055]In FIG. 2, the transaction log 200 includes fourteen lines 201 divided into a first set 202_1 of lines 201 comprising eight li...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

An aspect of the invention relates to a method for automatically analysing a transaction log of a distributed computing system, comprising a plurality of lines, the method comprising the following steps:
    • For each line:
      • Cutting the line into words;
      • Constructing a comparison vector by comparing the line with the other lines of the same size as the line;
      • Constructing a pattern from the comparison vector;
    • Creating an event per pattern;
Constructing at least one prediction model by training an artificial neural network on a group of training events, the prediction model being configured to predict the next event in the transaction log;
For at least one event:
  • Using the prediction model to predict the event, from a group of prediction events;
  • Generating from the prediction model, a causal graph of the event comprising a causal relation for each event of the group of prediction events responding to a relevance condition.
  • Description

    BACKGROUND OF THE INVENTIONField of the Invention[0001]The technical field of the invention is that of distributed computing systems and in particular that of analysing transaction logs of distributed computing systems.[0002]The present invention relates to a method for analysing at least one transaction log of a distributed computing system and in particular a method for automatically analysing at least one transaction log of a distributed computing system.Description of the Related Art[0003]The transaction logs of distributed computing systems are very useful sources of information in many cases such as for example, discovering the root case of an error, assistance with installation or automatic “debugging”.[0004]However, fully analysing these transaction logs by a user, in particular identifying the events that have occurred and the causal relations between the different events identified, is extremely difficult and even impossible in certain cases, even for an experienced user.[...

    Claims

    the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
    Login to view more

    Application Information

    Patent Timeline
    no application Login to view more
    Patent Type & Authority Applications(United States)
    IPC IPC(8): G06N3/08G06N3/04G06F40/166G06F40/279
    CPCG06N3/088G06F40/279G06F40/166G06N3/0445G06F11/079G06F11/0709G06F11/3608G06F11/2263G06N3/044
    Inventor PLATINI, MARCPELLETIER, BENOÎTPAULETTO, LOÏC
    Owner BULL SA
    Who we serve
    • R&D Engineer
    • R&D Manager
    • IP Professional
    Why Eureka
    • Industry Leading Data Capabilities
    • Powerful AI technology
    • Patent DNA Extraction
    Social media
    Try Eureka
    PatSnap group products