Method for detecting anomalies in the data traffic of a communication system

Abstract

The invention relates to a method for detecting anomalies in communications implemented in a network, comprising performing a multi-scale analysis of observational data collected with N automatic communication classification modules Mi associated with a respective observation window size Ti, where Ti ≠ Tj for i,j = 1 to N and i ≠ j. An anomaly is determined to be present or absent at a given time based on the detection results of each module Mi associated with the respective observation window size Ti. Figure for the abstract: Fig. 3

Description

Title of the invention: Method for detecting anomalies in the data traffic of a communication system. Technical field

[0001] The invention lies in the field of cybersecurity and computer network monitoring. More specifically, it relates to anomaly detection techniques based on network traffic analysis. Previous technique

[0002] There are several approaches for detecting anomalies in communications implemented in a network.

[0003] Signature-based approaches use predefined rules to identify anomalies. They are effective at detecting known anomalies, but have difficulty detecting new anomalies and require frequent updates to the signatures.

[0004] Behavior-based approaches model the normal behavior of the network and detect deviations. They can detect new anomalies, but are susceptible to false positives if the normal behavior model is not sufficiently accurate.

[0005] Reinforcement learning approaches learn interactively by receiving rewards. They do not require labeled data, but may have difficulty converging towards an optimal policy in complex environments.

[0006] Time series analyses exploit the temporal dimension of traffic to detect anomalies. The choice of the analysis scale (time window) and noise filtering techniques are important challenges.

[0007] There is therefore a need to detect anomalies in network communications more effectively and quickly while avoiding false alarms and adapting to the diversity and evolution of anomalies, as well as the need to contextualize events for relevant decision-making. Summary of the invention

[0008] To this end, according to a first aspect, the present invention describes a method for detecting anomalies in communications implemented in a network, said method comprising the following steps implemented by an electronic device for monitoring communications in the network: - collect successive observation data on communications; - perform a multi-scale analysis of these collected observational data by exploiting said observational data collected on each of N processing channels VTR1 ... VTRN, i = 1 to N, N>2, each comprising an automatic communication classification module named Mi associated with a respective observation window size Ti, Ti Tj for ij, by implementing the following steps: for each of successive observation windows of size Ti, a respective vector of communication characteristics is determined as a function of those of the collected observation data selectively present in said observation window of size Ti;

[0009] the feature vector is provided as input to the automatic classification module Mi and the classification model Mi determines, based on said feature vector, a detection result relating to said observation window of size Ti, indicating whether the communications that occurred during said observation window of size Ti contain an anomaly; - determine whether an anomaly is present or not at a given time based on the detection results of each processing channel VTRi, i = 1 to N, each relative to an observation window of size Ti corresponding to said given time.

[0010] The invention extracts features over time windows of different sizes, thereby capturing the short- and long-term dynamics of anomalies. It improves the detection of various anomalies, leveraging information specific to each level of granularity, resulting in more robust and contextualized detection.

[0011] In embodiments, such a method will further comprise at least one of the following features:

[0012] - the following steps are implemented by the monitoring device in a preliminary phase:

[0013] collect successive observation data of communications;

[0014] considering a fixed size Ti, i = 1 to N:

[0015] determine communication characteristic vectors, each vector being determined based on observation data selectively from a respective observation window of communications, of size Ti,

[0016] perform machine learning of the classification module Mi on the basis of the determined communication characteristic vectors, so that it determines, based on any of the determined communication characteristic vectors, whether the traffic that took place during said observation window of size Ti contains an anomaly;

[0017] - an anomaly is determined to be present or not at a given time depending on the combination of said detection results;

[0018] - an anomaly is determined to be present or not at a given time depending on from the combination of said detection results, each weighted according to a respective weighting coefficient;

[0019] - an anomaly is determined to be present or not at a given time depending on the combination of said detection results, each weighted according to the relative size Ti of the respective observation window used for the determination of said result by the classification module Mi. an anomaly being determined to be present or not at a given time by comparison, with at least one threshold, of the combination of the N detection results, each weighted according to the relative size of the respective observation window used Ti for the determination of said result;

[0020] - given an observation window of size Ti, the average is calculated, on the observation window Ti, detection results for each classification module Mj on the observation windows of size Tj included within said observation window of size Ti,

[0021] and the sum of the following terms is determined:

[0022] the average calculated by each classification module Mj,

[0023] the detection result of the classification module Mi on the observation window of size Ti,

[0024] the detection result of each classification module Mk on the observation window of size Tk comprising said observation window of size Ti,

[0025] each of said terms being weighted by a respective weighting coefficient assigned to each classification module Mi, Mj, Mk.

[0026] According to another aspect, the invention describes an electronic device for monitoring communications in a network, said device being adapted to collect successive observation data of communications and to perform a multi-scale analysis of this collected observation data by exploiting said observation data collected on each of N processing channels VTR1 ... VTRN, i = 1 to N, N>2,

[0027] each processing channel VTRi, i = 1 to N, N>2, of the device comprising an automatic communication classification module named Mi associated with a respective size Ti of observation window, with Ti Tj for i,j = 1 to N and ij;

[0028] said device being adapted to determine, for each of successive observation windows of size Ti, a respective vector of communication characteristics as a function of those of the collected observation data present selectively in said observation window of size Ti;

[0029] said device being adapted to provide the feature vector as input to the automatic classification module Mi, the classification model Mi being adapted to determine, as a function of said feature vector, a detection result relating to said observation window of size Ti, indicating whether the communications occurring during said observation window of size Ti contain an anomaly;

[0030] said device being adapted to determine whether an anomaly is present or not at a given time according to the detection results of each processing channel VTRi, i = 1 to N, each relating to an observation window of size Ti corresponding to said given time.

[0031] In embodiments, such a device will further comprise at least one of the following features:

[0032] - the monitoring device is adapted to determine that an anomaly is present or not at a given moment depending on the combination of said detection results;

[0033] - the monitoring device is adapted to determine that an anomaly is present or not at a given moment depending on the combination of said detection results, each weighted according to a respective weighting coefficient.

[0034] According to another aspect, the invention describes a computer program product intended to be stored in the memory of a traffic monitoring device and further comprising a microcomputer, said computer program comprising instructions which, when executed on the microcomputer, implement the steps of a process as described above.

[0035] The invention also describes a computer-readable recording medium for storing such a computer program. Such recording media may include a storage means, such as a ROM, for example a CD-ROM or a microelectronic circuit ROM, or a magnetic recording means, for example a USB flash drive or a hard drive. Such recording media may be transmissible, such as an electrical or optical signal, which can be transmitted via an electrical or optical cable, by radio, or by other means, so that the computer program it contains is executable remotely. The programs according to the invention can, in particular, be downloaded onto a network, for example, the Internet.Such recording media may include an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the aforementioned display control method. Brief description of the drawings

[0036] The invention will be better understood and other features, details and advantages will become clearer from the following description, given by way of non-limiting reason, and from the accompanying figures, given by way of example.

[0037] [Fig-1] [Fig.1] illustrates the impact of time series size on detection anomalies;

[0038] [Fig.2] The [Fig.2] schematically represents a data communication system implementing a traffic monitoring solution in one embodiment of the invention;

[0039] [Fig.3] The [Fig.3] is a flowchart of the steps of a process for detecting anomalies in an embodiment of the invention;

[0040] [Fig.4] The [Fig.4] is a chronogram of steps of a method for detecting anomalies in an embodiment of the invention;

[0041] [Fig.5] Fig.5 represents the logs taken into account in a sample in an example embodiment of the invention.

[0042] Identical references may be used in different figures when they refer to identical or comparable elements. Description of the implementation methods

[0043] A communication system 1, represented in [Fig. 2], is considered as an example of an embodiment of the invention. This system comprises a set of user terminals 10 (mobile phones, computers, etc.) adapted to communicate with, for example, the servers 30 of an online hotel booking system, offering services such as checking availability, booking rooms, and canceling / modifying reservations. The network communications considered include, for example, requests transmitted from the user terminals to the resources (servers), as well as responses to these requests. These communications are transmitted over communication networks comprising transport links that can be of various types: wired, wireless (Radio Frequency), or other.In the example considered, the exchanged data passes through a gateway module 20, for example of the API gateway type (acronym for 'Application Programming Interface', i.e. Application Programming Interface) which analyzes the data traffic, deduces traffic observation data and transmits it to the traffic monitoring device 40.

[0044] In the case considered, the monitoring device 40 is an electronic device comprising in particular an electronic control module 44 and an electronic learning module 50.

[0045] This observation data, which describes the exchanged traffic data, includes, for example, in the case under consideration, the logs recording the requests between users and resources. In some embodiments, the observation data instead includes IP packets circulating on the network, or any other type of data describing communications.

[0046] Figure 3 illustrates the steps of a method for detecting anomalies in communications implemented, for example, in communication system 1 in one embodiment of the invention. This method comprises two phases: a learning phase 100 and a production phase 200, detailed below.

[0047] Learning phase

[0048] The learning phase 100 includes a step 101 of collecting data describing the communications by the control module 44 by the gateway 20, for example in real time from the network or extracted from a recorded history database.

[0049] In the case considered where this data is of the log type, the specific information varies according to the logging method used, but certain essential fields are generally present, including the following fields: - "Timestamp": timestamp of the request; - "http method": type of request (GET, POST, etc.); - "HTTP status code": server response (200, 404, etc.); - "Request duration": time taken by the server to complete the request ; - "Client IP address": IP address identifying the client, i.e., the terminal user 10; - “User-agent”: information about the browser or tool used by the customer ; - “Geolocation”: geographical location of the customer; - “Client ID”: unique identifier for each client; - “Request ID”: request identifier; - URL of the requested resource.

[0050] Examples of logs and the contents of these fields are shown in the following table.

[0051] [Tables 1] Timestamp Method of HT TP Status code HT TP Request duration Client IP User-agent Geolocation Client tlD URL (E ndpoint) Request ID 1622146 800 GET 200 250m s 192.168. 1.10 Chrome / 90.0 Paris, FR userl 23 / api / ress ourcel req_abcdefl23 456 1622146 805 POST 201 400m s 192.168. 1.11 Firefox / 88.0 Lyon, FR user4 56 Api / ress ource2 req_lmnop345 678

[0052] In the case of using IP packets, the usable header fields include: - Source and destination IP addresses - Source and destination port numbers Protocol (TCP, UDP, ICMP...) - Package size Flags (SYN, ACK, FIN...) - TTL (Time-to-Live).

[0053] Other information can be extracted from the payload (payload data, i.e., the data to be transmitted, not header, control, or metadata data) if it is unencrypted. Furthermore, in one embodiment, the capture time is recorded at the time of packet capture, which is particularly useful when the packet header does not contain a timestamp field.

[0054] Still with reference to [Fig.3], in an optional step 102 implemented by the control module 44, this observation data is pre-processed and enriched with business expertise information, for example, by adding fields such as: - function of the resource 30 (for example in the case of the service considered here: search, reservation...) - role of the user of the user terminal 10 (here for example individual, agency...) - HTTP method family (read, write...) - family of status codes (success, client error, server error...)

[0055] Examples of enriched logs are shown in the following table.

[0056] [Tables2] Time stamp P Method HTTP Status Code HTTP Request Duration Client IP User-agent Geolocation Client ID Request ID Endpoint URL Endpoint Function User Role Family Method Family Status 1622 GE 20 250 192. Chro Paris, use req_ab https: / / plmlatex Rech indiv Lire Suc 1468 00 T 0 ms 168. 1.10 me / 9 0.0 FR rl2 3 cdefl2 3456 .math.cnrs.fr / 15 44293832yzfvb vfyrynr search for hotels ideal cès 1622 PO 20 400 192. Firef Lyon, use req_gh / api / bookings Ré se trave Écr Suc 1468 05 ST 1 ms 168. 1.11 ox / 8 8.0 FR r45 6 ijk789 012 rvati ons lAge ncy ire these Next, in step 103, these possibly enriched observational data are exploited to extract feature vectors that will be used as inputs for machine learning algorithms.

[0057]

[0058]

[0059]

[0060]

[0061]

[0062] In one embodiment, in a substep 103_l, observation samples are first generated. To achieve this, observational data relating to the same time interval of duration Te are grouped together to generate a sample of observations. The frequency Fe = 1 / Te governing this grouping, which is called the sampling frequency below, is chosen according to one or more criteria, for example: - average frequency of requests; - detection requirements: a sufficient frequency to quickly detect anomalies; - storage and processing constraints, for a balance between granularity and costs. Each sample is then described by statistical characteristics calculated by the control module 44 from the observation data grouped over the time interval Te to generate this sample, such as: - Frequency metric(s) and ratio: absolute frequency, relative frequency, ratios... - diversity metric(s): distinct values, entropy, Gini indices / Simpson... - Metric(s) of central tendency and dispersion: quartiles, interquartile range, truncated mean... - distribution shape metric(s): asymmetry, kurtosis... The definitions and uses of such metrics are well known. However, not all of these metrics apply to all fields, depending on whether they are categorical or continuous. As an example, the following statistical characteristics are calculated for a sample of logs over Te=10 seconds, determined for the calculation time t (therefore calculated selectively on logs whose timestamp is in the range t and t+Te): - number of requests (in the range t and t+Te) - proportion of each HTTP method - quartiles of request durations - entropy of status codes - coefficient of skewness of the distribution of query sizes.

[0063] Considering IP packets, for example, the following statistical characteristics are calculated for a sample of packets over Te=10 seconds (s), determined for the calculation time t (therefore calculated selectively on packets captured in the range t and t+Te): - number of packets - average package size - ratio of incoming / outgoing packets - 90th percentile of inter-arrival time - proportion of packets with SYN flag.

[0064] The control module 44 thus obtains a vector of features per sample.

[0065] An example of a feature vector per sample is given in the table Below, in the case of logs:

[0066] [Tables3] Start timestamp Number of requests % GE T %PO ST Q1 duration Median duration Q3 duration Entropy codes Skewness sizes 1622146800 215 0.72 0.28 120ms 230ms 410ms 1.28 1.42

[0067] An example of a characteristic vector per sample is given in the table below in the case of IP packets:

[0068] [Tables4] Start timestamp Number of packets Average size In / out ratio 90p Inter-arrivals % SY N 1622146800 12450 782 bytes 1.32 15ms 0.03

[0069] In a step 103_2, feature vectors for each observation window of size Ti, i = 1 to N, are calculated from these sample feature vectors. In what follows, N = 3, but of course any value of N greater than 2 can be chosen, for example, N greater than or equal to 3.

[0070] The size Ti of an observation window defines the granularity of the detection.

[0071] In one embodiment, it is considered that: Tl = 20 s, T2 = 60 s, T3 = 120 s.

[0072] Considering a given size Ti, for each successive observation window of size Ti, a characteristic vector specifically characterizing the traffic during the considered observation window is determined based on the characteristic vectors of the samples within that observation window. Thus, an observation window of size T1 contains n1 = 2 samples, an observation window of size T2 contains n2 = 6 samples, and an observation window of size T3 contains n3 = 12 samples.

[0073] In the embodiment considered, the feature vectors for an observation window of size Ti are obtained by the control block 44 by aggregating together the n; feature vectors of the n; samples in the window, which amounts to creating time series of different sizes, which amounts to providing as input to each of three parallel processing channels VT1, VT2, VT3, the sample feature vectors, each channel by aggregating the number n;.

[0074] During the training phase, for a set of traffic observation data (logs or IP packets) dedicated to training, a set of feature vectors is thus constructed for each observation window size. The minimum number of samples required for training depends on several factors, including the type of model. For simple models (e.g., linear regression or basic decision trees), a few hundred samples may suffice. On the other hand, for more complex models, such as deep neural networks, a larger number of data points is required (e.g., one or more thousand). A rule of thumb for deep learning is to have 10 times more examples than the number of model parameters. For example, if a model has 10,000 parameters, having at least 100,000 training samples can be a good starting point.

[0075] Then, in step 104, the learning module 50 is adapted to execute a supervised or unsupervised machine learning algorithm for an artificial intelligence model Mi, i = 1 to N (N=3 in the case considered) which, for any feature vector from the training set characterizing an observation window of size Ti, teaches the model Mi to determine whether the traffic observed on this window contains an anomaly. Anomaly detection is formulated here as a classification problem.

[0076] The goal of machine learning is to enable models to generate decisions based on input data, without the need for explicit rules. This process improves as the model is exposed to more data. Several approaches can be used, depending on data availability: supervised learning, which uses labeled data, includes methods such as linear regression or support vector machines (SVMs). In the absence of For labeled data, unsupervised learning can be used, with examples such as clustering techniques and autoencoders. Semi-supervised learning combines these two approaches when there is very little labeled data. Deep learning becomes particularly relevant when dealing with large volumes of data.

[0077] At the end of the learning 104, an artificial intelligence model Mi is thus obtained for each size Ti of observation window: an anomaly detection module 4i implementing the model Mi (typically a memory storing the model definition data and a processor capable of executing the model defined in the memory) is integrated into the monitoring device 40, and the production phase 200 is then triggered.

[0078] Production launch phase

[0079] During the production phase, 200, steps 201, 202 similar to steps 101, 102 are implemented by the control module 44 on observation data which are received in real time and continuously from the gateway 20.

[0080] A step 203 similar to step 103 is then carried out by the control block 44 from these collected real-time data, possibly enriched, and considering three processing channels TRi, i = 1 to 3: it is generated, for example in parallel, continuously, for each successive observation window of size Ti of the traffic, on each processing channel TRi, a vector of statistical characteristics characterizing the traffic selectively occurring during this observation window of size Ti.

[0081] In a step 204, the vector of statistical features characterizing the traffic that occurred during this observation window of size Ti is provided as input to the detection module 4i, and therefore as input to the model Mi, which determines, as a function of this vector, whether the observation window of size Ti has an anomaly or not (decision i).

[0082] In [Fig.4] are represented the successive observation windows of traffic data of different sizes Ti, the classification for each window by the model Mi resulting in a decision "Decision i", of presence or absence of anomalies on the window.

[0083] In step 205, the control block 44 analyzes the decisions provided as output by each model and, if an anomaly detection was positive relative to a time interval (an anomaly was detected), in step 206, it triggers the sending of a pre-alert (i.e., as soon as a decision reports an anomaly, a pre-alert concerning the sample classified as an anomaly is immediately sent, without waiting for the decisions of the other models relative to their observation windows corresponding to this time interval); according to the modes depending on the implementation and, for example, the Mi model that led to the positive detection, following the pre-alert, various forms of protection and / or analysis measures are triggered, for example: total or partial blocking of traffic, more detailed traffic analysis, for example by zooming in on the traffic of the user who caused the problem, etc.

[0084] Each window of size Ti is, in the example considered, totally "contained" in a window of size Ti+1, i = 1, 2 so that the data classified by a model Mi are also evaluated by the model Mi+1 operating on a larger observation window size, as well as other additional data.

[0085] The control block 44 is adapted to resolve conflicting decisions in a set of 207 steps and to determine, for each observation window or at least some of these windows, a decision based on the different decisions made by the three models M1, M2, M3, for example by weighted combination.

[0086] When a model trained on a smaller observation window makes a decision (for example, it detects an anomaly and triggers a pre-alert), this decision can be challenged by models trained on larger observation windows, which have a broader temporal context. Similarly, these latter models take into account the decisions of models trained on smaller observation windows to adjust their own decisions.

[0087] To manage these potential conflicts, in one embodiment, a post-processing correction of decisions is implemented. The frequency of this correction can be chosen according to the needs and constraints of system 1 so as to offer a balance between responsiveness and stability, allowing for periodic review of decisions while avoiding overly frequent changes that could disrupt the system. For example, the correction can occur after each decision of the model trained on the largest time series (model M3) (the frequency equal to the inverse of the largest size TN) or at a lower frequency). In one embodiment, the control block 44 is adapted to select the correction frequency from several fixed frequencies according to the needs and constraints of system 1.

[0088] During this correction, in step 208, the decisions of all the models "included" in the largest time series are combined by the control block 44 in a weighted manner for each corresponding time window. The weights assigned to each model may depend, in one embodiment, on their relevance and reliability, determined during the training phase.

[0089] In step 209, a confidence threshold is then applied by the control block 44 to the combined decision to determine the final decision (the threshold is determined empirically). If the combined decision indicates that there has been a false detection of anomaly (false positive, or false negative) and that it had previously been reported via a pre-alert (case of false positive) by a smaller scale model (test step 210), or conversely, if no pre-alert had been sent when it should have been (case of false negative), a correction notification is issued to inform stakeholders of this revision, in a step 211.

[0090] In an optional step 212, model decisions contradicted by the final (combined) decision, corresponding to false alarms and false negatives, are selected: they give rise (possibly after comparison with thresholds, as described below) to the storage of characteristic vectors and corresponding corrected decisions in a database, which will then be used for the continuous relearning of the models, periodically or not.

[0091] An example of an embodiment of this set of 207 steps is described below, considering 3 trained models M1, M2, M3, a time interval value per sample of Te=10 s, and observation windows of respective sizes T1 = 20 s for M1, T2 = 60 s for M2 and T3 = 120 s for M3.

[0092] The following weighting coefficients are considered for the decisions of the models, depending on their observation window size: 0.1 for decisions from M1, 0.3 for decisions from M2 and 0.6 for decisions from M1.

[0093] The decisions taken by the 3 models considering an observation flowing over 12 successive windows of size Tl (t_l to t_12), before correction, are shown in the table below, in which "1" indicates a decision indicating an anomaly detection and "0" indicates that no anomaly was detected (special case of a binary type classification):

[0094] [Tables5] U L2 t_3 t 4 L5 t_6 t 7 t_8 L9 M0' LU t_12 0 0 1 1 1 0 1 1 1 0 9 Decisions 1 (by M1) 0 0 0 1 Decisions 2 (by M2) 0 1 Decisions 3 (by M3)

[0095] Model M1 delivers a decision (Decision 1) at the end of each t_n, n= 1 to 12 indicating the presence or absence of an anomaly on the window t_n; model M2 delivers a decision (Decision 2) at the end of each sequence starting at t_3n+1 and ending at t_3n+3, n= 0 to 3 indicating the presence or absence of an anomaly on the sequence; model M3 delivers a decision (Decision 3) at the end of each sequence starting at t_6n+1 and ending at t_6n+6, n= 0, 1 indicating the presence or absence of an anomaly on the sequence.

[0096] Correction of the decisions of the Ml model:

[0097] In the example, to correct the decisions of model M1 at the end of period t_6 (similarly at the end of t_12), the weighted average of the decisions of the three models is calculated for each time window t_n, n = 1 to 12. The decision made by a model with a larger observation window size including the considered time t_n is assigned to t_n.

[0098] Let's take a few time windows t_i to illustrate this:

[0099] For t_l: (0 x 0.1) + (0 x 0.3) + (0 x 0.6) = 0

[0100] For t_3: (1 x 0.1) + (0 x 0.3) + (0 x 0.6) = 0.1

[0101] For t_7: (1 x 0.1) + (0 x 0.3) + (1 x 0.6) = 0.7

[0102] Pourt_ll: (Ox 0.1) + (1 x 0.3) + (1 x 0.6) = 0.9

[0103] Thus, if the weighted average is greater than or equal to a threshold, for example 0.8, the control module 44 considers that there is an anomaly on the window t_n. Otherwise, the traffic is considered to be normal.

[0104] [Tableauxô] t 2 t Jr î 4 t 5 16 17 5 8 1 9 t 10 1 il ÈJ2 Q 0 0.1 0.1 0.1 0 0.7 07 0.7 0.9 0.9 Weighted average on each Lo. G 0 t 1 1 0 1 t 1 1 fi 0 Decisions 1 [to be corrected in bold] 0 0 0 Decisions 2 0 4 Decisions 3

[0105] The table above shows the results of the weighted averages and, in bold, the decisions of M1 which need to be corrected, corresponding to the false positives and false negatives (those relating to t3, t4, t5, t7, t8, t9, t11, t12).

[0106] In one embodiment, among these false decisions (false positives and false negatives), the control module 44 selects those to be reinjected (step 212) into the learning update of the model Ml, for example only those corresponding to an anomaly with a weighted mean > 0.9 or refuted with a weighted mean < 0.03 (grey boxes in table 6, corresponding to windows t_l 1, t_12).

[0107] Correction of the decisions of model M 2:

[0108] To correct the decisions of model M2 at the end of period t_6, the control module 44 also takes into account the decisions of M1 and M3: it calculates the weighted average of the decisions over windows of size T2, comprising 3 periods T1 (because model 2 has a window size T2 of 3 x T1), i.e. comprising half a period T3.

[0109] If the weighted average is greater than or equal to a threshold, for example 0.8, the control module 44 considers that there is an anomaly on the considered T2 size window. Otherwise, it considers the traffic to be normal.

[0110] For example, for the window from t_l to t_3, the average of M1's decisions is determined and then weighted by 0.1, M2's decision is weighted by 0.3 and M3's decision over the period T3 including the window t_l to t_6 is weighted by 0.6:

[0111] For example, for the window of size T2 going from t_1 to t_3: (1 / 3 x 0.1) + (0 x 0.3) + (0 x 0.6) = 0.03

[0112] Similarly, for the window of size T2 ranging from t_4 to t_6:

[0113] (2 / 3 x 0.1) + (0 x 0.3) + (0 x 0.6) = 0.06;

[0114] [Tables?] t_1 t_2 L3 t_4 t_5 t_6 L7 L8 t_9 MO LD t_12 a 0 1 1 3 0 1 1 1 0 0 Decisions 1 0.03 0.06 0.7 0.903 Average decisions 2 0.0 0 1 Decisions 2 (to be corrected in bold) o 1 Decisions 3

[0115] The table above shows the results of the weighted averages and, in bold, the decisions of M2 that need to be corrected, corresponding to false positives and false negatives: in the present case, no decision needs to be corrected.

[0116] In one embodiment, among these false decisions (false positives and false negatives), the control module 44 selects those to be reinjected (step 212) into the learning update of the model M2, for example only those corresponding to an anomaly with a weighted mean > 0.9 or refuted with a weighted mean < 0.03 (in the example in Table 7, no false decision of M2; a fortiori, no false decision to be reinjected).

[0117] Correction of the decisions of the M3 model:

[0118] Similarly, to correct the decision of model M3 at the end of period t_6 and relating to the observation window of size T3 starting at t_l and ending at t_6, we calculate the weighted average of the decisions over this observation window (because model 3 has a window size of 6 x Tl): the average of the decisions of M1 is determined, then weighted by 0.1, the average of the decisions of M2 is determined then is weighted by 0.3 and the decision of M3 over the period T3 including the window t_l to t_6 is weighted by 0.6:

[0119] For example, for the observation window of size T3, over the periods t_1 to t_6:

[0120] (3 / 6 x 0.1) + (0 x 0.3) + (0 x 0.6) = 0.05

[0121] [Tables8] L! L2 t_3 t_4 t_5 Ls L7 t_8 L9 t_to t_i i t_12 0 0 1 0 1 ii 0 0 Decisions 1 0 o G t Decisions 2 0.05 0.81 Average decisions 3 0 1 Decisions 3 (in bold: to be corrected)

[0122] The table above shows the results of the weighted averages and, in bold, the decisions of M3 that need to be corrected, corresponding to false positives and false negatives: in the present case, no decision needs to be corrected.

[0123] In one embodiment, among these false decisions, the control module 44 selects those to be reinjected (step 212) into the training update of the model M3, for example only those corresponding to an anomaly with a weighted mean > 0.9 or refuted with a weighted mean < 0.03 (in the example in Table 8, no false decisions, a fortiori no false decisions to be reinjected into the training of M3).

[0124] In summary, this invention enables precise and responsive network anomaly detection, taking advantage of a multi-scale approach to adapt the granularity and context, and a decision correction mechanism exploiting the benefits of different time scales.

[0125] It should be noted that the threshold values ​​indicated above are only by way of example; other values ​​may be used; they may be determined empirically.

[0126] In the example described above, the traffic monitoring device 40 implemented the learning phase 100 and the production phase 200. In another embodiment, separate modules perform the steps of these respective phases.

[0127] Figure 1 illustrates the impact of the observation window size on anomaly detection and false alarm reduction. It represents the evolution of the number of queries R (on the y-axis) over time t (on the x-axis). The circled sample indicated by an arrow could be detected as an anomaly if the number of queries is compared locally (small observation window); however, with a broader view (larger observation window), this can be considered normal behavior, as the same pattern is repeated in subsequent periods.

[0128] In the preceding example, suitable machine learning models were considered for classifying samples as normal / abnormal; in other embodiments, the classifications are more refined and further sort anomalies, for example, according to the type of anomaly (i.e., with more than two classes, for example: "normal behavior", "type 1 anomaly", "type 2 anomaly", etc.), for example "Man in the Middle", "Cross-Site Scripting", "Denial of Service", etc.

[0129] The invention thus proposes an efficient method for detecting anomalies in network communications, by exploiting, at each detection time, statistical characteristics calculated over time windows of different sizes, This allows for the capture of both short- and long-term anomaly dynamics. This improves the detection of various anomalies, whether sudden or gradual. Short windows offer rapid and localized detection of sudden anomalies but can be prone to false positives and may fail to detect slowly developing anomalies. Conversely, long windows are more robust to normal variations and are capable of detecting gradual anomalies, but at the cost of reduced responsiveness and granularity, as short-term anomalies may be masked.

[0130] In embodiments, the solution according to the invention comprises training traffic classification models into normal or abnormal, each specialized at different time scales, thereby leveraging contextual information specific to each level of granularity. Their combination, weighted by their temporal context, produces a more robust, faster, and contextualized detection.

[0131] In some embodiments, detection decisions are corrected a posteriori by exploiting the decisions of models trained on windows of different sizes, which makes it possible to invalidate certain detected anomalies and reduce false alarms.

[0132] False positives and false negatives are fed back for continuous relearning, allowing the models to constantly improve.

[0133] The applications are numerous: examples include the monitoring and security of enterprise information systems, the protection of critical infrastructure and industrial systems, intrusion detection for service providers and telecom operators, and traffic monitoring for network optimization and sizing. Beyond attack detection, it can also be used to identify malfunctions, causes of performance degradation, or suboptimal use of network resources. Its generic nature makes it easily adaptable to various application contexts requiring detailed and responsive time series analysis.

[0134] In the foregoing, it was assumed that communications passed through a gateway module 20. In other embodiments, the invention is implemented without a gateway module, with a logging system that records observation data relating to communications taking place in the system between users and resources. Depending on the embodiment, the traffic data considered is represented, for example, in the form of PCAP (Packet Capture Application Protocol) files, or in the form of application-level logs, or any other form representing communications in the network, network traffic, which is obtained simply by capturing the traffic passing between users and resources at a probe placed in the network. In one embodiment, the observations are obtained by capturing the traffic passing between users and resources, at the level of a probe placed in the network.

[0135] In the preceding text, sizes for observation windows or for timing sample production were considered, expressed in terms of time. In another embodiment, the sizes considered are expressed in terms of the number of logs (or IP packets).

[0136] In step 103 described above, samples corresponding to 10 s were considered, and then feature vectors calculated for each of these samples were time-seriesed in number n; for the Mi model, i = 1 to 3, to generate the feature vectors of the observation windows of size Ti. In another embodiment, instead of the 10 s samples used in parallel on each channel, samples grouping the logs (or packets) over a duration Ti are considered on each VTi channel, and the feature vectors obtained from these samples are those corresponding to each size Ti. Thus, instead of performing fixed sampling and then multi-scale grouping to create time series of variable sizes, multi-scale sample generation is performed (which incorporates the idea of ​​multi-scale time series).The idea here is to consider several sample sizes in the parallel processing channels. The sample size can be defined by the number of observations, for example 3 logs / packets, or by a time duration, such as all logs / packets recorded for 10 seconds.

[0137] Such a multi-scale sample generation consists of "accumulating" observations on several levels. For example, if we define the samples temporally and consider three scale levels: from a time t, a first scale consists of describing all the observations (logs / packets, etc.) passing for 20 seconds, i.e. between t and t+20 s, a second describes the observations recorded for 60 seconds between t and t+60 s, and a third, for 120 seconds, between t and t+120s.

[0138] Example of constructing a feature vector of a sample

[0139] With reference to [Fig.5] describing the logs to be grouped into a sample, the following frequency and ratio metrics are calculated, considering the aspects "HTTP Method" and "Status Code". Absolute frequency#: HTTP method:

[0140] GET: 3

[0141] POST: 2

[0142] PUT: 1

[0143] DELETE: 1 Status code:

[0144]

[0145]

[0146]

[0147]

[0148] 200: 3 201: 1 404: 1 204: 1 400: 1 Relative frequency (proportion)#: HTTP method:

[0149]

[0150]

[0151]

[0152] GET: 3 / 7 = 0.429 POST: 2 / 7 = 0.286 PUT: 1 / 7 = 0.143 DELETE: 1 / 7 = 0.143 Status code:

[0153]

[0154]

[0155]

[0156]

[0157]

[0158] 200: 3 / 7 = 0.429 201: 1 / 7 = 0.143 404: 1 / 7 = 0.143 204: 1 / 7 = 0.143 400: 1 / 7 = 0.143 Ratio between the two most frequent values: HTTP Method:

[0159] Ratio (GET / POST): 3 / 2 = 1.5 Status Code:

[0160] Ratio (200 / 201): 3 / 1 = 3.0 Top N of the most frequent values#: Top 2 HTTP Method:

[0161]

[0162] GET: 3 occurrences POST: 2 occurrences Top 2 Status Code:

[0163]

[0164] 200: 3 occurrences 201: 1 occurrence (with 404, 204, 400) Ratio of the most frequent value HTTP Method:

[0165] Ratio (GET): 3 / 7 = 0.429 Status code:

[0166] Ratio (200): 3 / 7 = 0.429

[0167] The following diversity metrics are calculated for these aspects: "http Method" and "Status Code". HTTP method:

[0168] Observations: GET, POST, PUT, DELETE, POST, GET

[0169] Frequencies: GET: 3, POST: 2, PUT: 1, DELETE: 1

[0170] Total number of observations: 7

[0171] Number of distinct values: 4 (GET, POST, PUT, DELETE)

[0172] Ratio between the number of distinct values ​​and the total number of observations:

[0173] 4 / 7«0.57174 47^0.57

[0174] Shannon entropy: = 1.842 bits

[0175] Simpson's Diversity Index "0.653

[0176] Gini index «0.653 «0.653 «0.653 Status code:

[0177] Observations: 200, 201, 404, 200, 204, 400, 200

[0178] Frequencies: 200:3, 201:1, 404:1, 204:1, 400:1

[0179] Total number of observations: 7

[0180] Number of distinct values: 5 (200, 201, 404, 204, 400)

[0181] Ratio between the number of distinct values ​​and the total number of observations: 5 / 7 «0.71475 «0.714

[0182] Shannon entropy "2.128 bits

[0183] Simpson's Diversity Index "0.775

[0184] Gini coefficient "0.775

[0185] The feature vector thus comprises, for the "HTTP Method" aspect, the following components:

[0186] 1. Total number of observations: 7

[0187] 2. Absolute frequency of GET: 3

[0188] 3. Absolute POST frequency: 2

[0189] 4. Absolute frequency of PUT: 1

[0190] 5. Absolute frequency of DELETE: 1

[0191] ô.Relative frequency of GET: 0.429

[0192] 7. Relative frequency of POST: 0.286

[0193] 8. Relative frequency of PUT: 0.143

[0194] 9. Relative frequency of DELETE: 0.143

[0195] Ratio between GET and POST: 1.5

[0196] 11. Top 2 most frequent methods: GET, POST

[0197] 12. Most Frequent Value (GET) Ratio: 0.429

[0198] 13. Number of distinct values: 4

[0199] 14. Diversity ratio: 0.571

[0200] 15. Shannon entropy: 1.842 bits

[0201] lô.Simpson Diversity Index: 0.653

[0202] This part of the feature vector relating to the "HTTP Method" aspect can be represented as follows:

[0203] HTTP_Features=[7,3,2,1,1,0.429,0.286,0.143,0.143,1.5, "GET,POST",0.42 9,4,0.571,1.842,0.653,0.653]

[0204] To manage a fixed size of the "Absolute Frequency" part, which can vary depending on the different values ​​taken by the method, we can limit ourselves to the most common and well-known methods: GET, POST, PUT, HEAD etc.

[0205] Similarly, the feature vector thus comprises, for the "Status Code" aspect, the following components:

[0206] 1. Total number of observations: 7

[0207] 2. Absolute frequency of 200:3

[0208] 3. Absolute frequency of 201:1

[0209] 4. Absolute frequency of 404:1

[0210] 5. Absolute frequency of 204:1

[0211] ô. Absolute frequency of 400:1

[0212] 7. Relative frequency of 200: 0.429

[0213] 8. Relative frequency of the others (201, 404, 204, 400): 0.143 each

[0214] 9. Ratio between 200 and 201: 3.0

[0215] Top 2 status codes: 200, 201

[0216] 11. Ratio of the most frequent value (200): 0.429

[0217] 12. Number of distinct values: 5

[0218] 13. Diversity ratio: 0.714

[0219] 14. Shannon entropy: 2.128 bits

[0220] 15.1 Simpson Diversity Index: 0.775

[0221] lô.Gini index: 0.775

[0222] This part of the feature vector relating to the "status code" aspect can be represented as follows:

[0223] Status_Features=[7,3,1,1,1,1,0.429,0.143,0.143,0.143,0.143,3.0, "200, 201",0.429,5,0.714,2.128,0.775,0.775]

[0224] The same consideration as before applies to having a fixed size.

[0225] The steps of the process pertaining to the control module 44, respectively the learning module 50, can be implemented by executing software instructions (stored in a memory of the monitoring device 40) on a processor of the monitoring device 40. Alternatively, they can be implemented by dedicated hardware, typically a digital integrated circuit, or specific (ASIC) or based on programmable logic (e.g. FPGA / Field Programmable Gate Array).

Claims

1.

2. Demands Method for detecting anomalies in communications implemented in a network (1), said process comprising the following steps implemented by an electronic communications monitoring device (40) in the network: - collect successive observation data on communications; - perform a multi-scale analysis of these observational data collected by exploiting said observational data collected on each of N processing channels VTRi, i = 1 to N, N>2, each comprising an automatic communication classification module named Mi (41, 42, 43) associated with a respective observation window size Ti, with Ti Tj for i,j = 1 to N and ij, by implementing the following steps: for each of successive observation windows of size Ti, a respective vector of communication characteristics is determined as a function of those of the collected observation data selectively present in said observation window of size Ti; the feature vector is provided as input to the automatic classification module Mi (41, 42, 43) and the classification model Mi determines, based on said feature vector, a detection result relative to said observation window of size Ti, indicating whether the communications that occurred during said observation window of size Ti contain an anomaly; - determine whether an anomaly is present or not at a given time based on the detection results of each processing channel VTRi, i = 1 to N, each relative to an observation window of size Ti corresponding to said given time. Anomaly detection method according to claim 1, wherein the following steps are implemented by the monitoring device (40) in a preliminary phase: - collect successive observation data of communications; - considering a fixed size Ti, i = 1 to N: determine communication characteristic vectors, each vector being determined based on observation data selectively from a respective observation window of communications, of size Ti, perform machine learning of the classification module Mi on the basis of the determined communication characteristic vectors, so that it determines, based on any of the determined communication characteristic vectors, whether the traffic which took place during said observation window of size Ti has an anomaly.

3. A method for detecting anomalies according to any one of the preceding claims, wherein an anomaly is determined to be present or not at a given time as a function of the combination of said detection results.

4. Anomaly detection method according to the preceding claim, wherein an anomaly is determined to be present or not at a given time as a function of the combination of said detection results, each weighted according to a respective weighting coefficient.

5. Anomaly detection method according to the preceding claim, wherein an anomaly is determined to be present or not at a given time as a function of the combination of said detection results, each weighted according to the relative size Ti of the respective observation window used for the determination of said result by the classification module Mi. An anomaly being determined to be present or not at a given time by comparison, with at least one threshold, of the combination of the N detection results, each weighted according to the relative size of the respective observation window used Ti for the determination of said result.

6. A method for detecting anomalies according to any one of the preceding claims, wherein, given an observation window of size Ti, the average of the detection results for each module is calculated over the observation window Ti. classification Mj on the observation windows of size Tj included in said observation window of size Ti, and the sum of the following terms is determined: - the average calculated by each classification module Mj, - the detection result of the classification module Mi on the observation window of size Ti, - the detection result of each classification module Mk on the observation window of size Tk including said observation window of size Ti, each of the said terms being weighted by a respective weighting coefficient assigned to each classification module Mi, Mj, Mk.

7. Computer program, intended to be stored in the memory of a traffic monitoring device (40) and further comprising a microcomputer, said computer program comprising instructions which, when executed on the microcomputer, implement the steps of a method according to any one of the preceding claims.

8. Electronic communication monitoring device (40) in a network (1), said device being adapted to collect successive observation data of communications and to perform a multi-scale analysis of this collected observation data by exploiting said collected observation data on each of N processing channels VTR1 ... VTRN, i = 1 to N, N>2, each processing channel VTRI, i = 1 to N, N>2, of the device comprising an automatic communication classification module named Mi (41, 42, 43) associated with a respective size Ti of observation window, with Ti Tj for i,j = 1 to N and ij; said device (40) being adapted to determine, for each of successive observation windows of size Ti, a respective vector of communication characteristics as a function of those of the collected observation data present selectively in said observation window of size Ti; said device (40) being adapted to provide the feature vector as input to the automatic classification module Mi, the classification model Mi being adapted to determine, as a function of said feature vector, a detection result relating to said observation window of size Ti, indicating whether the communications occurring during said observation window of size Ti contain an anomaly; said device (40) being adapted to determine whether an anomaly is present or not at a given time as a function of the detection results of each processing channel VTRi, i = 1 to N, each relating to an observation window of size Ti corresponding to said given time.

9. Electronic communications monitoring device (40) according to claim 8, adapted to determine whether an anomaly is present or not at a given time as a function of the combination of said detection results.

10. Electronic communications monitoring device (40) according to claim 9, adapted to determine whether an anomaly is present or not at a given time as a function of the combination of said detection results, each weighted according to a respective weighting coefficient.

Images