Method for the detection of anomalies in data from the monitoring of complex systems
By employing two forecasting models to compare predictions in complex systems, the method addresses the limitations of traditional anomaly detection, enabling early and sensitive anomaly identification in 5G networks and server farms.
Patent Information
- Application Number
- PCT/EP2026/063152
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-07
- Filing Date
- 2026-05-06
- Publication Date
- 2026-11-12
AI Technical Summary
Traditional anomaly detection methods in complex systems like 5G networks and server farms are inadequate due to their reliance on predefined thresholds and supervised learning, which require exhaustive labeled data and struggle with new anomalies, while unsupervised and semi-supervised methods fail to leverage historical anomalous data effectively.
A computer-implemented method using two forecasting models, one trained on normal data and another on mixed normal and unlabeled anomalous data, compares their forecasts to identify anomalies through a metric-based criterion, allowing early detection by analyzing discrepancies.
This approach reduces detection time and improves sensitivity by exploiting historical anomalous data, providing robust and rapid anomaly identification in complex environments.
Smart Images

Figure EP2026063152_12112026_PF_FP_ABST
Abstract
Description
DESCRIPTION Title: Method for detecting anomalies in data from the monitoring of complex systems technical field
[0001] The present invention relates to the general field of data analysis and, more particularly, to methods and systems for detecting anomalies in data. The invention finds a particularly advantageous, though not exclusive, application in monitoring complex systems such as telecommunications networks, especially 5G networks and beyond, as well as IT infrastructures such as server farms. Prior art
[0002] Continuous monitoring of operational systems is essential to ensure their reliability, performance, and security. Anomaly detection, which aims to identify unexpected or undesirable states or behaviors (errors, failures, attacks, etc.) that deviate from the norm, is an important component of this monitoring. In the context of modern systems (such as 5G networks, large server farms, or industrial systems), characterized by increased complexity, scale, and dynamism, traditional anomaly detection methods are showing their limitations.
[0003] Simple statistical approaches, based on predefined thresholds on individual metrics, are often insufficient to capture complex interactions and subtle changes in behavior in multivariate data.
[0004] Machine learning-based techniques have been extensively explored. Supervised methods can be effective but require exhaustively labeled training datasets, including both normal data and representative examples of all possible anomalies. Obtaining such data is often expensive and difficult, and these methods generally struggle to detect new or unexpected anomalies because their generalization capacity is limited to the types of anomalies seen during training.
[0005] Unsupervised and semi-supervised methods aim to address the need for labeled data. Unsupervised methods attempt to distinguish anomalies as rare or isolated points without prior knowledge. Semi-supervised methods are often trained only on data considered normal and then identify as abnormal anything that deviates significantly from this learned model of normality. While promising for detecting unexpected anomalies, these approaches cannot exploit the potentially useful information contained in historical data that, although not perfectly labeled, is known to contain occurrences of anomalies.
[0006] Among semi-supervised approaches, those based on prediction are common. They train a model to predict future values of data based on recent past data. An anomaly is typically detected when the error between the predicted value and the value actually observed later exceeds a certain threshold. A major drawback of this approach is the inherent detection delay: it is necessary to wait for the actual data to become available before comparing it to the prediction. In contexts requiring a rapid response, this delay can be detrimental. Summary
[0007] This disclosure improves the situation.
[0008] To this end, a computer-implemented method for detecting anomalies in data is proposed, the method comprising: - generate, using a first forecasting model trained using a first training dataset containing segments of normal data, a first forecast corresponding to an input data; - generate, using a second forecasting model trained on a second training dataset comprising mixed data segments, where the mixed data segments include both normal data segments and unlabeled abnormal data segments, a second forecast corresponding to the input data; and - identify an anomaly associated with the input data when a metric satisfies a predefined criterion, said metric being based on a comparison between the first forecast and the second forecast.
[0009] The term "computer-implemented process" refers to any process in which one or more steps are executed by at least one processor or computing unit following the execution of software instructions stored in memory, inherently requiring the use of computing resources to process data and obtain a technical result.
[0010] The term "anomaly detection" refers to the technical process of identifying elements, events, or observations (data points, sequences, patterns) in a dataset that do not conform to expected or normal behavior, with an "anomaly" representing a significant deviation from the majority of the data or a reference model of normal behavior.
[0011] The term "data" refers to information, in digital form, obtained during the monitoring of an operational system or process. This data represents measurements, observations, or characteristic states of said system or process. This data can be structured, potentially organized sequentially, and can be univariate or multivariate.
[0012] The term "forecasting model" refers to an algorithmic or mathematical model, often derived from machine learning or statistics, trained on historical data to predict future values, based on observed past values.
[0013] The term "normal data" refers to data that represents the expected, typical, stable, or desired behavior of the monitored system, in the absence of known or significant anomalies.
[0014] The term "abnormal data" refers to data that contains patterns, values, or dynamics that deviate from the defined normal behavior, corresponding to periods when the monitored system experiences an anomaly.
[0015] The term "unlabeled" refers to the fact that individual data points are not explicitly marked or classified as "normal" or "abnormal".
[0016] The term "metric" refers to a quantitative measure calculated to assess, for example, the dissimilarity or difference between the first forecast (from the first model) and the second forecast (from the second model) for the same forecast horizon. Such a metric is, for example, a Euclidean distance, a Mean Squared Error (MSE), which represents the average of the squared differences between the forecasts, a Mean Absolute Error (MAE), which represents the average of the absolute values of the differences between the forecasts, or a score based on the relative position of the differences within a distribution (for example, a score based on error percentiles).
[0017] The term "predefined criterion" refers to a specific rule or condition, established in advance (e.g., a threshold value, a range of values, a statistical test), which is applied to the metric value to decide whether to identify an anomaly.
[0018] Such a method substantially reduces the detection time. Indeed, while previous methods typically require waiting for the arrival of future real data to compare it to the forecast and calculate an error, the method according to the invention performs detection by directly comparing the forecasts from the first and second models. This comparison can be carried out as soon as the forecasts are generated, thus enabling potentially much earlier identification of an anomaly and a faster response.
[0019] Furthermore, by training the second model on mixed data including unlabeled anomalies, the invention exploits the implicit information contained in this historical anomalous data. This exposure allows the second model to potentially better anticipate certain anomalous dynamics, leading to a more pronounced divergence between the predictions of the two models in the presence of an anomaly. This can improve the sensitivity and robustness of detection, including for anomalies similar to those observed unsupervised during the training of the second model.
[0020] The steps in this process can be carried out sequentially or, at least partially, in parallel. Thus, the training of the first and second models can be performed simultaneously. Similarly, the generation of the first and second forecasts can be carried out simultaneously.
[0021] The data can be time series data.
[0022] The term "time series data" refers to a sequence of data points measured or observed at successive times, typically ordered chronologically and collected at regular or irregular time intervals, which may be univariate or multivariate.
[0023] This data may originate from the monitoring of telecommunications networks or server farms.
[0024] The term "telecommunications networks" refers to all the infrastructure, equipment, protocols, and systems that enable the transmission of information (such as voice, data, and video) over a distance. This includes, but is not limited to, mobile networks (e.g., 5G, LTE / 4G, 3G), wireless local area networks (e.g., Wi-Fi), wide area networks (WANs), fixed networks (e.g., DSL, fiber optics), radio access networks (RANs), core networks, software-defined networks (SDNs), and virtualized network functions (VNFs). Monitoring can focus on metrics such as performance, quality of service, traffic, equipment status, and more.
[0025] The term "server farms" refers to a collection of interconnected computer servers, collectively managed to provide computing resources. This includes, but is not limited to, enterprise data centers, public or private cloud infrastructures, web server farms, high-performance computing (HPC) clusters, and virtualization environments. Monitoring can focus on metrics such as resource utilization (CPU, memory, disk, network), application latency, error rates, temperature, and more.
[0026] The first forecasting model and the second forecasting model can be of the same type of architecture.
[0027] The term "architecture", in the context of a forecasting model, refers to the fundamental design and organization of said model, including the types of processing layers used (e.g., recurrent layers, linear layers, attention mechanisms), their mode of connection, and the overall structure (e.g., sequential, encoder-decoder), which collectively determine how information is processed and transformed by the model to generate a forecast.
[0028] This characteristic allows for a more reliable and meaningful comparison between their respective predictions. Indeed, using the same architecture minimizes the influence of the model structure on the difference in predictions.
[0029] The first forecasting model and / or the second forecasting model may feature a type of forecasting model architecture selected from the group consisting of Recurrent Neural Networks, RNNs, LSTM networks, Transformers, and Linear models.
[0030] "Recurrent Neural Network (RNN)" refers to a type of neural model architecture designed for sequential data processing, characterized by recurrent connections that allow information from previous time stages to influence the processing of the current time stage via a hidden internal state, making it suitable for modeling temporal dependencies in series.
[0031] "Long Short-Term Memory (LSTM) Network" refers to a specific variant of RNN architecture, designed to better capture long-term dependencies in sequences by mitigating the vanishing gradient problem, characterized by a complex internal structure comprising memory units (cells) and gate mechanisms (such as forget gate, in gate, out gate) that regulate the flow of information and allow the network to selectively remember or forget information over long periods.
[0032] "Transformer" refers to a type of model architecture, which relies primarily on self-attention mechanisms to assess relative importance and model relationships between different (temporal) positions in the input sequence in order to generate the output.
[0033] "Linear Models" refers to a type of forecasting model architecture that primarily assumes a linear relationship between past observations and future forecasts. These models can operate on raw data or, often, after a temporal decomposition step of the data (e.g., into trend and seasonality).
[0034] The DLinear model, for example, is a type of linear model that first decomposes the input data into a trend component and a residual (seasonality / rest) component, then applies a separate linear neural network layer to each of these components to forecast over the desired horizon, and finally combines the forecasts of the two components to obtain the final forecast. Other linear models might use different forms of linear regression or exponential smoothing approaches that are considered linear.
[0035] The data can be multivariate and include several variables, and the determination of the metric may include: calculate, for each variable in said multivariate data, a variable-specific difference between the first and second predictions for said variable; and aggregate said specific differences to obtain the metric.
[0036] The term "multivariate data" refers to data where several different variables or metrics are observed and recorded simultaneously over time for the same system or process under surveillance.
[0037] The term "aggregating said specific differences" refers to the process of mathematically combining the values of the specific differences (calculated individually for each variable in the multivariate data) into a single overall representative value, which constitutes the final metric.
[0038] By individually analyzing the discrepancy between the two models' predictions for each variable, the method can capture anomalies that might only affect a subset of variables or manifest differently across metrics—anomalies that could be masked or diluted in a direct overall comparison. Furthermore, this variable-based calculation allows for the application of normalization or weighting mechanisms (such as the use of percentile-based scores) before aggregation, ensuring that variables with very different scales or variances contribute equally to the final metric, without any large-scale variable unduly dominating the result.The aggregation step then consolidates this multi-dimensional information (one dimension representing one variable) into a single anomaly score, facilitating decision-making while maintaining a comprehensive view of the system's behavioral deviation across all monitored variables. This approach improves the robustness and sensitivity of detection in complex multivariate environments.
[0039] The aggregation of variable-specific differences may include: determine, for each variable, a score based on a percentile of its variable-specific difference within a predefined or calculated error distribution; and calculate the average of the scores across the variables to obtain the metric.
[0040] The term "score" refers to the numerical value, for example normalized (e.g. between 0 and 1, or 0 and 100), which serves to quantify the degree of unusualness or anomaly of the difference observed for the specific variable, relative to its historical or expected behavior, in a way that is comparable between different variables.
[0041] The term "percentile of its variable-specific difference" refers to the relative rank of a commonly measured difference between the prediction of the first model and the prediction of the second model for a given variable, expressed as a percentage. This rank is calculated relative to the historical or expected error distribution associated with that same variable. The percentile value indicates the proportion (as a percentage) of historical differences recorded in the error distribution that are less than or equal to the commonly measured difference for that variable.
[0042] As an example, consider a multivariate time series monitoring a server, and focus on a specific variable: CPU load (expressed as a percentage).
[0043] Situation at time t = 105: The first model (trained on normal data only) predicts a CPU load of 30% for the next time step. The second model (trained on mixed data) predicts a CPU load of 55% for the same next time step.
[0044] The difference specific to the variable (CPU load) is therefore |55% - 30%| = 25%.
[0045] Historical Context (Error Distribution for CPU Load): Assume that the last 1000 time steps (or a representative training period) have been analyzed. For each of these time steps, the absolute difference is calculated between the CPU load predictions of the two models.
[0046] This historical analysis (the error distribution for CPU load) reveals that most differences were small, often between 0% and 5%, that 90% of all historical differences were less than 18%, and that only 2% of historical differences were greater than or equal to 25%.
[0047] To calculate the percentile, we compare the current difference of 25% to this historical distribution. Since only 2% of past differences were equal to or greater than 25%, this means that 98% of past differences were less than 25%.
[0048] The percentile of its specific difference to the variable for CPU load at time t = 105 is therefore 98%.
[0049] The term "error distribution" refers to the statistical representation (e.g., as a histogram, a set of quantified values, or an estimated probability density function) of the historically observed or expected differences between the predictions of the first model and those of the second model, specifically for that variable. This distribution serves as a reference for assessing the unusualness of a new difference measured for that variable, notably through the calculation of its percentile within that distribution.
[0050] The term "mean score across variables" refers to the value resulting from calculating the arithmetic mean of all individual scores (previously defined, typically based on percentiles) that were determined for each of the multiple variables in the multivariate data. This mean constitutes the final aggregate metric, providing a single, overall indication of the level of anomaly or divergence between models across all considered variables.
[0051] Using percentiles based on variable-specific error distributions, followed by averaging the resulting scores, offers several technical advantages. First, it ensures effective normalization, making the degrees of anomaly comparable for variables with very different scales and dynamics. Second, it inherently takes into account the reference levels of variability specific to each variable. Third, averaging the scores allows for the detection of multivariate anomalies, where several variables may exhibit moderate but collectively significant deviations, even if none exceeds an extreme threshold individually. Fourth, the final aggregate score provides an interpretable and consistent measure of the overall magnitude of the anomaly. Finally, the use of ranks (via percentiles) lends a degree of robustness to extreme outliers in the raw differences.
[0052] The calculation of the variable-specific difference for a given variable may also include: divide a forecast horizon for said variable into a plurality of slices; calculate a difference within each slice; and determine the variable-specific difference based on the differences calculated for the slices, optionally by selecting the maximum difference across the slices.
[0053] The term "dividing the forecast horizon for said variable into a plurality of slices" refers to the act of segmenting the sequence of values predicted by the models (the first and second) for a given variable over the entire forecast horizon (e.g., the next N time steps that the models attempt to predict) into several shorter, typically consecutive, time subsequences of equal or predefined size. For example, a forecast horizon of 96 time steps could be divided into 12 consecutive slices of 8 time steps each.
[0054] Thus, rather than calculating the difference between the predictions of the first and second models at each individual time step of the forecast horizon, this approach aims to obtain a more robust measure of the divergence for the variable under consideration over this horizon.
[0055] The process is as follows.
[0056] The forecast horizon (the sequence of N values predicted by each model for the variable) is divided into m slices. Then, for each individual slice, a measure of the difference between the predictions of the first model and those of the second model within that slice is calculated. This intra-slice measure could be, for example, the average difference over the time steps of the slice, the maximum difference observed in the slice, or another aggregate metric representative of the divergence within that time subsequence.
[0057] The differences calculated for each of the m slices are then used to determine the final variable-specific difference for the entire horizon. One method for this aggregation, as optionally mentioned in the claim, is to select the maximum value among the differences calculated for each slice. In other words, the slice with the greatest divergence (according to the intra-slice measure used) determines the value of the variable-specific difference for the entire horizon.
[0058] The result of this process is a single value (the "variable-specific difference") that represents the most significant divergence observed over a sub-period (slice) of the forecast horizon for that particular variable.
[0059] The predefined criterion can be a threshold value, with an anomaly being identified when the metric exceeds said threshold value.
[0060] The threshold value can be determined based on an expected percentage of anomalies in the data or by evaluating multiple potential thresholds.
[0061] Thus, the predefined threshold value can be determined using different approaches. In one approach, the threshold value can be based on an estimate or requirement regarding an expected percentage of anomalies in the data, this percentage being, for example, defined by an operator or derived from operational constraints. In this case, the threshold value can be chosen as the value of the metric corresponding to a specific percentile (for example, the 99th percentile if 1% anomalies are expected) in the distribution of difference metrics observed in a representative dataset.
[0062] According to a second approach, determining the threshold value can be based on the empirical evaluation of multiple potential thresholds. This evaluation may involve applying different candidate threshold values to an evaluation dataset, preferably labeled with the location of the actual anomalies. For each candidate threshold, detection performance metrics (such as the F1 score, the subsequently defined F-rate, precision, recall, or a combination thereof) are calculated. The predefined threshold value is then selected as the one that optimizes the desired performance metric or offers a compromise deemed optimal between different metrics on the evaluation dataset.
[0063] The first and second forecasts can be generated for a predefined forecast horizon.
[0064] The data may include selected metrics from the group consisting of: CPU load, network usage, memory usage, TCP / UDP statistic, packet loss rate, latency metric.
[0065] The process may also include: generate, using the first forecasting model, a third forecast corresponding to the input data; and combine the first metric and a second metric to identify an anomaly, optionally by averaging anomaly scores derived from each metric, said second metric being based on a comparison between the third forecast and actual data corresponding to the time period of the third forecast.
[0066] The process can thus implement a hybrid anomaly detection approach. This hybrid approach aims to combine information from the comparison between the two forecast models (as described previously and resulting in a first metric) with information from a more traditional comparison between a forecast and the actual observed data (ground truth).
[0067] In this hybrid embodiment, the process includes the additional step of generating, using only the first forecasting model (the one trained on normal data), a third forecast corresponding to the input data. This third forecast covers the same future forecast horizon as the first and second forecasts. Essentially, this third forecast is the standard prediction that the first model would provide over the considered future horizon.
[0068] The process may then include the step of obtaining the actual data, also known as "ground truth," which corresponds to the time period covered by the third forecast. Obtaining this ground truth inherently requires waiting for time to elapse and for these observations to be actually measured or collected by the monitored system. Once the ground truth data is obtained, the process determines a second metric. This second metric is based on a comparison between the third forecast (from the first model) and the corresponding ground truth data. It quantifies the forecast error of the first model relative to the observed reality, which is a classic approach to forecast-based anomaly detection.
[0069] The final step in this hybrid approach involves combining the first metric (derived from the comparison between the first and second models) and the second metric (derived from the comparison between the first model and the actual data) to perform the final anomaly identification. In one specific embodiment, this combination can be achieved by averaging anomaly scores derived independently from each metric. For example, a first anomaly score can be derived from the first metric (potentially via a percentile calculation as described previously), a second anomaly score can be derived from the second metric (potentially also via a percentile calculation based on a distribution of historical forecast errors), and these two scores are then averaged to obtain a final hybrid anomaly score. This final hybrid score can then be compared to a predefined threshold to identify an anomaly.
[0070] To implement this combination effectively, particularly in contexts requiring a rapid response, a specific implementation can be used. At a given time or for a given input (time t), the hybrid anomaly score can be calculated by combining: a first anomaly score derived from the first metric, calculated immediately by comparing the first and second forecasts generated for the future horizon from the input data at time t; and a second anomaly score derived from the second metric, which was calculated using the third forecast that had been generated at the previous time step (time t-1) and the corresponding actual data, the latter having become available between time t-1 and time t.
[0071] This temporal approach makes it possible to incorporate the information from the comparison with reality (via the second metric) with only one time step of delay in the final score, while benefiting from the immediate information from the comparison between the two models (via the first metric).
[0072] The goal of this hybrid approach is to leverage the complementary strengths of the two detection strategies. The first metric allows for potentially faster detection (not requiring ground truth) and is sensitive to behavioral divergences learned by the two models, while the second metric offers a direct comparison with reality, which can be useful for identifying anomalies not seen or poorly represented in the mixed training data of the second model, or for confirming an anomaly. Such a hybrid approach can improve the robustness or overall performance of detection in certain scenarios (as illustrated by some experimental results).
[0073] The process may also include triggering an alert or initiating a mitigation action when an anomaly is identified.
[0074] The term "alert" refers to a signal or notification generated by the system following the identification of an anomaly. The main purpose of an alert is to inform a human operator, a supervisory system, or a recording mechanism of the occurrence of the detected anomaly, so that appropriate attention can be given to it.
[0075] Non-exhaustive examples of such an alert include: the display of a visual indicator (for example, a red light, a warning icon) on a dashboard interface; the recording of a detailed entry describing the anomaly in a log file; the sending of an email, text message (SMS) or notification via an instant messaging application to one or more predefined administrators; the issuance of an API call to a higher-level management system, such as an orchestrator, an incident management system ("ticketing system") or a centralized analysis platform; the activation of a sound alarm in an operations center.
[0076] The term "mitigation action" refers to an operation or procedure, typically predefined and potentially automated, initiated by a system in response to the identification of an anomaly. The objective of a mitigation action is to counteract the negative effects of the detected anomaly, resolve its underlying cause, restore normal operation, or prevent further degradation of the performance or security of the monitored system.
[0077] Non-exhaustive examples of such mitigation actions, relevant in the context of telecommunications networks or server farms, include: automatic rerouting of network traffic to bypass a faulty link or equipment identified as the cause or consequence of the anomaly; blocking or filtering traffic from suspicious IP addresses identified during an attack-type anomaly (e.g., distributed denial of service - DDoS); adjusting Quality of Service (QoS) parameters to preserve critical flows during a congestion anomaly; controlled restart of a software service, virtualized network function (VNF) or virtual machine exhibiting abnormal behavior; the dynamic allocation of additional resources (e.g., CPU computing power, RAM, bandwidth) to an overloaded service; the return to a previous stable system configuration ("rollback") if the anomaly is suspected to be related to a recent change; the logical or physical isolation of equipment or a network segment suspected of being compromised or failing; the activation of in-depth diagnostic mechanisms to collect additional information about the anomaly.
[0078] This document also relates to a computer system for the detection of anomalies in data, the system comprising processing means configured to implement the aforementioned type of process.
[0079] The term "processing means" refers, in the context of the claimed computer system, to all the hardware and / or software resources within said system that are arranged and configured to execute the logical steps and operations defined by the anomaly detection process described above. These means are responsible for the actual execution of the calculations, comparisons, data manipulation, and decision-making inherent in the implementation of said process.
[0080] The processing means may include one or more hardware components designed to execute instructions and perform calculations. Non-limiting examples of such hardware components include: one or more central processing units (CPUs), which are general-purpose processors capable of executing a wide range of software instructions; one or more graphics processing units (GPUs), which are specialized in parallel computing and are often used to accelerate the training and inference of machine learning models; specialized processors for artificial intelligence, such as tensor processing units (TPUs) or neural network accelerators (NPUs); programmable logic circuits, such as field-programmable gate arrays (FPGAs), which can be configured to perform specific tasks very efficiently; application-specific integrated circuits (ASICs), custom-designed to perform a particular function, such as anomaly detection, with optimized performance or energy efficiency; digital signal processors (DSPs); as well as random access memory (RAM), cache memory and the interfaces needed to access stored data and communicate with other system components.
[0081] In addition to or as an alternative to hardware components, processing means may include software components that contain the instructions and logic to implement the process.
[0082] The processing capabilities can be implemented within a single physical machine (e.g., a server, a workstation, an intelligent network device) or distributed across multiple interconnected physical or virtual machines (e.g., in a cloud computing architecture, a computing cluster, an edge computing system). Regardless of the physical or logical implementation, the processing capabilities are specifically configured—that is, programmed or hardware-arranged—to execute the distinct steps of the anomaly detection process sequentially or in parallel.
[0083] This document also relates to a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the process of the type described above.
[0084] This document also relates to a computer-readable, non-transient storage medium storing the computer program product of the aforementioned type. Brief description of the drawings
[0085] Other features, details, and advantages will become apparent upon reading the detailed description below and analyzing the attached drawings, on which: [Fig. 1] schematically illustrates the main steps of the process, including the training of the two models and the detection phase by comparing their predictions; [Fig. 2] illustrates the principle of segment evaluation, showing how a time series with an anomaly is divided into segments and how the labels are adjusted for evaluation; [Fig. 3] represents the evolution of the F1-score as a function of a detection threshold for the proposed process, a hybrid approach, and a classical approach, on a given dataset; [Fig. 4] represents the evolution of the F-rate as a function of a detection threshold for the same approaches and dataset; [Fig. 5] represents the ROC curves for the different approaches on a given dataset; [Fig. 6] represents the relationship between the normalized detection delay (DR) and the false positive rate for the different approaches on a given dataset; [Fig. 7] schematically illustrates one form of implementation of the computer system according to this document. Description of the implementation methods
[0086] The drawings and description can not only serve to better explain the proposed technique, but also contribute to its definition, where applicable. The detailed description below, with reference to the attached figures, illustrates non-limiting embodiments of the invention.
[0087] The method described in this document aims to detect anomalies in time series data, particularly multivariate data, obtained from the monitoring of complex systems. It is illustrated in Figure 1 and comprises a training phase and a detection phase.
[0088] During the training phase, two distinct forecasting models are trained: A first forecasting model (NB Model) is trained (TRAIN1) exclusively on an initial dataset containing time segments considered normal. It thus learns to predict future evolution based on the "ideal" or expected behavior of the system. A second forecasting model (Model MB) is trained (TRAIN2) on a second dataset containing mixed time segments, including both normal and unlabeled abnormal data. It thus learns to predict future developments by taking into account a broader spectrum of behaviors observed in the past, including potentially abnormal dynamics.
[0089] Preferably, the two models share the same architecture (e.g., same layer types, same structural hyperparameters) so that the difference between their future predictions primarily reflects the impact of different training data. Linear architectures (such as DLinear), LSTM, RNN, or Transformers can be used.
[0090] During the detection phase, for a recent input data time segment that is obtained (OBT DATA): the first model (NB) generates a first forecast (P1) for a defined future horizon (PREV1), for example 96 time steps. the second model (MB) generates a second forecast (P2) for the same input segment and the same future horizon (PREV2).
[0091] The metric is then determined (DET_DIFF) by comparing P1 and P2.
[0092] More specifically, if the data is multivariate (i.e., a number d of variables for each data point), the detailed calculation of the metric proceeds as follows:
[0093] For each variable i (from 1 to d) and for each time step t of the forecast horizon (of size n, starting at time f), we calculate the elementary error e t L between the PI and P2 forecasts: e t l = |P4 - P2*|
[0094] To improve robustness, the horizon n can be divided into m slices of size. An aggregate error is calculated for each slice j (ranging from time f + j). at f + (j + 1) - 1, for example the average of the elementary errors in the slice.
[0095] The specific difference E for the variable i over the horizon f is then the maximum of these aggregated errors over the m slices:
[0096] Next, for each variable i, a score is calculated based on the percentile of its specific difference Ef t compared to a reference distribution {E k ] k historical differences for this variable. This score t may be the proportion of historical values less than or equal to Eft: 1 7" i . . : historical number Z dEt £ E i)
[0097] The final metric At is the average of these percentile scores across all d variables:
[0098] The use of percentiles ensures normalization and comparability between variables. Techniques such as histogram binning can be used to effectively manage reference distributions.
[0099] An anomaly is identified if the final metric At exceeds a predefined threshold (ANO_DET). This threshold can be determined, for example, by setting a high percentile (e.g., 99%) on the historical distribution of At scores, or by empirical optimization on a validation set to maximize a performance metric such as the F1 score or the F-rate.
[0100] An alert can then be lifted or mitigation action can be initiated in the event of detection or identification of an anomaly (RESP).
[0101] As a non-limiting example of implementation, the training of the first and second forecasting models can be configured with the following parameters. The length of the input sequence provided to the models can be set, for example, to 96 time steps. Similarly, the forecast horizon, i.e., the number of future time steps the models are trained to predict, can also be set to 96 time steps.
[0102] For the model architecture, an efficient linear model (e.g., DLinear) can be used. The DLinear model is notably known from Zeng, Ailing, et al. “Are transformers effective for time series forecasting?.” Proceedings of the AAAI conference on artificial intelligence. Vol. 37. No. 9. 2023 (See also https: / / doi.org / 10.48550 / arXiv.2205.13504).
[0103] Training can be performed using batch data, for example with a batch size of 32 samples, and the entire training dataset can be iterated over multiple times, for example, over 10 epochs. The cost function used to guide training can be the mean squared error, which measures the mean squared difference between the predicted and actual values (or target values, depending on the model). An optimizer such as the Adam algorithm can be used to adjust the model parameters to minimize this cost function.
[0104] To introduce non-linearity into the model (if necessary depending on the exact architecture), an activation function such as GELU (Gaussian Error Linear Unit) can be used. Finally, to prevent overfitting, a regularization technique like dropout can be applied during training, randomly deactivating a fraction of the neurons, for example, with a rate of 0.1, corresponding to 10% of the neurons being deactivated. These specific parameters represent an example of a configuration that has shown good results, but other values and configurations remain possible within the framework of this method.
[0105] To rigorously assess process performance, specific metrics adapted to time series and sequential anomalies are preferable to classic point-by-point classification metrics.
[0106] Instead of evaluating each time point individually, the time series (predicted and ground truth) is divided into segments of size k (e.g., k = size of the forecast horizon n). If a segment contains at least one actual anomaly point (ground truth), the entire segment is considered anomaly (label 1). If a segment contains at least one anomaly detection by the process, the entire segment is considered predicted anomaly. The metrics (Accuracy, Recall, F1 score) are then calculated on these segments.
[0107] Figure 2 illustrates this principle: a ground truth sequence with an anomaly is transformed into new labels where the segments containing the anomaly are marked 1. This approach mitigates the bias of the point-adjusted evaluation and better measures the system's ability to detect the presence of an anomaly over a relevant period.
[0108] To quantify the detection speed, for each anomaly i (starting at time Si and ending at time Ei), we calculate the time Ti of the first alarm triggered in the interval [Si, Ei]. If no alarm is triggered, then Ti = Ei.
[0109] The Response Time (RT) is the average delay normalized by the duration of the anomaly: N 1 y T t - S t DR = N Ei - Si i=l where N is the total number of anomalies. A low DR (close to 0) is desirable, indicating immediate detection.
[0110] For a measurement combining precision and low delay, the F-rate is used, defined as the harmonic mean of the precision and the term (1 - DR]: 2 • Precision • (1 — DR) F-rate = - — - — — — Precision + (1 — DR)
[0111] A high F-rate indicates both accurate and rapid detection.
[0112] Other metrics such as the F1 score, accuracy, recall, and ROC curves (for Receiver Operating Characteristic - True Positive Rate vs False Positive Rate) can also be used to characterize performance.
[0113] The process has been tested and validated on various datasets, including SMD (server metrics), PSM (anonymized server metrics), and 5G3E (complex 5G emulation data with thousands of variables and injected anomalies: CPU overload, bandwidth limitation, packet loss, link failure), as well as on public reference datasets from various domains such as SMAP (satellite monitoring data), SWAT (water treatment process monitoring data) and MSL (space telemetry data).
[0114] The tests typically compare three approaches: the proposed method (double model NB / MB compared, At score). A hybrid approach: averaging the scores from the aforementioned method and the classical approach. A classical approach: the NB model alone, whose forecast is compared to the future ground truth to calculate a forecast error used as an anomaly score (classical semi-supervised method). The score is calculated using the same refinements for a fair comparison.
[0115] Figures 3 and 4 illustrate the results for the F1 score and F-rate as a function of the detection threshold applied to the anomaly score. These graphs show that the proposed method (referenced DREAM) and the hybrid approach achieve higher peak F1 scores and F-rates than the classical approach over a wide range of thresholds, particularly on complex datasets such as 5G3E. Results tables (Figure 5) summarize the best values achieved and the average of the five best values.
[0116] Figure 6 illustrates the ROC curves obtained. These curves typically show that the proposed method and the hybrid method dominate the classical approach, offering a better true positive rate for a given false positive rate, especially in the region of low false positive rates, which is useful for limiting false alarms in production.
[0117] Figure 6 illustrates the relationship between the normalized detection delay (DR) and the false positive rate. It typically shows that for low and acceptable false positive rates, the proposed method achieves a significantly lower DR than the conventional approach. The hybrid approach can sometimes achieve an even lower DR, potentially at the cost of a slight increase in false positives compared to the proposed method alone. Analyses by anomaly type also demonstrate the effectiveness of the method across different scenarios (e.g., rapid detection of link failures, CPU overloads in 5G3E), including its ability to detect anomalies not seen during MB model training (although sometimes with a longer delay than for observed anomalies).
[0118] As mentioned, a hybrid approach can be implemented. In such an approach, the first metric, At, is calculated as described previously. Then, a second metric is calculated based on the error between the P1 prediction (of the NB model) and the future ground truth (thus requiring waiting for this data). The two metrics (or derived scores) are then combined (e.g., averaged) to obtain the final anomaly score. This approach loses the advantage of speed but can improve robustness against very new anomalies or confirm detections.
[0119] Following the identification of an anomaly (score exceeding the threshold), the system can trigger an alert or initiate an automatic mitigation action (RESP step in Figure 1).
[0120] Figure 7 schematically illustrates a computer system (COMP SYS) configured to implement the process described above. The system includes processing means (PROC). These processing means may include one or more processors (central processing unit (CPU), graphics processing unit (GPU), AI accelerator (TPU / NPU), programmable logic circuits (FPGA) or dedicated logic circuits (ASIC), as well as the necessary memory (RAM, cache) and interfaces.These processing methods are configured, for example by executing software instructions stored in non-volatile memory, to receive time series data as input (INPUT_DTA – for example, from network monitoring modules or device / server monitoring), execute the process steps (model training, segment acquisition, forecast generation, metric calculation according to the described modalities, comparison to the criterion, anomaly identification, and optionally the hybrid approach and triggering of alerts / actions), and generate an output result (OUTPUT), such as an anomaly indicator, an alert, or a mitigation command. The system can be implemented on a single machine or in a distributed manner.
Claims
DEMANDS
1. A computer-implemented method for detecting anomalies in data, the method comprising: - generate (PREV1), using a first trained forecasting model (TRAIN1) using a first training dataset containing normal data segments, a first forecast corresponding to an input data; - generate (PREV2), using a second trained forecasting model (TRAIN2) with a second training dataset comprising mixed data segments, where the mixed data segments include both normal data segments and unlabeled abnormal data segments, a second forecast corresponding to the input data; and - identify an anomaly associated with the input data when a metric (DET_DIFF) satisfies a predefined criterion, said metric being based on a comparison between the first forecast and the second forecast.
2. A method according to the preceding claim, wherein the first forecasting model and the second forecasting model are of the same type of architecture.
3. A method according to any one of the preceding claims, wherein the first forecasting model and / or the second forecasting model features a type of forecasting model architecture selected from the group consisting of Recurrent Neural Networks, RNNs, LSTM networks, Transformers, and Linear models.
4. A method according to any one of the preceding claims, wherein the data are multivariate and comprise several variables, and wherein the determination of the metric comprises: calculate, for each variable in said multivariate data, a variable-specific difference between the first and second predictions for said variable; and aggregate said specific differences to obtain the metric.
5. A method according to the preceding claim, wherein the aggregation of variable-specific differences comprises: determine, for each variable, a score based on a percentile of its variable-specific difference within a predefined or calculated error distribution; and calculate the average of the scores across the variables to obtain the metric.
6. A method according to claim 4 or 5, wherein the calculation of the variable-specific difference for a given variable further comprises: divide a forecast horizon for said variable into a plurality of slices; calculate a difference within each slice; and determine the variable-specific difference based on the differences calculated for the slices, optionally by selecting the maximum difference across the slices.
7. A method according to any one of the preceding claims, wherein the predefined criterion is a threshold value, an anomaly being identified when the metric exceeds said threshold value.
8. A method according to the preceding claim, wherein the threshold value is determined on the basis of an expected percentage of anomalies in the data or by evaluating multiple potential thresholds.
9. A method according to any one of the preceding claims, wherein the first and second forecasts are generated for a predefined forecast horizon.
10. A method according to any one of claims 4 to 9, wherein the data include metrics selected from the group consisting of: CPU load, network utilization, memory utilization, TCP / UDP statistic, packet loss rate, latency metric.
11. A method according to any one of the preceding claims, further comprising: generate, using the first forecasting model, a third forecast corresponding to the input data; and combine the first metric and a second metric to identify an anomaly, optionally by averaging anomaly scores derived from each metric, said second metric being based on a comparison between the third forecast and actual data corresponding to the time period of the third forecast.
12. A method according to any one of the preceding claims, further comprising triggering an alert or initiating a mitigation action (RESP) when an anomaly is identified.
13. Computer system (COMP_SYS) for detecting anomalies in data (INPUT_DTA), the system comprising processing means (PROC) configured to implement the method according to any one of the preceding claims.
14. Product computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the process according to any one of claims 1 to 12.
15. Computer-readable and non-transient storage medium storing the computer program product according to the preceding claim.
