Method for generating counterfactual data from real data

Abstract

The invention relates to a computer-implemented method for generating counterfactual data from real data composed of time series {X i } i=1, n representing data flows in a telecommunications network, the real data being representative of the telecommunications network, the method comprising the steps of: (a) training a contrastive autoencoder on real data, the autoencoder comprising an encoder and a decoder, the representation in the latent space comprising at least one mixing dimension and at least one separating dimension, (b) creating representations of counterfactual data from representations of the real data in the latent space, by modifying all or some of the separating dimensions, (c) generating counterfactual data from the representations of counterfactual data, characterized in that, in step (a), the cost function l tot used is of the type: l tot (X i , E, D, A) — sim(Xi,Xl) + λ m (l unif (Z i m ) + l regist (Z i m , A i )) +λ s (l unif (Z i s ) + l align (z i s , A i )), where - X i <sb / >is an input datum or observation, E is the encoder function, D is the decoder function, Z i is a representation of a datum or observation in the latent space, such that Z i = E(X i ), the latent space z at the output of the encoder being subdivided into two parts or two sets of dimensions, respectively denoted z m and z s , respectively forming the mixing and separating dimensions or variables, sim(X i , X l ) is a similarity penalty, - λ m and λ s are weights, l(z j m ) = -Sk(ζ1, ζ2), where Sk(ζ1, ζ2)} is the Sinkhorn distance between two sub-groups of the latent observations ζ, these observations ζ being subdivided into two sets ζ1, ζ2 of similar size, l align = Sk(ζ 1 A0 , ζ 2 A0 ) + Sk(ζ 1 A1 , ζ 2 A1 ), where ζ40 and ζ41} are the observations obtained from ζ and in the groups A = 0, namely the second group or negative group, and A = 1, namely the first group or positive group, where the ζ40 are randomly subdivided into two sets {ζ1 A0, ζ2 A0} of similar size and where the ζA1 are subdivided randomly into two sets {ζ1 A1, ζ2 A1} of similar size, l regist = Sk {ζA0, ζA1}.

Description

Description Title: Method for generating counterfactual data from real data

[0001] This document relates to a method for generating counterfactual data from real data, said real data being representative of a technical system. State of the art

[0002] Detecting bias in an artificial intelligence system is a crucial issue to ensure its acceptability and reliability. The term "bias" refers to an imbalance in the predictions generated by the system with respect to a variable determined as sensitive. The sensitive variable here separates the observations into two groups, one of which has potentially less accurate decisions than the other. In this context, it is essential to develop methods to identify, quantify, and correct these biases in order to preserve the accuracy or effectiveness of the decisions made by artificial intelligence systems.

[0003] It is common in practice to have training and testing datasets that exhibit imbalances with respect to sensitive variables. More subtly, new forms of bias can emerge both during the training phase and during subsequent decision-making when new data are incorporated. This underlines the need to be able to accurately and reliably assess the presence of bias, with the aim of certifying the conformity of algorithmic decisions.

[0004] It is common practice to have training and testing bases that are unbalanced with respect to sensitive variables. More subtly, new types of bias can appear both in training bases and in artificial intelligence decisions based on these bases when new data are used. Being able to quantify a bias and measure it accurately and reliably is then a major problem. Indeed, if the notion of bias can be defined mathematically using the notion of dependence in probability, providing a strategy to certify the presence or absence of bias in training bases and automatic decisions on test data is fundamental to proving the conformity of algorithmic decisions.

[0005] It therefore appears imperative to develop a new approach to assess biases towards sensitive variables, by integrating robustness characteristics with respect to the data used. This new approach aims in particular to provide more reliable and consistent bias measures, with a view to guaranteeing the certification and acceptability of artificial intelligence systems in various contexts.

[0006] This document proposes a process that is part of this approach.

[0007] There is therefore a need to define a new way of providing bias measurements in relation to a sensitive variable having robustness properties with respect to the data used, particularly with a view to their certification.

[0008] To overcome the instability of bias measurement tools with respect to a sensitive variable, one solution is to generate a counterfactual observation for each observation from which the bias will be measured. Since the sensitive variable divides all known observations into two groups, the counterfactual of an observation corresponds to what an equivalent observation would likely be in the other group. As shown in the document “De Lara L., González-Sanz A., Asher N., Risser L. & Loubes JM (2021). Transport-based counterfactual models. https: / / arxiv.org / abs / 2108.13025”, using a bias measurement metric on all observations as well as their counterfactual observation makes it possible to stabilize and increase the robustness of the bias measurement.

[0009] Comparing an individual observation to its counterfactual observation also helps explain what criteria distinguish this observation from an observation in the other group. If this group is the prediction of an automatic classifier on technical data, such as time series from sensors, this helps explain what criteria the classifier uses to make its decisions. If the person comparing the observations and their counterfactual has expertise on the data, this can also help validate that the decisions are made for good reasons.

[0010] Computing the counterfactual of complex observations, such as those of time series from sensors, remains a complex task because only a part of the data structure can distinguish two groups of observations. Similarly, this discriminating structure may not be represented by a directly observed variable but only be inferred using a non-linear combination of the observed variables. This is called a latent variable.

[0011] The objective of this document is specifically to calculate the counterfactual of complex observations, in the sense that the sensitive variable is not directly expressed by an observed variable. Disclosure of the invention

[0012] To this end, the present document relates to a computer-implemented method for generating counterfactual data from real data composed of time series {ÿ ÿ} ÿ=1,&,^representing data flows in a telecommunications network, said real data being representative of said telecommunications network, said method comprising the steps of: (a) training a contrastive autoencoder from real data belonging respectively to a first group or positive group, and to a second group or negative group, said autoencoder comprising an encoder capable of generating a representation of data in a latent space and a decoder capable of generating output data from said representation in the latent space, said representation in the latent space comprising at least one so-called mixing dimension or variable representing characteristics extracted from the data which are as independent as possible of the group to which said data belong, and at least one so-called separating dimension or variable, representing characteristics extracted from the data which are as dependent as possible on the group to which said data belong, (b) creating counterfactual data representations from the representations of the real data in the latent space, obtained using the encoder of the trained contrastive autoencoder, by modifying all or part of the separating dimensions of said representations of the real data, (c) generating counterfactual data from said counterfactual data representations, using the decoder of the trained contrastive autoencoder characterized in that, during step (a), the cost function ^^^^ used is of the type: - ÿ ÿis an input data or observation, ^ is the encoder function, ÿ is the decoder function, ÿÿ is a representation of a data or observation in the latent space, so thatÿÿ = ^(ÿÿ) , the latent space ÿ at the output of the encoder being subdivided into two parts or two sets of dimensions, noted respectively ÿ ^ and ÿ ^ , respectively forming the mixing and separating dimensions or variables,2 ýÿ^(ÿÿ , ÿ^ÿ ) is a similarity penalty, is the Sinkhorn distance between two subsets of the latent observations ý , these observations ý being subdivided into two sets similar, - are the observations from ý in the groups ý = 0 , namely the second or negative group, and ý = 1, namely the first or positive group, where the ý ý0 are randomly subdivided into two sets {ý1 ý0 , ý2 ý0} of similar size and where the ýý1 are randomly subdivided into two sets {ý1 ý1 , ý2 ý1} of similar size, - ^ ÿÿ^ÿ^^ = ÿ^ ( ý ý0 , ý ý1) .

[0013] Real data refers to an observation that was actually collected. It is authentic data from a concrete source and reflects real events, behaviors, or phenomena. This real data can include a variety of data types, such as images, texts, time series, audio recordings, other types of sensor data, etc.

[0014] A counterfactual of a technical system refers to a hypothetical observation that represents an alternative reality or scenario for that system. In other words In other words, it is data that describes how the system would have reacted or functioned differently if particular conditions had been changed.

[0015] In the context of a technical system, counterfactual data is used to understand how the system would have reacted if key variables or parameters had been adjusted. This can be useful for assessing the impact of different changes on the system's operation, predicting potential outcomes, or analyzing cause-and-effect relationships.

[0016] For example, consider an industrial furnace equipped with a temperature control system. Data on furnace temperatures as well as furnace control signals can be collected. A counterfactual in this context could be a hypothetical observation that shows how furnace temperatures would have changed if the control signal had been adjusted in a specific way. This allows one to evaluate how different control strategies can influence the system's behavior.

[0017] In summary, counterfactual data for a technical system are simulated or hypothetical data that help explore potential outcomes by changing key variables in the system while keeping the rest of the environment constant. This can be a useful approach for making informed decisions about adjustments or improvements to the technical system.

[0018] The resulting counterfactual data can be used to rebalance data distributions in the event of bias. In particular, if a particular group is underrepresented, counterfactual examples for that group can be created to increase its representation in the dataset.

[0019] A technical system is an interconnected set of components, devices or processes, whether mechanical, electronic, electrical, computer or other, working together to perform a specific function. Said technical system may be equipped with sensors. Said technical system is capable of providing information or a signal capable of providing information on the state of said technical system.

[0020] A non-exhaustive list of physical quantities or properties that can be measured by sensors in technical systems, as well as output information that can be provided by different technical systems, is defined below as examples: - Mechanical Systems: Position (encoders, position sensors), Speed ​​(speed sensors, revolution counters), Acceleration (accelerometers), Force (force sensors, dynamometers), Pressure (manometers, pressure sensors), Strain (strain sensors), Vibration (vibration sensors), Temperature (thermometers), Rotation (rotation sensors), Level (level sensors) - Electronic or Electrical Systems: Voltage (voltmeters), Current (ammeters), Resistance (ohmmeters), Frequency (frequency meters), Power (wattmeters), Power factor (power factor indicators), Frequency response curve (spectrum analyzers), Ripple (oscilloscopes), Threshold voltage (threshold detectors), Light intensity (light meters) - Computer Systems or Software: Data (data flow), Status (system state, errors), Execution time (timestamps), CPU usage (performance monitors), Memory used (memory monitors), Latency (latency measurements), Network throughput (traffic analyzers), Events (event logs) - Environmental Systems: Ambient temperature (thermometers), Humidity (hygrometers), Atmospheric pressure (barometers), Sound level (sound level meters), Air quality (air quality sensors), Solar radiation (pyranometers), Wind speed (anemometers), Precipitation (rain gauges) - Control and Automation Systems: Commands (control signals), States (system states), Alarms (fault alerts), Errors (error messages), Feedback signals (system feedback),Switching states (states of switches, relays), Activation states (states of motors, actuators).

[0021] A contrastive autoencoder, also known as a contrastive pair autoencoder, is a variant of a classic autoencoder that aims to learn useful representations by focusing on differences and similarities between input data. This type of autoencoder is typically used to learn discriminative features for classification, dimension reduction, or anomaly detection.

[0022] The contrastive approach in a contrastive autoencoder is to encourage the model to capture the relevant information to distinguish between different pairs of training samples. Rather than simply minimizing the reconstruction distance between the input and output (as in a traditional autoencoder), the contrastive autoencoder maximizes the similarity between pairs of positive examples (which are expected to be similar) and minimizes the similarity between pairs of negative examples (which are expected to be different).

[0023] In other words, the contrastive autoencoder attempts to project similar examples close together into the latent representation space (or latent space), while moving dissimilar examples further apart. This allows the model to learn representations that capture meaningful variations and differences between examples.

[0024] The contrastive autoencoder is notably known from the document “Contrastive Variational Autoencoder Enhances Salient Features, Abubakar Abid and James Zou, 2019, arXiv 1902.04601”.

[0025] The architecture and / or hyperparameters of the encoder and decoder can be conventional and adapted to the type and dimensions of the input data. Similarly, the number of dimensions of the latent space can be adapted to the use case.

[0026] Said method may comprise, before step (a), a preliminary step of measuring real data using at least one sensor equipping said telecommunications network.

[0027] Said method may comprise a subsequent step of analyzing the real data and the counterfactual data, then a step of correcting operating parameters of the technical system, based on said analysis.

[0028] The autoencoder can be a variational autoencoder. Such an autoencoder is notably known from the document "Diederik P. Kingma and Max Welling, Auto-Encoding Variational Bayes, arXiv:1312.6114".

[0029] When training the autoencoder, the cost function, also called the error or penalty function, is classically used to measure the difference between the input data and the output data reconstructed by the model, for a training dataset.

[0030] The main objective of training is to minimize this cost function, which means that the neural network seeks to adjust its parameters in such a way as to reduce the errors between predictions and target values. Optimization consists of finding parameter values ​​that minimize the cost function, which leads to more accurate predictions that are more faithful to the training data. and ÿ ^can be defined by a user.

[0032] ýÿ^(ÿ ÿ , ÿ ^ ÿ ) can be equal to 6ÿ ÿ 2 Other classical similarity functions can be used.

[0033] The penalty ^ ^^ÿ^ tends to scatter the data. The penalty ^ ÿ^ÿ^^ tends to independently group the data of the first group together, and the data of the second group together. The penalty ^ ÿÿ^ÿ^^ tends to mix data from the two groups.

[0034] The Sinkhorn distance can be calculated as follows.

[0035] Let ^, ^ * ÿ(= þ )probability measures and let ÿ >0, then the entropy-regularized version of the optimal transport problem is defined as follows:

[0036] Ce problème admet une formulation duale ÿ ÷(^, ^) =

[0037] When the probabilities are discrete, there exists an algorithm in ÿ(^ 2 log(^)) to calculate the Sinkhorn transport plan ÿ = ÿ ÿ,^ and therefore the Sinkhorn distance which is described in the document “M. Cuturi, Sinkhorn distances: Lightspeed computation of optimal transport, Advances in neural information processing systems, 26, 2013.”

[0038] In step (b), the dimensions or latent variables of a representation ÿ ÿ of a data or observation in the latent space being separated into two groups ÿ ÿ ^ and ÿ ÿ ^ , respectively forming the mixing and separating dimensions or variables of ÿ ÿ , the corresponding counterfactual data representation ÿ^ can be obtained by concatenating the groups ÿ^ ^ et ÿ ^ ^ , respectively forming the mixing and separating dimensions or variables of ÿ ÿ , where ÿ ÿ ^ =ÿ^ ^ and where ÿ^ ^ = ýý(ÿ^ ÿ ), ÿ^ ^ being the separating part of the latent space representing an observation ^matched to ÿ using the function ý ý , that is to say minimizing the Sinkhorn distance between two point clouds from groups ý = 0 , namely the second or negative group, and ý = 1, namely the first or positive group.

[0039] In particular, in order to calculate ÿ ^ ^ = ý ý (ÿ ÿ ^ ), consider the separating part ÿ ^ ^ of the latent space of a set of observations ^ = 1, & obtained using the encoder ^. Let us also consider that the category ý of each observation ^ is known. The observations ^ = 1, & , ÿ will have been selected so as to contain a sufficiently large number of observations in each class ý = 0 and ý = 1 (typically > 50 observations per class). The matching of the observations of {ÿÿ ^ , {ÿ ^ ^} ^=1,&,ÿ} from the groups ý = 0 and ý = 1 is then performed by minimizing the Sinkhorn distance between these two point clouds. Once this matching is done, the ÿ ^ ^ will be the {ÿ ^ ^} ^=1,&,ÿ estimated to be closest to ÿ ÿ ^ by matching in group ý different from that of observation ÿ.

[0040] In the context of this document, the technical system may be a telecommunications network. In this case, the actual data may consist of time series representing data flows in a network of

[0041] Network flow monitoring is the collection, analysis, and monitoring of traffic passing through a given network or network segment.

[0042] Such monitoring makes it possible to obtain data from network flows. This data can be numerous and can include data from various categories, such as the volume of information exchanged, connections made, incoming connections, outgoing connections, port numbers for source and destination endpoints, protocols, processor activities, memory access activities, data concerning the IP addresses of the connections, processor temperature, volumes of data exchanged, etc. This data can therefore be multi-categorical (binary, continuous, ordinal, etc.) and is all time-stamped.

[0043] First, a learning base can be created where the data are represented as series of time events using timestamps. Thus we obtain learning bases of time series that are observed. For each of these data ÿ ÿ , a binary variable (ý ÿ = 0) or (ý ÿ = 1) can indicate whether ÿ ÿ is linked or not to an attack.

[0044] In a second step, the contrastive autoencoder can help to find a compact representation space. ÿ of each ÿ ÿ . We speak of latent space to designate this space. For example ÿ ÿ will be able to contain a time series of 128 time steps will be able to represent 5 characteristics extracted from ÿ ÿ . By the very principle of these auto-encoders, this compact representation of ÿ ÿ will be particularly informative in the sense that it contains enough information to reconstruct a faithful approximation of ÿ ÿ .

[0045] Le but de la fonction de coût précitée est alors de séparer en deux la représentation latenteÿ by distinguishing characteristics ^ÿ common to all ÿÿ , here the characteristics ÿÿ , and the characteristics ÿ ÿ ^ which make it possible to distinguish between an attack context and a non-attack context. The method according to this document then makes it possible to transform network flow signals into normal signals and thus understand, by comparing them with the initial signals, why an attack was detected. It is thus possible to obtain both a detector and a signal creator who explains what should have changed for the signals to be classified as normal.

[0046] The invention also relates to a computer program comprising instructions for implementing the method of the aforementioned type, when this program is executed by a processor.

[0047] The invention also relates to a non-transitory computer-readable recording medium on which is recorded a program for implementing the method of the aforementioned type, when this program is executed by a processor.

[0048] The invention also relates to a computer system comprising - an input interface for receiving data composed of time series representing data flows in a telecommunications network, - a memory for storing at least the instructions of a computer program of the aforementioned type, - a processor accessing the memory to read said instructions and then execute the method of the aforementioned type, - an output interface for providing counterfactual data generated during step (c). Description of the figures

[0049] Other characteristics, details and advantages will appear on reading the detailed description below, and on analyzing the attached drawing showing various figures, in which: [Fig. 1] schematically represents an example of a computer system according to the present invention, [Fig. 2] illustrates the training of the contrastive autoencoder, [Fig. 3] illustrates the generation of the counterfactual data using the trained contrastive autoencoder, Detailed description of at least one embodiment

[0050] Figure 1 schematically represents a computer system 1 comprising - an input interface 2 for receiving data composed of time series {ÿ ÿ} ÿ=1,&,^representing data flows in a telecommunications network, - a memory 3 for storing at least the instructions of a computer program according to this document, - a processor 4 accessing the memory 3 to read said instructions and then execute the method according to this document, - an output interface 5 for providing counterfactual data.

[0051] In the remainder of the description, the following notations will be used: - ÿ : A set of input observations. Each input observation can be, for example, a set of tables, images, or curves. - ý * {0,1} : Group of input observations of ÿ. By convention ý ÿ = 0 denotes that observation i is in a potentially discriminated group, and ý ÿ= 1 denotes the rest of the observations. - ^: Encoder part of the contrastive autoencoder. - ÿ: Decoder part of the contrastive autoencoder. - ÿ: Latent representation of the data in the contrastive autoencoder. We thus have ÿ = ^(ÿ) - The latent space ÿ at the output of the encoder ÿ is subdivided into two parts, ÿ ^ and ÿ ^ . The latent variables of ÿ ^ represent features extracted from ÿ that are as independent as possible of group ý. The latent variables of ÿ ^ represent features extracted from ÿ that are as dependent as possible on the group ý . These properties of ÿ ^ and ÿ ^ will be favored by the use of penalties or cost functions during learning. - We note ý ý (ÿ ^ ) the counterfactual of ÿ ^ with respect to the binary variable ý . If for an observation ÿ , ý ÿ = 0, then ý ý ( ÿ ÿ ^)is equal to ÿ ^ ^ , where ^ is an observation such that ý ^ = 1. The matching between the observations of group ý = 0 and ý = 1 will be done by optimal transport as described below. -ÿ^: Reconstruction of the data by the contrastive autoencoder. We thus have ÿ^ = ÿ(ÿ) =ÿ (^(ÿ)). - Learning the contrastive autoencoder (AEC)

[0052] Learning AEC differs from learning a classical autoencoder by using penalties or cost functions that will give the latent space properties that favor the computation of counterfactuals. This learning step is illustrated in Figure 2.

[0053] Note that, during learning, two weights and ÿ ^ can be set by the user.

[0054] During a learning step, let an observation ÿ come from a mini-batch of a training data set, then:

[0055] We first calculate ÿ ÿ = ^(ÿ ÿ ). The latent variables of ÿ ÿ are then separated into two groups ÿ ^ and ÿ ^ . We then calculate the predictions = ÿ ( ÿ ÿ ) .

[0056] The total prediction penalty is then determined as follows: 2 where ýÿ^(ÿ ÿ , ÿ ^ ÿ ) is a common similarity penalty between ÿ ÿ and ÿ ^ ÿ (For example || 2), and other penalties ^ ^^ÿ^ , ^ ÿÿ^ÿ^^ and ^ ÿ^ÿ^^ are described below.

[0057] The gradient of ^ ^^^ ( ÿ ÿ , ^, ÿ, ý ) with respect to the parameters of ^ and ÿ is then calculated by back-propagation. The parameters of ^ and ÿ are finally updated.

[0058] The purpose of the penalties on ÿ ^is to mix the latent representation of observations in the groups ý = 0 and ý = 1 while promoting a general spread of the data. The purpose of the penalties on ÿ ^ is to independently group the latent representation of observations into each of the groups ý = 0 and ý = 1, while promoting a general spread of the data. - Penalties used during learning

[0059] Let ÿ^(ý1, ý2) be the Sinkhorn distance between two subgroups of the latent observations ý.

[0060] Based on this distance, it is possible to define three types of penalties that apply to subsets of the autoencoder's latent variables. - The penalty ^ ^^ÿ^

[0061] The observations ý are randomly subdivided into two sets of similar size. Alors :

[0062] This penalty tends to scatter the data. - The penalty ^ ÿ^ÿ^^

[0063] Let ý ý0 and ý ý1 the observations from ý and in the groups ý = 0 and ý = 1 , respectively.

[0064] The ý ý0 are randomly subdivided into two sets {ý1 ý0 , ý2 ý0} of similar size.

[0065] The ý ý1 are randomly subdivided into two sets {ý1 ý1 , ý2 ý1} of similar size.

[0066] Alors :

[0067] This penalty tends to independently group the data of group ý = 0 together, and those of group ý = 1 together. - The penalty ^ ÿÿ^ÿ^^

[0068] We call ý ý0 and ý ý1 the observations from ý and in the groups ý = 0 and ý = 1 , respectively.

[0069] Alors : ^ ý0 ÿ^ÿ^^ = ÿ^(· , ·ý1)

[0070] This penalty tends to mix the data of group ý = 0 and those of group ý = 1. - Estimation of a counterfactual observation

[0071] A match between latent observations ÿ ^ of the group ý = 0 and of the group ý = 1 is carried out as described below. The notation ÿ ^ ^ = ý ý (ÿ ÿ ^ ) means that ÿ ÿ ^ and ÿ ^ ^ are different groups, and that ÿ ^ ^ is the observation paired with ÿ ÿ ^ .

[0072] The different stages of this calculation are illustrated in Figure 3.

[0073] The calculation of counterfactual observations between groups ý = 0 and ý = 1 is done by optimal transport on ÿ ^ which is low-dimensional compared to ÿ.

[0074] The calculus of counterfactual observation ÿ ÿ ý to observation ÿ is carried out in the following way, from the trained autoencoder.

[0075] On détermine ÿÿ = ^(ÿÿ). Les variables latentes de ÿÿ sont ensuite séparées en deux groupes ÿ^ ÿ et ÿ^ ÿ . ýý(ÿ^ÿ ) is then determined by calculation and then ÿ^ÿ and ýý(ÿ^ÿ ) are concatenated. The resulting counterfactual latent observation is denoted ÿ ÿ ý . Finally, we determine ÿ ÿ ý = ÿ(ÿ ÿ ý ).

[0076] The notation ÿ ^ ^ = ý ý (ÿ ÿ ^ ) means that ÿ ÿ ^ and ÿ ^ ^ are different groups, and that ÿ ^ ^ is the observation paired with ÿ ÿ ^ . The matching is the one that minimizes ÿ^(ÿ ý0 , ÿ ý1 ). The details of its calculation are described in the document “M. Cuturi, Sinkhorn distances: Lightspeed computation of optimal transport, Advances in neural information processing systems, 26, 2013.” - Application to the field of cybersecurity

[0077] In a cybersecurity context for example, consider a set of observations composed of time series representing data streams in a telecommunications network. Each of them has a constant number of time steps and has been labeled as either normal ( ý ÿ = 0), or as at risk ( ý ÿ = 1 ). This labeling may correspond to the output of a classifier pre-trained to detect risky flows. It may also have been carried out by an expert. The reasons for belonging to the normal or risky group are assumed to be unknown. If a classifier was used, it may be based on a model such as a neural network or random forest, for example, whose internal mechanisms are then difficult to explain if we refer to the algorithmic processing of the data. If an expert has labeled the series, he may also no longer be available to explain his choices.

[0078] The purpose of the method described above is to enable a user to be informed of the reasons why a specific time series is at risk. Consider the series ÿ ÿ as being at risk, that is, such that ý ÿ = 1. We assume that this series is due to a mixture of normal criteria (e.g., traffic variations due to schedules over a day) and risky criteria (e.g., sudden overactivity due to an attack). In order to understand the reasons why is at risk, the contrastive autoencoder can be used to generate a counterfactual series ÿý ÿ which is in all respects similar to ÿ ÿ on criteria not related to risk, but different from ÿ ÿ on the criteria related to this risk. The series ÿ ÿ ý can then be interpreted as what would have been ÿ ÿ if there was no risk. Compare visually ÿ and ÿ ÿ ýcan then allow a user to interpret the risk detection rules (which are unknown) in this specific case.

[0079] Generating counterfactual data allows the use of classic bias measurement tools from the literature on raw data and counterfactual data.

[0080] Of course, this same process can be used in other technical fields.

Claims

Claims

1. A computer-implemented method for generating counterfactual data from real data composed of time series representing data flows in a telecommunications network, said real data being representative of said telecommunications network, said method comprising the steps of: (a) training a contrastive autoencoder from real data belonging respectively to a first group or positive group, and to a second group or negative group, said autoencoder comprising an encoder capable of generating a representation of data in a latent space and a decoder capable of generating output data from said representation in the latent space, said representation in the latent space comprising at least one so-called mixing dimension or variable representing characteristics extracted from the data which are as independent as possible of the group to which said data belong, and at least one so-called separating dimension or variable,representing features extracted from the data that are as dependent as possible on the group to which said data belong, (b) creating counterfactual data representations from the representations of the real data in the latent space, obtained using the encoder of the trained contrastive autoencoder, by modifying all or part of the separating dimensions of said representations of the real data, (c) generating counterfactual data from said counterfactual data representations, using the decoder of the trained contrastive autoencoder,characterized in that, during step (a), the cost function ^^^^ used is of the type:, - ÿ ÿis an input data or observation, ^ is the encoder function, ÿ and the decoder function, ÿÿ is a representation of a data or observation in the latent space, so thatÿÿ = ^(ÿÿ) , the latent space ÿ at the output of the encoder being subdivided into two parts or two sets of dimensions, noted respectively ÿ ^ and ÿ ^ , respectively forming the mixing and separating dimensions or variables,2 ýÿ^(ÿÿ , ÿ^ÿ ) is a similarity penalty, where ÿ^ ( ý1, ý2 ) is the Sinkhorn distance between two subsets of the latent observations ý , these observations ý being subdivided into two sets similar, - are the observations from ý and in the groups ý = 0, namely the second or negative group, and ý = 1, namely the first group or positive group, where the ý ý0 are randomly subdivided into two sets {ý1 ý0, ý2 ý0} of similar size and where the ý ý1 are randomly subdivided into two sets {ý1 ý1 , ý2 ý1} of similar size, - ^ ÿÿ^ÿ^^ = ÿ^(ý ý0 , ý ý1 ). .

2. Method according to the preceding claim, comprising, before step (a), a prior step of measuring the real data using at least one sensor equipping said telecommunications network.

3. Method according to one of the preceding claims, in which, during step (b), the dimensions or latent variables of a representation ÿ ÿ of a data or observation in the latent space being separated into two groups ÿ ÿ ^ and ÿ ÿ ^ , respectively forming the mixing and separating dimensions or variables of ÿ ÿ , the counterfactual data representation ÿ ^ corresponding is obtained by concatenation of the groups ÿ ^ ^ and ÿ^ ^ , respectively forming the mixing and separating dimensions or variables of ÿ ÿ , where ÿ ÿ ^ = ÿ ^ ^ and where ÿ ^ ^ = ý ý (ÿ ÿ ^ ), ÿ ^ ^ being the separating part of the latent space representing an observation ^ matched to ÿ using the function ý ý, i.e. minimizing the Sinkhorn distance between two point clouds from the groups ý = 0, namely the second or negative group, and ý = 1, namely the first or positive group.

4. Computer program comprising instructions for implementing the method according to one of the preceding claims, when this program is executed by a processor.

5. Non-transitory computer-readable recording medium on which is recorded a program for implementing the method according to one of claims 1 to 3, when this program is executed by a processor.

6. Computer system (1) comprising - an input interface (2) for receiving data composed of time series representing data flows in a telecommunications network, - a memory (3) for storing at least the instructions of a computer program according to claim 5, - a processor (4) accessing the memory (3) to read said instructions and then execute the method according to one of claims 1 to 3, - an output interface (5) for providing counterfactual data generated during step (c) of the method according to one of claims 1 to 3.

Images