Abnormal traceability method and system for digital service network

Abstract

The invention discloses an anomaly tracing method and system for a digital service network, and the method comprises the following steps: carrying out the coding of normal operation data as first reference data based on causal priori, obtaining a corresponding first potential representation which has a plurality of first dimensions, and enabling the first dimensions to be in one-to-one correspondence with causal factors; sequentially intervening the values of the first dimensions in the first potential characterization to generate corresponding intervention characterization; each causal factor has a plurality of intervention representations corresponding to the causal factor; reconstructing each intervention representation to obtain corresponding intervention data; calculating the similarity between the intervention data and the abnormal operation data serving as the target data; and based on the similarity, screening out corresponding candidate data from the intervention data, and generating a corresponding anomaly traceability result based on the causal factors corresponding to the candidate data. According to the method, the abnormity of the digital service network with the complex causal relationship can be traced.

Description

Technical Field

[0001] The present invention relates to the field of data analysis, and in particular to a method and system for tracing the anomaly of a digital service network. Background Art

[0002] Driven by the new generation of information technology, digital services are continuously evolving towards the multi-dimensional cloud, edge, and end. Digital services are evolving from the traditional isolated form to a cloud-edge-end integrated digital service network form that is cross-CPH three-dimensional space (Cyberspace-Physical-Human) and heterogeneous service collaboration enhanced, showing the characteristics of cross-network and cross-domain, large-scale, complex, trustworthy, and dynamic. By sharing and coordinating cross-network and cross-domain digital services, it is possible to promote the complex service integration of human, machine, and material resources in the physical space and software services in the information space.

[0003] With the rapid development of digital service networks, their architecture is becoming increasingly complex, their scale is constantly expanding, and the dependencies between services are becoming more intricate. In this context, how to efficiently detect anomalies in the service network and accurately trace and locate them has become a key issue to ensure the reliability of the service network and improve user satisfaction.

[0004] Existing anomaly detection and tracing methods mainly rely on monitoring data, log information, statistical analysis and other technical means. However, these methods often have problems such as insufficient accuracy and limited applicability when faced with complex causal relationships and high-dimensional data. Summary of the invention

[0005] In view of the shortcoming of the prior art that it is difficult to trace the anomalies of a digital service network with complex causal relationships, the present invention provides an anomaly tracing technology for a digital service network based on causal decoupling learning.

[0006] In order to solve the above technical problems, the present invention is solved by the following technical solutions: In a first aspect, one or more embodiments of this specification provide a method for tracing the source of anomalies in a digital service network, for tracing the source of anomalies in a digital service network, the method comprising the following steps: Acquire first reference data and target data corresponding to the anomaly to be traced, wherein the target data is the abnormal operation data corresponding to the anomaly to be traced, and the first reference data is the normal operation data within a preset time range before the anomaly to be traced occurs; The first reference data is encoded based on a causal prior to obtain a corresponding first potential representation, wherein the first potential representation has a plurality of first dimensions, and the first dimensions correspond to causal factors one by one; the causal prior is a prior distribution generated based on a causal structure graph corresponding to the digital service network; the causal structure graph is used to indicate the causal factors corresponding to the digital service network, and the relationship between the causal factors; The values ​​of the first dimensions in the first potential representation are intervened in turn to generate the exogenous variable ϵ into corresponding intervention representations, and each causal factor has at least one intervention representation corresponding to it; Reconstruct each intervention representation to obtain corresponding intervention data; Calculating the similarity between the intervention data and the target data; Based on the similarity, corresponding candidate data are screened out from the intervention data, and corresponding abnormality tracing results are generated based on the causal factors corresponding to the candidate data.

[0007] In one or more embodiments of the present specification, the first dimension to be intervened is used as the target dimension, and the method of intervening the value of the target dimension in the first potential representation is: The outlier samples corresponding to the target dimension are used as intervention values; Assigning values ​​to the target dimensions based on the intervention values ​​to obtain intervention representations that correspond one to one with the intervention values; The outlier samples include simulated outliers, and the steps for obtaining the simulated outliers are: Acquire a plurality of second reference data, where the second reference data are all normal operating data within a preset time range before the abnormality to be traced occurs; Encoding the second reference data based on the causal prior to obtain a corresponding second latent representation, wherein the second latent representation has a plurality of first dimensions, and the first dimensions correspond to the causal factors one by one; Statistics are performed on the values ​​of each first dimension in the second potential representation, and at least one simulated abnormal value corresponding to the first dimension is generated based on the statistical result.

[0008] In one or more embodiments of this specification, the outlier sample also includes historical outliers, and the historical outliers are obtained in the following manner: Encoding the abnormal operation data of the determined root cause to obtain a corresponding third potential representation, wherein the third potential representation has a plurality of first dimensions, and the first dimensions correspond to the causal factors one by one; Based on the root cause, the value corresponding to the first dimension in the third potential representation is used as the corresponding historical abnormal value.

[0009] In one or more embodiments of this specification: The first N intervention data whose similarity reaches the preset similarity threshold are taken as candidate data, N≥1; When there is no candidate data, the root cause to be processed is output as others; When candidate data exists, the causal factors corresponding to the candidate data are output as the root causes to be processed.

[0010] In a second aspect, one or more embodiments of this specification provide a digital service network anomaly tracing system for tracing anomalies in a digital service network, including: A data extraction module, used to obtain first reference data and target data corresponding to the anomaly to be traced, wherein the target data is the abnormal operation data corresponding to the anomaly to be traced, and the first reference data is the normal operation data within a preset time range before the anomaly to be traced occurs; an encoder, configured to encode the first reference data based on a causal prior to obtain a corresponding first latent representation, wherein the first latent representation has a plurality of first dimensions, and the first dimensions correspond one-to-one to causal factors; the causal prior is a prior distribution generated based on a causal structure graph corresponding to the digital service network; the causal structure graph is used to indicate the causal factors corresponding to the digital service network, and the relationship between the causal factors; An intervention module, used for intervening the values ​​of each first dimension in the first potential representation to generate a corresponding intervention representation, wherein each causal factor has at least one intervention representation corresponding thereto; A generator, used to reconstruct the intervention representation and obtain the corresponding intervention data; A calculation module, used for calculating the similarity between the intervention data and the target data; The analysis module is used to screen out corresponding candidate data from the intervention data based on the similarity, and generate corresponding abnormality tracing results based on the causal factors corresponding to the candidate data.

[0011] In a third aspect, one or more embodiments of this specification provide a method for training an anomaly tracing model, comprising the following steps: Obtaining operational data of the digital service network during normal operation as sample data; Based on the sample data, the discriminator and the prior model, the encoder and the generator are alternately trained adversarially, and the trained encoder and generator are used as an anomaly tracing model; in: Building a priori model based on the causal structure diagram corresponding to the digital service network, wherein the input of the priori model is an exogenous variable and the output is a corresponding causal prior; The causal structure diagram is used to indicate the causal factors corresponding to the digital service network and the relationship between the causal factors; The exogenous variable is a random variable that obeys a standard normal distribution; The steps of conducting adversarial training on the discriminator are as follows: Construct discriminative training data; The discriminator determines the authenticity of the discriminant training data, and adjusts the model parameters of the discriminator based on the discrimination result; The method for constructing the discriminant training data is: Encode the sample data based on the encoder and use the obtained latent variable as a first latent variable; construct discriminant training data based on the first latent variable and the sample data; A causal prior is generated based on the prior model, and the causal prior is used as a second latent variable; virtual operation data corresponding to the second latent variable is generated by the generator, and discriminant training data is constructed based on the corresponding second latent variable and the virtual operation data.

[0012] In one or more embodiments of this specification, the expression of the prior model is:

[0013] in: Represents the prior model, which is used to generate exogenous variables The corresponding causal prior; I represents the identity matrix; A The adjacency matrix corresponding to the causal structure graph representing the digital service network.

[0014] In one or more embodiments of this specification, the steps of performing adversarial training on the prior model, the encoder, and the generator are: generating a causal prior based on the prior model; The encoder encodes the sample data based on the causal prior and uses the obtained latent variable as a third latent variable; Generate reconstructed sample data corresponding to the third latent variable by the generator; Calculating a first joint distribution corresponding to the encoder based on the corresponding sample data and a third latent variable; Calculate a second joint distribution corresponding to the generator based on the corresponding third latent variable and the reconstructed sample data; The KL divergence of the first joint distribution and the second joint distribution is calculated, and model parameters of the prior model, the encoder, and the generator are adjusted based on the KL divergence.

[0015] In one or more embodiments of this specification: Obtain the observable label corresponding to the sample data as the sample label; In the adversarial training step of the prior model, encoder and generator, the supervision loss corresponding to the sample label is also calculated; The model parameters of the prior model, the encoder, and the generator are adjusted based on the KL divergence and the supervised loss.

[0016] In one or more embodiments of this specification: Determine a total loss based on the KL divergence and the supervised loss, and adjust model parameters of the prior model, the encoder, and the generator based on the total loss; The loss function for calculating the total loss is:

[0017] in: L(E,G,F) represents the total loss; represents the first joint distribution q E (x,z) and the second joint distribution KL divergence of q E ( x,z ) is used to characterize the encoder E For sample data x and the third latent variable z The joint posterior distribution of Used to characterize the joint distribution of the generator; λ is the balance coefficient; L sup (E) is the supervised regularization term with sample labels as supervisory signals, {L}_{sup}\left ( {E} \right )={\mathbb{E}}_{x,y}\left [ {{l}_{s}\left ( {E;x,y} \right )} \right ] , is the sample label y The corresponding supervision loss is Express expectations.

[0018] In a fourth aspect, one or more embodiments of the present specification provide a training system for an anomaly tracing model, which is used to execute any of the training methods for an anomaly tracing model described above.

[0019] With regard to the first and second aspects, the present invention encodes the normal operating data serving as the first reference data based on causal priors, so that the resulting potential representation has several first dimensions corresponding one to one with the causal factors; by intervening in each first dimension and reconstructing the resulting intervention representation, the causal factor serving as the root cause of the abnormality is identified based on the degree of difference between the reconstructed intervention data and the abnormal operating data serving as the target data, and even when encountering abnormal operating data that has never been seen before, the abnormality can be traced, thereby ensuring the interpretability of unknown abnormal service data.

[0020] For the third and fourth aspects, the present invention utilizes a causal structure diagram containing complex causal relationships to generate causal priors to participate in model training, so that the latent variables output by the encoder have several first dimensions that correspond one-to-one to the causal factors, so as to facilitate the identification of causal factors that are the root cause of the anomaly in the anomaly tracing application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 is a schematic diagram of a process of performing adversarial training on a discriminator in one embodiment of the present invention; Figure 2 is a schematic diagram of a process of performing adversarial training on a priori models, encoders, and generators in one embodiment of the present invention; Figure 3 It is a flowchart of a method for tracing the source of anomalies in a digital service network in one embodiment of the present invention; Figure 4 It is a structural schematic diagram of an abnormality tracing model in one embodiment of the present invention. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0024] It should be noted that: in other implementations, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other implementations, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other implementations; and multiple steps described in this specification may be combined into a single step for description in other implementations.

[0025] In some embodiments, this specification provides a method for training an anomaly tracing model, comprising the following steps: S100, data preparation; S110, collecting sample data; In this implementation, operation data of the digital service network in a normal operating state is collected as sample data.

[0026] In some implementations, observable labels corresponding to sample data are also collected as sample labels, and those skilled in the art may select observable labels, such as performance indicators, resource utilization, etc., according to actual needs.

[0027] S120, determining a causal structure diagram corresponding to the digital service network; An existing publicly available causal structure diagram or a causal structure diagram constructed based on the prior art may be used, and this specification does not introduce it in detail.

[0028] In this embodiment, the causal structure graph is represented by an adjacency matrix A. , m represents the number of causal factors. Those skilled in the art can obtain the adjacency matrix corresponding to the causal structure diagram based on the existing causal discovery technology; in A ij ≠0 means i The causal factor is the j The reason for the causal factors is A ij The value of i The causal factors of j The influence weight of causal factors.

[0029] S200, model building; S210, constructing a priori model; Building a priori model based on the causal structure diagram corresponding to the digital service network, wherein the input of the priori model is an exogenous variable and the output is a corresponding causal prior; The causal structure diagram is used to indicate the causal factors corresponding to the digital service network and the relationship between the causal factors; The exogenous variable is a random variable that obeys a standard normal distribution; In this implementation, the expression of the prior model is:

[0030] in: Exogenous variables The corresponding causal prior, in this implementation, the exogenous variable Follows standard normal distribution N(0,I) exogenous variables; I represents the identity matrix; A The adjacency matrix corresponding to the causal structure graph representing the digital service network; f and h is an element-wise nonlinear transformation function.

[0031] Existing exception tracing technologies mainly rely on call chain data and monitoring indicators, lacking modeling of deep causal relationships between services, making it difficult to accurately identify the true root cause of the exception; This implementation takes into account the causal structure diagram containing complex causal relationships to generate causal priors to participate in model training, which overcomes the defect of existing methods in insufficient modeling of complex causal relationships and can accurately identify the specific causal factors that lead to anomalies.

[0032] S220, build encoder E Φ (x) and generator G θ (z) ; Encoder E Φ (x) The input is the running data, and the output is the corresponding latent variable, that is, the encoder E Φ (x) Map the input sample data x to the latent variable z; In this embodiment, the latent variable z follows the prior model Generative causal priors; Latent variables z It has several first dimensions and several second dimensions, where the first dimensions correspond one to one with the causal factors, that is, , k represents the total number of dimensions of the latent variable z, the latent variable z Before m The first dimension is recorded as the first dimension, and the remaining kmThe dimension is recorded as the second dimension; The latent variable z m Dimensions satisfy the causal structure A SCM (StructuralCausalModel, causal structure diagram) prior; In this embodiment, the remaining latent variable z km The dimension uses other prior distributions (such as standard Gaussian distribution) to capture other factors required for the generator to generate data, ensuring the generation capability of the anomaly tracing model while capturing anomalies caused by unknown causal factors.

[0033] Generator G θ (z) The input is the latent variable, and the output is the reconstructed running data, that is, the generator G θ (z) Reconstruct running data based on the input latent variable z .

[0034] S230, build discriminator D ψ;

[0035] Discriminator D ψ The input of is the discriminant training data, and the output of the discriminator is the authenticity of the discriminant training data; Identify training data into corresponding latent variables and running data; The real data indicates that the operating data is the collected sample data, and the latent variable is the latent variable output by the encoder when encoding the sample data; Fake data means that the running data is the running data reconstructed based on the causal prior, and the causal prior generated by the prior model is used as the latent variable.

[0036] S300, model training; Based on the sample data, the discriminator and the prior model, the encoder and the generator are alternately trained adversarially until a preset iteration completion condition is reached, and the trained encoder and generator are used as an anomaly tracing model.

[0037] In this implementation, when the number of iterations reaches a preset maximum number of iterations, the training is determined to be completed.

[0038] Each iteration performs the following steps: S310, conduct adversarial training on the discriminator: Reference Figure 1 , the training steps specifically include: S311. Sample the sample data to obtain real operation data.

[0039] S312: Encode the real operation data based on an encoder, and use the obtained latent variable as a first latent variable; and construct discriminant training data based on the first latent variable and the real operation data.

[0040] S313, from the standard normal distribution N(0,I) Sampling exogenous variables , and generate a corresponding causal prior based on the prior model, and use the causal prior as the second latent variable.

[0041] S314: Generate virtual operation data corresponding to the second latent variable through the generator, and construct discriminant training data based on the corresponding second latent variable and the virtual operation data.

[0042] S315, the discriminator determines the authenticity of each discriminant training data, and adjusts the model parameters of the discriminator based on the output discrimination result; The discrimination result is the probability that the input discrimination training data belongs to the real sample.

[0043] In this embodiment: By gradient Update the model parameters of the discriminator; Representation Discriminator D ψ The real running data x and its encoded latent variable E Φ (x) The judgment result; Representation Discriminator D ψ The causal prior of the latent variable z and the virtual operation data obtained by reconstruction G θ (z) , the discrimination result.

[0044] is the loss of the real data part; when When larger, Close to 0, it means that the real sample is correctly classified and the loss is small; when When is smaller, the loss increases, and the discriminator is encouraged to improve its ability to identify real samples.

[0045] Losses due to falsified data; when When it is small or negative, Close to 0, it means that the virtual sample reconstructed by the generator is correctly identified as forged, and the loss is small; when When it is larger, the loss increases, and the discriminator is encouraged to improve its ability to identify virtual samples.

[0046] S320, conduct adversarial training on the prior model, encoder, and generator: Reference Figure 2 The training methods include: S321. Sample the sample data x to generate training data.

[0047] S322, from the standard normal distribution N(0,I) Sampling exogenous variables , and generate corresponding causal priors based on the prior model.

[0048] S323, the encoder encodes the generated training data based on the causal prior, and uses the obtained latent variable as the third latent variable z=E Φ （x） .

[0049] S324, generating reconstructed sample data corresponding to the third latent variable by the generator; Right now: .

[0050] S325: Calculate the first joint distribution corresponding to the encoder based on the corresponding generated training data and the third latent variable. q E (x,z) ; First joint distribution q E (x,z) For characterizing encoders E For sample data x (the sampled generated training data) and the third latent variable z The joint posterior distribution of .

[0051] S326: Calculate the second joint distribution corresponding to the generator based on the corresponding third latent variable and the reconstructed sample data. ; Second joint distribution Used to characterize the joint distribution of the generator; Compute the second joint distribution The expression is: ; where p F (z) is determined by the causal prior Distribution of decisions; Used to characterize a given third latent variable z When , the generator G Generate reconstructed sample data The conditional probability distribution of .

[0052] S327, calculating the KL divergence of the first joint distribution and the second joint distribution ; KL divergence A measure of the difference between the first joint distribution and the second joint distribution.

[0053] S328. Adjust the model parameters of the prior model, the encoder and the generator based on the KL divergence.

[0054] In some embodiments, a supervised regularization term is also set, and the sample labels used to generate the training data are used as supervisory signals to guide the encoder to learn the latent variables. In such embodiments, during the adversarial training of the prior model, encoder, and generator, the supervised loss corresponding to the sample labels is also calculated; and the total loss is calculated based on the KL divergence and the supervised loss, and the model parameters of the prior model, the encoder, and the generator are adjusted based on the total loss.

[0055] The loss function for calculating the total loss is:

[0056] in: L(E,G,F) represents the total loss; represents the first joint distribution q E (x,z) and the second joint distribution KL divergence of q E (x,z) For characterizing encoders E For sample data x and the third latent variable z The joint posterior distribution of Used to characterize the joint distribution of the generator; , where p F (z) Causal Prior Distribution of decisions; Used to characterize a given third latent variable z When , the generator G Generate reconstructed sample data The conditional probability distribution of .

[0057] λ is the balance coefficient, λ>0 , used to weigh the generation loss (KL divergence) and supervision loss; L sup is a supervised regularization term that uses sample labels as supervisory signals; In this embodiment, {L}_{sup}\left ( {E} \right )={\mathbb{E}}_{x,y}\left [ {{l}_{s}\left ( {E;x,y} \right )} \right ] , is the supervision loss corresponding to the sample label y, which can be cross entropy loss or mean square error loss. Express expectations.

[0058] In this implementation, the supervised regularization term introduces the observable sample label y as a supervisory signal to guide the encoder E to the latent variable z The learning of latent variables z Before m The dimension can capture the sample label y Relevant key causal factors facilitate subsequent anomaly analysis and root cause location.

[0059] In some embodiments, one or more embodiments of this specification provide a training system for an anomaly tracing model, which is used to execute the above-mentioned training method for the anomaly tracing model.

[0060] In some embodiments, one or more embodiments of this specification provide a method for tracing the source of anomalies in a digital service network, for tracing the source of anomalies in a digital service network; Reference Figure 3 , including the following steps: S400, obtaining first reference data and target data corresponding to the anomaly to be traced; The target data is the abnormal operation data corresponding to the anomaly to be traced, and the first reference data is the normal operation data within a preset time range before the anomaly to be traced occurs. Those skilled in the art can determine the extraction conditions of the second reference data according to actual needs, and this specification does not limit it in detail.

[0061] When an anomaly is detected in the service network, the anomaly can be treated as an anomaly to be traced, and historical anomalies that need to be traced can also be treated as anomalies to be traced.

[0062] A person skilled in the art may obtain the corresponding first reference data and target data based on the occurrence time of the anomaly to be traced. This embodiment intervenes in the latent variables of the first reference data, reconstructs the operating data based on the intervention results, and analyzes the similarity between the reconstructed operating data and the target data, thereby identifying the root cause of the anomaly.

[0063] S500, encoding the first reference data based on causal priors to obtain a corresponding first potential representation; The first potential representation has a plurality of first dimensions, and the first dimensions correspond one-to-one to causal factors; The causal prior is a prior distribution generated based on a causal structure graph corresponding to the digital service network; The causal structure diagram is used to indicate the causal factors corresponding to the digital service network and the relationship between the causal factors; In this implementation, the first latent representation is the latent variable generated by the trained encoder encoding the first reference data, and the causal prior is the prior distribution generated by the trained prior model.

[0064] S600, sequentially intervening the values ​​of each first dimension in the first potential representation to generate corresponding intervention representations, where each causal factor has at least one intervention representation corresponding thereto; Taking the first dimension to be intervened as the target dimension, the method of intervening the value of the target dimension in the first potential representation is: S610, taking the outlier sample corresponding to the target dimension as an intervention value; Outlier samples include simulated outliers and / or historical outliers; S611, the steps for obtaining the simulated abnormal value are: Acquire several pieces of second reference data, where the second reference data are all normal operating data within a preset time range before the occurrence of the anomaly to be traced. Those skilled in the art can determine the extraction conditions of the second reference data according to actual needs, and this specification does not limit them in detail; Encoding the second reference data based on the causal prior to obtain a corresponding second latent representation, wherein the second latent representation has a plurality of first dimensions, and the first dimensions correspond to the causal factors one by one; Statistics are performed on the values ​​of each first dimension in the second potential representation, and at least one simulated abnormal value corresponding to the first dimension is generated based on the statistical result.

[0065] Those skilled in the art can set statistical rules and abnormal simulation rules according to actual needs; For example: Count the median, mean, maximum, minimum, and standard deviation corresponding to each first dimension; Based on the median, several simulated outliers corresponding to the first dimension are generated by increasing or decreasing the preset multiple standard deviation; On the basis of the mean, several simulated outliers corresponding to the first dimension are generated by increasing or decreasing the preset multiple standard deviation; On the basis of the maximum value, at least one simulated abnormal value corresponding to the first dimension is generated by increasing the unit growth value by a preset multiple; On the basis of the maximum value, at least one simulated abnormal value corresponding to the first dimension is generated by reducing the unit growth value by a preset multiple.

[0066] S612, the steps for obtaining the simulated abnormal value are: Encoding the abnormal operation data of the determined root cause to obtain a corresponding third potential representation, wherein the third potential representation has a plurality of first dimensions, and the first dimensions correspond to the causal factors one by one; Based on the root cause, the value corresponding to the first dimension in the third potential representation is used as the corresponding historical abnormal value.

[0067] S620, assigning a value to the target dimension based on the intervention value, and obtaining an intervention representation z' corresponding to the intervention value one by one o ; That is, let [z] i = c , [z] i represents the target dimension of the first latent representation, and c is an intervention value.

[0068] S700, characterize each intervention z' o Reconstruct and obtain corresponding intervention data x ' o ; In this implementation, each intervention representation is reconstructed based on the above-mentioned trained generator, that is, x ' o =G θ (z' o ).

[0069] S800, calculating the similarity between the intervention data and the target data; Those skilled in the art may use any existing disclosed similarity calculation method to calculate the similarity between the intervention data and the target data, and this specification does not limit it in detail.

[0070] S900. Based on the similarity, select corresponding candidate data from the intervention data, and generate corresponding abnormality tracing results based on causal factors corresponding to the candidate data.

[0071] Those skilled in the art may set the root cause analysis rules according to actual needs. In this implementation: The first N intervention data whose similarity reaches the preset similarity threshold are taken as candidate data, N≥1; When there is no candidate data, the root cause to be processed is output as others; When candidate data exists, the causal factors corresponding to the candidate data are output as the root causes to be processed.

[0072] When service data has anomalies of unknown types, that is, for abnormal operation data that appears for the first time, traditional methods often have difficulty in effectively detecting and locating its root cause, and lack the ability to explain the unknown abnormal causes; This implementation combines causal decoupling learning and intervention mechanisms, intervenes in each first dimension corresponding to the normal operating data, calculates the degree of difference between the intervention data and the abnormal operating data, and locates the root cause based on the degree of difference and the intervened first dimension. Even if the abnormal operating data has never appeared, it does not affect the generation of corresponding intervention data and the location of the abnormal root cause, thereby ensuring the interpretability of unknown abnormal service data.

[0073] In summary, this method utilizes the causal structure information of the service network and combines it with the intervention mechanism to accurately identify the specific causal factors that cause anomalies. Compared with existing anomaly detection and tracing methods, the present invention has the advantages of strong applicability and high interpretability. It can be used in digital service networks of various sizes and types to improve network reliability and user satisfaction.

[0074] In some embodiments, one or more embodiments of this specification provide a digital service network abnormality tracing system for digital service network abnormality tracing, referring to Figure 4 ,include: The data extraction module 10 is used to obtain first reference data and target data corresponding to the anomaly to be traced, wherein the target data is the abnormal operation data corresponding to the anomaly to be traced, and the first reference data is the normal operation data within a preset time range before the anomaly to be traced occurs; The encoder 20 is used to encode the first reference data based on the causal prior to obtain a corresponding first potential representation, wherein the first potential representation has a plurality of first dimensions, and the first dimensions correspond to the causal factors one by one; the causal prior is a prior distribution generated based on a causal structure graph corresponding to the digital service network; the causal structure graph is used to indicate the causal factors corresponding to the digital service network, and the relationship between the causal factors; An intervention module 30, configured to intervene in the values ​​of each first dimension in the first potential representation to generate a corresponding intervention representation, wherein each causal factor has at least one intervention representation corresponding thereto; A generator 40, for reconstructing the intervention representation to obtain corresponding intervention data; A calculation module 50, used to calculate the similarity between the intervention data and the target data; The analysis module 60 is used to screen out corresponding candidate data from the intervention data based on the similarity, and generate corresponding abnormality tracing results based on the causal factors corresponding to the candidate data.

[0075] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0076] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0077] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] The present invention is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0079] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0081] It should be noted that: The "one embodiment" or "embodiment" mentioned in the specification means that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment" or "embodiment" appearing in various places throughout the specification do not necessarily refer to the same embodiment.

[0082] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0083] In addition, it should be noted that the shapes and names of the parts and components of the specific embodiments described in this specification may be different. Any equivalent or simple changes made based on the structure, features and principles described in the patent concept of the present invention are included in the protection scope of the patent of the present invention. The technicians in the technical field of the present invention can make various modifications or supplements to the specific embodiments described or replace them in a similar manner, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A method for tracing the source of abnormalities in a digital service network, characterized in that: The following steps are involved: Acquire first reference data and target data corresponding to the anomaly to be traced, wherein the target data is the abnormal operation data corresponding to the anomaly to be traced, and the first reference data is the normal operation data within a preset time range before the anomaly to be traced occurs; The first reference data is encoded based on a causal prior to obtain a corresponding first potential representation, wherein the first potential representation has a plurality of first dimensions, and the first dimensions correspond to causal factors one by one; the causal prior is a prior distribution generated based on a causal structure graph corresponding to the digital service network; the causal structure graph is used to indicate the causal factors corresponding to the digital service network, and the relationship between the causal factors; sequentially intervening in the values ​​of the first dimensions of the first potential representation to generate corresponding intervention representations, wherein each causal factor has at least one intervention representation corresponding thereto; Reconstruct each intervention representation to obtain corresponding intervention data; Calculating the similarity between the intervention data and the target data; Based on the similarity, corresponding candidate data are screened out from the intervention data, and corresponding abnormality tracing results are generated based on the causal factors corresponding to the candidate data.

2. The method for tracing the source of abnormality in a digital service network according to claim 1, characterized in that: Taking the first dimension to be intervened as the target dimension, the method of intervening the value of the target dimension in the first potential representation is: The outlier samples corresponding to the target dimension are used as intervention values; Assigning values ​​to the target dimensions based on the intervention values ​​to obtain intervention representations that correspond one to one with the intervention values; The outlier samples include simulated outliers, and the steps for obtaining the simulated outliers are: Acquire a plurality of second reference data, where the second reference data are all normal operating data within a preset time range before the abnormality to be traced occurs; Encoding the second reference data based on the causal prior to obtain a corresponding second latent representation, wherein the second latent representation has a plurality of first dimensions, and the first dimensions correspond to the causal factors one by one; Statistics are performed on the values ​​of each first dimension in the second potential representation, and at least one simulated abnormal value corresponding to the first dimension is generated based on the statistical result.

3. The method for tracing the source of abnormality in a digital service network according to claim 2, characterized in that: The outlier samples also include historical outliers. The historical outliers are obtained as follows: Encoding the abnormal operation data of the determined root cause to obtain a corresponding third potential representation, wherein the third potential representation has a plurality of first dimensions, and the first dimensions correspond to the causal factors one by one; Based on the root cause, the value corresponding to the first dimension in the third potential representation is used as the corresponding historical abnormal value.

4. The method for tracing the source of abnormalities in a digital service network according to any one of claims 1 to 3, characterized in that: The first N intervention data whose similarity reaches the preset similarity threshold are taken as candidate data, N≥1; When there is no candidate data, the root cause to be processed is output as others; When candidate data exists, the causal factors corresponding to the candidate data are output as the root causes to be processed.

5. An abnormality tracing system for a digital service network, characterized in that: include: A data extraction module, used to obtain first reference data and target data corresponding to the anomaly to be traced, wherein the target data is the abnormal operation data corresponding to the anomaly to be traced, and the first reference data is the normal operation data within a preset time range before the anomaly to be traced occurs; an encoder, configured to encode the first reference data based on a causal prior to obtain a corresponding first latent representation, wherein the first latent representation has a plurality of first dimensions, and the first dimensions correspond one-to-one to causal factors; the causal prior is a prior distribution generated based on a causal structure graph corresponding to the digital service network; the causal structure graph is used to indicate the causal factors corresponding to the digital service network, and the relationship between the causal factors; An intervention module, used for intervening the values ​​of each first dimension in the first potential representation to generate a corresponding intervention representation, wherein each causal factor has at least one intervention representation corresponding thereto; A generator, used to reconstruct the intervention representation and obtain the corresponding intervention data; A calculation module, used for calculating the similarity between the intervention data and the target data; The analysis module is used to screen out corresponding candidate data from the intervention data based on the similarity, and generate corresponding abnormality tracing results based on the causal factors corresponding to the candidate data.

6. A training method for an anomaly tracing model, characterized in that: The steps include: Obtaining operational data of the digital service network during normal operation as sample data; Based on the sample data, the discriminator and the prior model, the encoder and the generator are alternately trained adversarially, and the trained encoder and generator are used as an anomaly tracing model; in: Building a priori model based on the causal structure diagram corresponding to the digital service network, wherein the input of the priori model is an exogenous variable and the output is a corresponding causal prior; The causal structure diagram is used to indicate the causal factors corresponding to the digital service network and the relationship between the causal factors; The exogenous variable is a random variable that obeys a standard normal distribution; The steps of conducting adversarial training on the discriminator are as follows: Construct discriminative training data; The discriminator determines the authenticity of the discriminant training data, and adjusts the model parameters of the discriminator based on the discrimination result; The method for constructing the discriminant training data is: Encode the sample data based on the encoder and use the obtained latent variable as a first latent variable; construct discriminant training data based on the first latent variable and the sample data; A causal prior is generated based on the prior model, and the causal prior is used as a second latent variable; virtual operation data corresponding to the second latent variable is generated by the generator, and discriminant training data is constructed based on the corresponding second latent variable and the virtual operation data.

7. The training method of the anomaly tracing model according to claim 6 is characterized in that: The expression of the prior model is: ; in: Represents the prior model, which is used to generate exogenous variables The corresponding causal prior; I represents the identity matrix; A The adjacency matrix corresponding to the causal structure graph representing the digital service network.

8. The training method of the anomaly tracing model according to claim 6 or 7, characterized in that: The steps of performing adversarial training on the prior model, the encoder and the generator are: generating a causal prior based on the prior model; The encoder encodes the sample data based on the causal prior and uses the obtained latent variable as a third latent variable; Generate reconstructed sample data corresponding to the third latent variable by the generator; Calculating a first joint distribution corresponding to the encoder based on the corresponding sample data and a third latent variable; Calculate a second joint distribution corresponding to the generator based on the corresponding third latent variable and the reconstructed sample data; The KL divergence of the first joint distribution and the second joint distribution is calculated, and model parameters of the prior model, the encoder, and the generator are adjusted based on the KL divergence.

9. The training method of the anomaly tracing model according to claim 8, characterized in that: Obtain the observable label corresponding to the sample data as the sample label; In the adversarial training step of the prior model, encoder and generator, the supervision loss corresponding to the sample label is also calculated; The model parameters of the prior model, the encoder, and the generator are adjusted based on the KL divergence and the supervised loss.

10. The training method of the anomaly tracing model according to claim 9, characterized in that: Determine a total loss based on the KL divergence and the supervised loss, and adjust model parameters of the prior model, the encoder, and the generator based on the total loss; The loss function for calculating the total loss is: ; in: L(E,G,F) represents the total loss; represents the first joint distribution q E (x,z) and the second joint distribution KL divergence of q E (x,z) For characterizing encoders E For sample data x and the third latent variable z The joint posterior distribution of Used to characterize the joint distribution of the generator; λ is the balance coefficient; Lsup(E) is the supervised regularization term that uses the sample label as the supervisory signal, , is the sample label y The corresponding supervision loss is Express expectations.

11. A training system for an anomaly tracing model, characterized in that: A training method for executing the anomaly tracing model described in any one of claims 6 to 10.

Images