Network service data processing method, device and storage medium

By clustering the time series data of multiple indicators and building a causal relationship network, the problem of high computational cost in high-dimensional time series data of Granger is solved, efficient causal relationship discovery is achieved, applicable scenarios are expanded and decision support is provided.

CN116821725BActive Publication Date: 2024-11-22BEIJING VOLCANO ENGINE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310315396.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-28
Publication Date
2024-11-22
Estimated Expiration
2043-03-28

AI Technical Summary

Technical Problem

When facing multi-index high-dimensional timing data, the Granger causal model has high calculation cost and limited processing power, making it difficult to effectively discover causal relationships.

Method used

By obtaining the time series data of multiple indicators of the causal relationship to be analyzed, clustering to reduce the dimension, constructing a causal relationship network structure, and obtaining the Bayesian belief network structure to express the causal relationship.

Benefits of technology

It greatly improves the efficiency of causal relationship discovery, reduces calculation costs, expands the applicable scenarios of causal relationship discovery, and provides auxiliary information support for decision-making or root causes of failures.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116821725B_ABST
    Figure CN116821725B_ABST
Patent Text Reader

Abstract

The disclosed embodiment provides a network service data processing method, device and storage medium, which obtains the time series data of multiple indicators of causal relationships to be analyzed; clusters the multiple indicators according to the probability distribution of the time series data of the multiple indicators; constructs a causal relationship network structure by analyzing the causal connection relationship and connection orientation between the indicators based on the clustering results; obtains the conditional probability table of each indicator node in the causal relationship network structure according to the time series data of the multiple indicators; obtains the Bayesian belief network structure according to the causal relationship network structure and the conditional probability table of each indicator node to represent the causal relationship between the indicators. In this embodiment, causal relationship discovery can be performed on high-dimensional time series data of multiple indicators, which expands the applicable scenarios of causal relationship discovery, reduces the computing cost, improves the efficiency of causal relationship discovery, and provides auxiliary information support for decision-making or fault root cause location.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer and network communication technology, and in particular, to a network service data processing method, device, and storage medium. Background Art

[0002] Causal relationship identification is a challenging relationship identification task in natural language processing and has attracted increasing attention in the field of natural language processing.

[0003] In the prior art, Granger causality model is usually used to discover the causal characteristics of time series data, so as to reveal the Granger causal relationship behind multivariate time series data. However, Granger causality model can usually only be used to detect the causal relationship between two indicators. When faced with high-dimensional time series data with multiple indicators, the calculation cost is high and the processing capacity is limited. Summary of the invention

[0004] The embodiments of the present disclosure provide a network service data processing method, device, and storage medium to reduce the cost of discovering causal relationships in multi-indicator high-dimensional time series data and improve efficiency.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for processing network service data, including:

[0006] Obtain time series data of multiple indicators whose causal relationships are to be analyzed;

[0007] Clustering the multiple indicators according to the probability distribution of the time series data of the multiple indicators, wherein indicators of the same category are independent and identically distributed indicators;

[0008] Analyze the causal connection relationship and connection direction between each indicator based on the clustering result, and construct a causal relationship network structure, wherein the causal relationship network structure includes indicator nodes and directed edges connecting the indicator nodes, and the directed edges are used to represent the causal relationship between the connected indicator nodes;

[0009] According to the time series data of the multiple indicators, a conditional probability table of each indicator node in the causal relationship network structure is obtained;

[0010] According to the causal relationship network structure and the conditional probability table of each indicator node, a Bayesian belief network structure is obtained to represent the causal relationship between the indicators.

[0011] In a second aspect, an embodiment of the present disclosure provides a network service data processing device, including:

[0012] A time series data acquisition unit, used to acquire time series data of multiple indicators whose causal relationships are to be analyzed;

[0013] A clustering unit, used to cluster the multiple indicators according to the probability distribution of the time series data of the multiple indicators, wherein the indicators of the same category are independent and identically distributed indicators;

[0014] A causal structure learning unit, used to analyze the causal connection relationship and connection direction between each indicator based on the clustering result, and construct a causal relationship network structure, wherein the causal relationship network structure includes indicator nodes and directed edges connecting the indicator nodes, wherein the directed edges are used to represent the causal relationship between the connected indicator nodes;

[0015] A conditional probability table learning unit, used to obtain a conditional probability table of each indicator node in the causal relationship network structure according to the time series data of the multiple indicators;

[0016] The model generation unit is used to obtain a Bayesian belief network structure according to the causal relationship network structure and the conditional probability table of each indicator node to represent the causal relationship between the indicators.

[0017] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor and a memory;

[0018] The memory stores computer-executable instructions;

[0019] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the data processing method described in the first aspect and various possible designs of the first aspect.

[0020] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the data processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0021] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including computer execution instructions. When a processor executes the computer execution instructions, it implements the data processing method described in the first aspect and various possible designs of the first aspect.

[0022] The network service data processing method, device and storage medium provided by the embodiment of the present disclosure obtain the time series data of multiple indicators of the causal relationship to be analyzed; cluster the multiple indicators according to the probability distribution of the time series data of the multiple indicators, wherein the indicators of the same category are independent and identically distributed indicators; analyze the causal connection relationship and connection direction between the indicators based on the clustering results, and construct a causal relationship network structure, wherein the causal relationship network structure includes indicator nodes and directed edges connecting the indicator nodes, wherein the directed edges are used to represent the causal relationship between the connected indicator nodes; obtain the conditional probability table of each indicator node in the causal relationship network structure according to the time series data of the multiple indicators; obtain the Bayesian belief network structure according to the causal relationship network structure and the conditional probability table of each indicator node to represent the causal relationship between the indicators. In this embodiment, causal relationship discovery can be performed on high-dimensional time series data of multiple indicators, which expands the applicable scenarios of causal relationship discovery, reduces the computational cost, improves the efficiency of causal relationship discovery, and provides auxiliary information support for decision-making or fault root cause location. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1 A diagram showing an example scenario of a data processing method provided by an embodiment of the present disclosure;

[0025] Figure 2 A flowchart of a data processing method provided by an embodiment of the present disclosure;

[0026] Figure 3 A flowchart of a data processing method provided by another embodiment of the present disclosure;

[0027] Figure 4 A schematic diagram of a data processing method system architecture provided by an embodiment of the present disclosure;

[0028] Figure 5 A structural block diagram of a data processing device provided in one embodiment of the present disclosure;

[0029] Figure 6 A structural block diagram of a data processing device provided in another embodiment of the present disclosure;

[0030] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0032] In the prior art, Granger causality model is usually used to discover the causal characteristics of time series data, so as to reveal the Granger causal relationship behind multivariate time series data. However, Granger causality model can usually only be used to detect the causal relationship between two indicators. When faced with high-dimensional time series data with multiple indicators, the calculation cost is high and the processing capacity is limited.

[0033] In order to solve the above technical problems, the present disclosure provides a data processing method, by obtaining the time series data of multiple indicators of causal relationships to be analyzed; clustering the multiple indicators according to the probability distribution of the time series data of the multiple indicators, wherein the indicators of the same category are independent and identically distributed indicators; analyzing the causal connection relationship and connection direction between the indicators based on the clustering results, constructing a causal relationship network structure, the causal relationship network structure includes indicator nodes and directed edges connecting the indicator nodes, and the directed edges are used to represent the causal relationship between the connected indicator nodes; according to the time series data of the multiple indicators, obtaining the conditional probability table of each indicator node in the causal relationship network structure; according to the causal relationship network structure and the conditional probability table of each indicator node, obtaining the Bayesian belief network structure to represent the causal relationship between the indicators. The present disclosure can perform causal relationship discovery on high-dimensional time series data of multiple indicators, greatly improve the dimension of time series data that can be processed in the process of causal relationship discovery, expand the applicable scenarios of causal relationship discovery, and reduce the computational cost through clustering dimensionality reduction, improve the efficiency of causal relationship discovery, and provide auxiliary information support for decision-making or fault root cause location in application scenarios such as network services.

[0034] The present invention provides a data processing method suitable for Figure 1 The application scenario shown includes a data acquisition system 101 and a data processing device 102, wherein the data acquisition system 101 can obtain time series data of multiple indicators of causal relationships to be analyzed, and send them to the data processing device 102 to execute the above data processing method, and finally obtain a Bayesian belief network structure to represent the causal relationship between the indicators.

[0035] The data processing method of the present invention will be described in detail below in conjunction with specific embodiments.

[0036] refer to Figure 2 , Figure 2 A flowchart of a data processing method provided by an embodiment of the present disclosure. The method of this embodiment can be applied in a terminal device or a server, and the data processing method includes:

[0037] S201. Obtain time series data of multiple indicators of causal relationships to be analyzed.

[0038] In this embodiment, multiple indicators of causal relationships to be analyzed can be determined according to specific application scenarios. For example, the causal relationships of indicators such as the concurrent user volume of network services, the server-side memory usage, and the CPU usage rate are determined, and the time series data of the concurrent user volume of network services, the server-side memory usage rate, and the CPU usage rate are obtained respectively. Optionally, the time series data of multiple indicators can be obtained from a third-party data collection system.

[0039] S202: Clustering the multiple indicators according to the probability distribution of their time series data, wherein indicators of the same category are independent and identically distributed indicators.

[0040] In this embodiment, since the multiple indicators may be massive indicators, in order to eliminate the indicators with no causal correlation or weak correlation among the multiple indicators and find a set of indicators with strong correlation, the multiple indicators can be clustered according to the time series data of the multiple indicators, wherein the indicators of the same category are independent and identically distributed (IID) indicators, that is, the indicators are clustered according to their probability distribution, for example, the indicators with normal distribution are clustered into one category, and the indicators with beta distribution are clustered into one category. The indicators with the same or similar probability distribution have a strong correlation and may have a causal relationship, while the indicators with different or dissimilar probability distributions have a weak correlation and may not have a causal relationship.

[0041] Optionally, in this embodiment, any clustering algorithm can be used to cluster multiple indicators. Among them, commonly used clustering algorithms can be divided into hierarchical clustering algorithms (Hierarchical clustering methods, also known as hierarchical clustering methods) and partition clustering algorithms (Partition clustering methods, also known as partition-based clustering algorithms), wherein the hierarchical clustering algorithm is to reveal the hierarchical structure of data, and to form a tree-shaped clustering structure by partitioning at different levels; the partition clustering algorithm is to divide multiple object sets into different categories, which are mutually exclusive, and each object belongs to and only belongs to one category; in this embodiment, a partition clustering algorithm can be used, and its principle is to calculate the distance between each indicator based on the time series data of multiple indicators (usually using Euclidean distance, dynamic time warping, etc.), and divide the indicators into multiple sets according to the distance. The clustering results can be represented and stored using an adjacency matrix, and the adjacency matrix includes the correlation coefficient between each indicator, indicating the correlation between each indicator in the probability distribution. According to the adjacency matrix, it can be determined whether any two indicators belong to the same category.

[0042] Optionally, the clustering algorithms in this embodiment include Kmeans, DBSCAN, and OPTICS, all of which are based on the maximum reachable density theory in principle.

[0043] Optionally, the clustering of multiple indicators in this embodiment belongs to high-dimensional data clustering. Non-parametric Bayesian clustering methods can be used for high-dimensional data, such as CRP (Chinese Restaruant Process) based on Dirichlet process and its derivative methods. Multiple indicators are clustered according to their time series data, and the final result is output as an adjacency matrix, which can quickly find similar correlations in high-dimensional time series data.

[0044] S203. Analyze the causal connection relationship and connection direction between the indicators based on the clustering results, and construct a causal relationship network structure, wherein the causal relationship network structure includes indicator nodes and directed edges connecting the indicator nodes, and the directed edges are used to represent the causal relationship between the connected indicator nodes.

[0045] In this embodiment, the score based structure learning algorithm can be applied to learn the causal network structure. Specifically, based on the cluster analysis results, the causal connection relationship and connection direction between indicators can be analyzed. For example, the correlation between verification indicators can be found through the conditional independence test method (such as the d / m-separation method), and the connection direction in the causal connection relationship between each indicator can be determined through the V-structure and Meek rules method.

[0046] Furthermore, indicators are taken as nodes, and a causal network structure is constructed according to connection relationships and connection directions, wherein the causal network structure has indicators as nodes and causal relationships as directed edges, wherein the causal network structure can be a directed acyclic graph (DAG) or a maximal ancestor graph (MAG), wherein indicator nodes with connection relationships indicate that there is a causal relationship between the indicators, which are connected by directed edges, and the connection directions indicate which indicator among the indicator nodes connected by the directed edges is the cause and which indicator is the effect.

[0047] S204: Obtain a conditional probability table of each indicator node in the causal relationship network structure according to the time series data of the multiple indicators.

[0048] In this embodiment, the indicator nodes in the causal network structure are connected by directed edges to indicate the existence of a causal relationship, but the strength of the connection relationship cannot be represented. Therefore, a conditional probability table (CPT) of each indicator node in the causal network structure can be obtained to facilitate the subsequent representation of the strength of the connection relationship through conditional probability.

[0049] Optionally, based on the causal network structure, the SimpleEstimator algorithm is executed to generate the conditional probability table CPT of each indicator node. For any indicator node with a parent indicator node, the conditional probability of the indicator node when its parent indicator node takes each possible value is obtained to obtain the conditional probability table of the indicator node; for any indicator node without a parent indicator node, the conditional probability table of the indicator node is determined according to the probability distribution (prior probability distribution) of the indicator node.

[0050] S205 . Obtain a Bayesian belief network structure according to the causal relationship network structure and the conditional probability table of each indicator node to represent the causal relationship between the indicators.

[0051] In this embodiment, after obtaining the causal network structure and the conditional probability table of each indicator node, the Bayesian belief network structure (BBN) of the directed acyclic graph can be spliced ​​based on the causal network structure and the conditional probability table of each indicator node to represent the causal relationship between the indicators. In the Bayesian belief network structure, indicators are nodes and connected by directed edges. The directed edges between nodes represent the mutual relationship between indicator nodes (from the parent indicator node to its child indicator node). The conditional probability is used to express the relationship strength, and the prior probability is used to express the information of the node without a parent.

[0052] On the basis of the Bayesian belief network structure, the most probable explanation (MPE) that meets the reasoning conditions can be solved according to the reasoning conditions and prior knowledge. For example, prior knowledge can include the relationship between any two indicators. For example, if the CPU usage increases by 2%, the memory usage increases by 10%. If the CPU usage increases by 4%, the memory usage increases by 30%. Based on this prior knowledge and the Bayesian belief network structure, the maximum probability solution of a 6% increase in CPU usage, memory usage and other indicators can be inferred to provide auxiliary information support for decision-making or fault root cause location.

[0053] The data processing method provided in this embodiment obtains the time series data of multiple indicators whose causal relationships are to be analyzed; clusters the multiple indicators according to the probability distribution of the time series data of the multiple indicators, wherein the indicators of the same category are independent and identically distributed indicators; analyzes the causal connection relationship and connection direction between the indicators based on the clustering results, and constructs a causal network structure, wherein the causal network structure includes indicator nodes and directed edges connecting the indicator nodes, wherein the directed edges are used to represent the causal relationship between the connected indicator nodes; obtains the conditional probability table of each indicator node in the causal network structure according to the time series data of the multiple indicators; obtains the Bayesian belief network structure according to the causal network structure and the conditional probability table of each indicator node to represent the causal relationship between the indicators. In this embodiment, causal relationship discovery can be performed on high-dimensional time series data of multiple indicators, which expands the applicable scenarios of causal relationship discovery, reduces the computational cost, improves the efficiency of causal relationship discovery, and provides auxiliary information support for decision-making or fault root cause location.

[0054] Based on the above embodiments, Figure 3 A flow chart of a data processing method provided in one embodiment of the present disclosure. The data processing method includes:

[0055] S301: Obtain time series data of multiple indicators of causal relationships to be analyzed.

[0056] In this embodiment, the acquisition of time series data of multiple indicators of the causal relationship to be analyzed can refer to the above S201.

[0057] S302 , clustering the multiple indicators according to the probability distribution of the time series data of the multiple indicators to obtain an adjacency matrix.

[0058] In this embodiment, the multiple indicators are clustered according to the probability distribution of their time series data, as can be seen in the above S202, and indicators with no causal relationship or weak correlation are eliminated to find indicators with strong correlation to achieve dimensionality reduction. The final clustering results are represented and stored in an adjacency matrix.

[0059] S303: Discretize the time series data of each indicator to obtain a discretized data set corresponding to each indicator.

[0060] In this embodiment, considering that the cost of analyzing the causal connection relationship and connection direction between each indicator is very high, the time series data of each indicator can be discretized to reduce the cost of subsequent analysis of the causal connection relationship and connection direction between each indicator and the construction of the causal network structure. In the discretization process, since the time series data is continuous, different preset intervals can be configured for each indicator. For example, five preset intervals are configured for a certain indicator, and the five preset intervals correspond to preset indicator values. When the time series data of the indicator falls into one of the preset intervals, the preset indicator value replaces the time series data, thereby realizing the discretization of the time series data of the indicator, and obtaining the discretized data set of the indicator. The above discretization process is performed on each indicator in the adjacency matrix to obtain the discretized data set corresponding to each indicator in the adjacency matrix. In addition, the probability distribution of each indicator can be obtained according to the discretized data set corresponding to each indicator, providing support for the calculation of the conditional probability table.

[0061] S304: Based on the discretized data sets corresponding to the indicators in the adjacency matrix, determine the causal connection relationship and connection direction between the indicators, and construct a causal relationship network structure.

[0062] In this embodiment, based on the discretized data sets corresponding to each indicator in the adjacency matrix, a causal structure discovery algorithm is used to determine the causal connection relationship and connection direction between each indicator, and a causal relationship network structure is constructed, which can greatly reduce costs and improve efficiency. The specific process can be seen in S203.

[0063] Optionally, in this embodiment, a conditional independence test method (such as the d / m-separation method) can be used to perform an independence test on each indicator in the adjacency matrix (the initial acyclic graph (DAG) or maximum ancestor graph (MAG) can be obtained according to the adjacency matrix), determine the conditional independence indicators of the directed acyclic graph (DAG) or the maximum ancestor graph (MAG), and eliminate the causal connection relationship between the conditional independence indicators and other indicators, wherein the d-separation method can quickly determine whether two nodes are conditionally independent for the acyclic graph (DAG), and the m-separation method can quickly determine whether two nodes are conditionally independent for the maximum ancestor graph (MAG); and determine the connection direction in the causal connection relationship between each indicator according to the V-structure and Meek rules method, and finally obtain a directed acyclic graph (DAG) or the maximum ancestor graph (MAG), and determine the directed acyclic graph (DAG) or the maximum ancestor graph (MAG) as the causal network structure.

[0064] S305 . Obtain a conditional probability table of each indicator node in the causal relationship network structure according to the discretized data set corresponding to each indicator.

[0065] In this embodiment, based on the causal network structure, the SimpleEstimator algorithm can be executed to generate the conditional probability table CPT of each indicator node. For any indicator node with a parent indicator node, the conditional probability of the indicator node when its parent indicator node takes each possible value is obtained to obtain the conditional probability table of the indicator node; for any indicator node without a parent indicator node, the probability distribution (prior probability distribution) of the indicator node is obtained, and the conditional probability table of the indicator node is determined according to the probability distribution of the indicator node. The probability distribution of the indicator node can be obtained by obtaining the probability distribution of the indicator node according to the discretized data set corresponding to the indicator node.

[0066] S306: According to the causal relationship network structure and the conditional probability table of each indicator node, a Bayesian belief network structure is obtained to represent the causal relationship between the indicators.

[0067] In this embodiment, the Bayesian belief network structure (BBN) of the spliced ​​directed acyclic graph can refer to the above S205.

[0068] On the basis of any of the above embodiments, when analyzing the causal connection relationship and connection direction between each indicator based on the clustering results and constructing a causal network structure, it is taken into account that the causal relationship between the indicators may be synchronous. For example, if indicator a fluctuates in the same time window, indicator b also fluctuates synchronously. In this case, the causal connection relationship and connection direction between each indicator are analyzed based on the time series data of the same time window of each indicator to construct a causal network structure, in which each indicator corresponds to a node.

[0069] Specifically, in S304, based on the discretized data sets corresponding to the indicators in the adjacency matrix, the causal connection relationship and the connection direction between the indicators are determined, and when the causal network structure is constructed, the discrete data of each indicator in the current time window can be firstly screened from the discretized data sets corresponding to the indicators in the adjacency matrix to form a first discretized data set; based on the first discretized data set, the causal connection relationship and the connection direction between the indicators are determined, and the causal network structure is constructed.

[0070] In addition, there may be a certain lag in the causal relationship between indicators. For example, indicator a fluctuates in the previous time window (t-1), and indicator b fluctuates in the next time window (t). That is, there is a causal relationship between indicator a in the previous time window (t-1) and indicator b in the next time window (t). In this case, the causal connection relationship and connection direction between indicators can be analyzed based on the time series data of different time windows of each indicator to construct a causal network structure, in which the indicator of only one time window corresponds to one node, and the indicator of two time windows analyzed at the same time corresponds to two nodes, which respectively represent the indicator in the two time windows.

[0071] Specifically, in S304, based on the discretized data sets corresponding to the indicators in the adjacency matrix, the causal connection relationship and connection direction between the indicators are determined, and the causal relationship network structure is constructed, including:

[0072] The discretized data of at least one first indicator in the previous time window (t-1) are screened from the discretized data sets corresponding to each indicator in the adjacency matrix to form a second discretized data set, and the discretized data of at least one second indicator in the current time window (t) are screened to form a third discretized data set; based on the second discretized data set and the third discretized data set, the causal connection relationship and connection direction between each first indicator in the previous time window (t-1) and each second indicator in the current time window (t) are determined to construct a causal network structure.

[0073] In this embodiment, the causal connection relationship and connection direction between each first indicator of the previous time window (t-1) and each second indicator of the current time window (t) can be analyzed through the above process, and the time window corresponding to each indicator can be identified in the constructed causal relationship network structure. It should be noted that if some indicators need to simultaneously analyze the causal connection relationship and connection direction between the previous time window (t-1) and the current time window (t) and other indicators, the first indicator and the second indicator both include the indicator, that is, the first indicator and the second indicator in this embodiment can overlap, and the causal connection relationship and connection direction between the indicator of the previous time window (t-1) and other indicators, as well as the causal connection relationship and connection direction between the indicator of the current time window (t) and other indicators can be analyzed through the causal structure discovery algorithm. When constructing the causal relationship network structure, the indicator corresponds to two nodes, which respectively represent the indicator of the previous time window (t-1) and the indicator of the current time window (t), indicating the causal connection relationship and connection direction between the indicator and other indicators under different time windows.

[0074] Based on any of the above embodiments, after obtaining the Bayesian belief network structure, the following may also be included:

[0075] S401, obtaining a priori knowledge of the relationship between reasoning conditions and any two indicators; wherein the reasoning conditions are the values ​​of some indicator nodes in the Bayesian belief network structure;

[0076] S402: According to the prior knowledge and the Bayesian belief network structure, a maximum probability solution that meets the reasoning conditions is obtained, and the values ​​of another part of the indicator nodes in the Bayesian belief network structure are determined based on the maximum probability solution.

[0077] In this embodiment, at the stage of applying the Bayesian belief network structure, reasoning can be performed based on prior knowledge and the Bayesian belief network structure, wherein the reasoning condition can be the value of certain given indicators, that is, the value of other indicators can be inferred based on prior knowledge and the Bayesian belief network structure under the condition of given values ​​of certain indicators. For example, given the value of the user concurrency of a network service (such as a 3% increase in user concurrency), the prior knowledge can be the known relationship between the memory usage rate and the CPU usage rate on the server side, such as a 4% increase in memory usage rate and a 2% increase in CPU usage rate. Based on the above prior knowledge and the Bayesian belief network structure, the maximum probability solution that meets the reasoning condition can be determined, that is, the predicted values ​​of other indicators such as the server side memory usage rate and the CPU usage rate. Optionally, in this embodiment, the RPLoc algorithm can be used to generate the maximum probability solution MPE based on prior knowledge and the Bayesian belief network structure.

[0078] Optionally, since the reasoning process is time-consuming, a publish / subscribe (pub / sub) mode may be used to asynchronously obtain the maximum probability solution that meets the reasoning conditions.

[0079] Based on any of the above embodiments, this embodiment also provides a data processing system architecture, such as Figure 4 As shown, the system architecture includes a time series causal relationship discovery server, a time series data analysis engine, a time series clustering dimensionality reduction engine, a discretization processor, a causal structure learning engine, a CPT table learning engine, and a model generation engine, wherein the various parts of the data processing system architecture can be arranged on the same server or on different servers. Figure 4 The time series data, adjacency matrix, discrete data set, causal network structure, conditional probability table CPT, Bayesian belief network structure, and maximum probability solution MPE shown in are only schematic diagrams.

[0080] Among them, the time series causal relationship discovery server: provides multiple ways of interaction for data applications (such as mq pub / sub asynchronous messages or REStful API interfaces), provides causal influence relationship retrieval and hypothesis condition reasoning services for data applications, and can input reasoning conditions and prior knowledge of the relationship between any two indicators into the time series causal relationship discovery server;

[0081] Time series data analysis engine: used to connect to third-party data collection systems, receive time series data sets of various indicators, and create causal relationship discovery tasks based on time series causal relationship discovery server session requests;

[0082] Time series clustering dimensionality reduction engine: used to receive the time series data sets of various indicators of the time series data analysis engine's associated causal relationship discovery task, execute the clustering dimensionality reduction algorithm, and output the adjacency matrix;

[0083] Discretization processor: used to discretize the time series data set corresponding to each indicator in the adjacency matrix according to the specified parameters (preset intervals) to obtain the discretized data set corresponding to each indicator in the adjacency matrix;

[0084] Causal structure learning engine: executes the causal structure discovery algorithm to generate a causal network structure based on the discretized data set corresponding to each indicator in the adjacency matrix;

[0085] CPT table learning engine: used to execute the RPLoc algorithm to generate the conditional probability table CPT of each node in the causal network structure based on the discretized data set corresponding to each indicator in the adjacency matrix and the causal network structure;

[0086] Model generation engine: used to obtain the Bayesian belief network structure according to the causal network structure and the conditional probability table CPT of each indicator node. It can also be used to generate the maximum probability solution MPE according to the reasoning conditions input by the time series causal relationship discovery server and the prior knowledge reasoning. The maximum probability solution MPE is returned to the time series data analysis engine. After the time series data analysis engine combines the tasks to generate the final result set, it is returned to the time series causal relationship discovery server, and then sent by the time series causal relationship discovery server to the data application end that initiated the session.

[0087] Corresponding to the data processing method of the above embodiment, Figure 5 The structure block diagram of the data processing device provided by the embodiment of the present disclosure. For the convenience of explanation, only the part related to the embodiment of the present disclosure is shown. Figure 5 The data processing device 500 includes: a time series data acquisition unit 501, a clustering unit 502, a causal structure learning unit 503, a conditional probability table learning unit 504, and a model generation unit 505.

[0088] The time series data acquisition unit 501 is used to acquire time series data of multiple indicators of causal relationships to be analyzed;

[0089] A clustering unit 502, configured to cluster the multiple indicators according to the probability distribution of the time series data of the multiple indicators, wherein indicators of the same category are independent and identically distributed indicators;

[0090] A causal structure learning unit 503 is used to analyze the causal connection relationship and connection direction between each indicator based on the clustering result, and construct a causal relationship network structure, wherein the causal relationship network structure includes indicator nodes and directed edges connecting the indicator nodes, and the directed edges are used to represent the causal relationship between the connected indicator nodes;

[0091] A conditional probability table learning unit 504 is used to obtain a conditional probability table of each indicator node in the causal relationship network structure according to the time series data of the multiple indicators;

[0092] The model generation unit 505 is used to obtain a Bayesian belief network structure according to the causal relationship network structure and the conditional probability table of each indicator node to represent the causal relationship between each indicator.

[0093] In one or more embodiments of the present disclosure, when clustering the multiple indicators according to the probability distribution of the time series data of the multiple indicators, the clustering unit 502 is used to:

[0094] According to the time series data of multiple indicators, the distances between the indicators are obtained, and the correlations between the indicators in probability distribution are determined according to the distances between the indicators. According to the correlations between the indicators, an adjacency matrix is ​​obtained, and the adjacency matrix is ​​used as the clustering result of the multiple indicators.

[0095] In one or more embodiments of the present disclosure, Figure 6 As shown, the data processing device 500 also includes a discretization unit 506, which is used to discretize the time series data of each indicator after clustering the multiple indicators according to the probability distribution of the time series data of the multiple indicators and before analyzing the causal connection relationship and connection direction between the indicators based on the clustering results to obtain a discretized data set corresponding to each indicator; the discretization unit 506 can also be used to obtain the probability distribution of each indicator according to the discretized data set corresponding to each indicator.

[0096] In one or more embodiments of the present disclosure, when analyzing the causal connection relationship and connection orientation between indicators based on the clustering results and constructing the causal relationship network structure, the causal structure learning unit 503 is used to:

[0097] Based on the discretized data sets corresponding to the indicators in the adjacency matrix, the causal connection relationship and connection direction between the indicators are determined, and a causal network structure is constructed.

[0098] In one or more embodiments of the present disclosure, when determining the causal connection relationship and connection direction between the indicators, the causal structure learning unit 503 should:

[0099] A conditional independence test method is used to perform an independence test on each indicator in the adjacency matrix to determine the conditional independent indicator, and the causal connection relationship between the conditional independent indicator and other indicators is eliminated. The connection direction in the causal connection relationship between each indicator is determined according to the V-shaped structure and Meek's rule method to obtain a directed acyclic graph or a maximum ancestor graph, and the directed acyclic graph or the maximum ancestor graph is determined as the causal network structure.

[0100] In one or more embodiments of the present disclosure, the causal structure learning unit 503 determines the causal connection relationship and connection direction between the indicators based on the discretized data sets corresponding to the indicators in the adjacency matrix, and constructs the causal relationship network structure, and is used to:

[0101] Selecting the first discretized data set of each indicator in the current time window from the discretized data sets corresponding to each indicator in the adjacency matrix;

[0102] Based on the first discretized data set, the causal connection relationship and connection direction between the indicators are determined, and a causal network structure is constructed.

[0103] In one or more embodiments of the present disclosure, the causal structure learning unit 503 determines the causal connection relationship and connection direction between the indicators based on the discretized data sets corresponding to the indicators in the adjacency matrix, and constructs the causal relationship network structure, and is used to:

[0104] Selecting, from the discretized data sets corresponding to the indicators in the adjacency matrix, a second discretized data set of at least one first indicator in a previous time window and a third discretized data set of at least one second indicator in a current time window;

[0105] Based on the second discretized data set and the third discretized data set, the causal connection relationship and connection direction between each first indicator of the previous time window and each second indicator of the current time window are determined to construct a causal relationship network structure.

[0106] In one or more embodiments of the present disclosure, when acquiring the conditional probability table of each indicator node in the causal relationship network structure, the conditional probability table learning unit 504 is used to:

[0107] For any indicator node that has a parent indicator node, obtain the conditional probability of the indicator node when its parent indicator node takes each possible value, and obtain the conditional probability table of the indicator node; or

[0108] For any indicator node that does not have a parent indicator node, the probability distribution of the indicator node is obtained, and a conditional probability table of the indicator node is determined according to the probability distribution of the indicator node.

[0109] In one or more embodiments of the present disclosure, after obtaining the Bayesian belief network structure, the model generation unit 505 is further used to:

[0110] Acquire the prior knowledge of the relationship between the inference condition and any two indicators; wherein the inference condition is the value of some indicator nodes in the Bayesian belief network structure;

[0111] According to the prior knowledge and the Bayesian belief network structure, a maximum probability solution that meets the reasoning conditions is obtained, and the values ​​of another part of the indicator nodes in the Bayesian belief network structure are determined based on the maximum probability solution.

[0112] In one or more embodiments of the present disclosure, when acquiring the maximum probability solution that satisfies the reasoning condition, the model generation unit 505 may adopt a publish / subscribe mode to asynchronously acquire the maximum probability solution that satisfies the reasoning condition.

[0113] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0114] refer to Figure 7, which shows a schematic diagram of the structure of an electronic device 700 suitable for implementing the embodiment of the present disclosure, and the electronic device 700 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0115] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 to a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0116] Typically, the following devices may be connected to the I / O interface 705: input devices 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 709. The communication devices 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0117] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0118] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0119] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0120] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0121] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0122] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0123] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0124] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0125] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0126] In a first aspect, according to one or more embodiments of the present disclosure, a data processing method is provided, comprising:

[0127] Obtain time series data of multiple indicators whose causal relationships are to be analyzed;

[0128] Clustering the multiple indicators according to the probability distribution of the time series data of the multiple indicators, wherein indicators of the same category are independent and identically distributed indicators;

[0129] Analyze the causal connection relationship and connection direction between each indicator based on the clustering result, and construct a causal relationship network structure, wherein the causal relationship network structure includes indicator nodes and directed edges connecting the indicator nodes, and the directed edges are used to represent the causal relationship between the connected indicator nodes;

[0130] According to the time series data of the multiple indicators, a conditional probability table of each indicator node in the causal relationship network structure is obtained;

[0131] According to the causal relationship network structure and the conditional probability table of each indicator node, a Bayesian belief network structure is obtained to represent the causal relationship between the indicators.

[0132] According to one or more embodiments of the present disclosure, clustering the multiple indicators according to the probability distribution of the time series data of the multiple indicators includes:

[0133] According to the time series data of multiple indicators, the distances between the indicators are obtained, and the correlations between the indicators in probability distribution are determined according to the distances between the indicators. According to the correlations between the indicators, an adjacency matrix is ​​obtained, and the adjacency matrix is ​​used as the clustering result of the multiple indicators.

[0134] According to one or more embodiments of the present disclosure, after clustering the multiple indicators according to the probability distribution of the time series data of the multiple indicators, and before analyzing the causal connection relationship and connection direction between the indicators based on the clustering result, the method further includes:

[0135] The time series data of each indicator is discretized to obtain the discretized data set corresponding to each indicator, and the probability distribution of each indicator is obtained based on the discretized data set corresponding to each indicator.

[0136] According to one or more embodiments of the present disclosure, the causal connection relationship and connection direction between the indicators are analyzed based on the clustering results to construct a causal network structure, including:

[0137] Based on the discretized data sets corresponding to the indicators in the adjacency matrix, the causal connection relationship and connection direction between the indicators are determined, and a causal network structure is constructed.

[0138] According to one or more embodiments of the present disclosure, determining the causal connection relationship and connection direction between the indicators includes:

[0139] A conditional independence test method is used to perform an independence test on each indicator in the adjacency matrix to determine the conditional independent indicator, and the causal connection relationship between the conditional independent indicator and other indicators is eliminated. The connection direction in the causal connection relationship between each indicator is determined according to the V-shaped structure and Meek's rule method to obtain a directed acyclic graph or a maximum ancestor graph, and the directed acyclic graph or the maximum ancestor graph is determined as the causal network structure.

[0140] According to one or more embodiments of the present disclosure, the determining of the causal connection relationship and connection direction between the indicators based on the discretized data set corresponding to each indicator in the adjacency matrix, and constructing a causal network structure includes:

[0141] Selecting the first discretized data set of each indicator in the current time window from the discretized data sets corresponding to each indicator in the adjacency matrix;

[0142] Based on the first discretized data set, the causal connection relationship and connection direction between the indicators are determined, and a causal network structure is constructed.

[0143] According to one or more embodiments of the present disclosure, the determining of the causal connection relationship and connection direction between the indicators based on the discretized data set corresponding to each indicator in the adjacency matrix, and constructing a causal network structure includes:

[0144] Selecting, from the discretized data sets corresponding to the indicators in the adjacency matrix, a second discretized data set of at least one first indicator in a previous time window and a third discretized data set of at least one second indicator in a current time window;

[0145] Based on the second discretized data set and the third discretized data set, the causal connection relationship and connection direction between each first indicator of the previous time window and each second indicator of the current time window are determined to construct a causal relationship network structure.

[0146] According to one or more embodiments of the present disclosure, the step of obtaining a conditional probability table of each indicator node in the causal relationship network structure includes:

[0147] For any indicator node that has a parent indicator node, obtain the conditional probability of the indicator node when its parent indicator node takes each possible value, and obtain the conditional probability table of the indicator node; or

[0148] For any indicator node that does not have a parent indicator node, the probability distribution of the indicator node is obtained, and a conditional probability table of the indicator node is determined according to the probability distribution of the indicator node.

[0149] According to one or more embodiments of the present disclosure, obtaining the probability distribution of the indicator node includes:

[0150] The probability distribution of the indicator node is obtained according to the discretized data set corresponding to the indicator node.

[0151] According to one or more embodiments of the present disclosure, after obtaining the Bayesian belief network structure, the method further includes:

[0152] Acquire the prior knowledge of the relationship between the inference condition and any two indicators; wherein the inference condition is the value of some indicator nodes in the Bayesian belief network structure;

[0153] According to the prior knowledge and the Bayesian belief network structure, a maximum probability solution that meets the reasoning conditions is obtained, and the values ​​of another part of the indicator nodes in the Bayesian belief network structure are determined based on the maximum probability solution.

[0154] According to one or more embodiments of the present disclosure, obtaining a maximum probability solution that satisfies the reasoning condition includes:

[0155] The publish / subscribe mode is used to asynchronously obtain the maximum probability solution that meets the reasoning conditions.

[0156] In a second aspect, according to one or more embodiments of the present disclosure, a data processing device is provided, including:

[0157] A time series data acquisition unit, used to acquire time series data of multiple indicators whose causal relationships are to be analyzed;

[0158] A clustering unit, used to cluster the multiple indicators according to the probability distribution of the time series data of the multiple indicators, wherein the indicators of the same category are independent and identically distributed indicators;

[0159] A causal structure learning unit, used to analyze the causal connection relationship and connection direction between each indicator based on the clustering result, and construct a causal relationship network structure, wherein the causal relationship network structure includes indicator nodes and directed edges connecting the indicator nodes, wherein the directed edges are used to represent the causal relationship between the connected indicator nodes;

[0160] A conditional probability table learning unit, used to obtain a conditional probability table of each indicator node in the causal relationship network structure according to the time series data of the multiple indicators;

[0161] The model generation unit is used to obtain a Bayesian belief network structure according to the causal relationship network structure and the conditional probability table of each indicator node to represent the causal relationship between the indicators.

[0162] According to one or more embodiments of the present disclosure, when clustering the multiple indicators according to the probability distribution of the time series data of the multiple indicators, the clustering unit is used to:

[0163] According to the time series data of multiple indicators, the distances between the indicators are obtained, and the correlations between the indicators in probability distribution are determined according to the distances between the indicators. According to the correlations between the indicators, an adjacency matrix is ​​obtained, and the adjacency matrix is ​​used as the clustering result of the multiple indicators.

[0164] According to one or more embodiments of the present disclosure, the device also includes a discretization unit, which is used to discretize the time series data of each indicator after clustering the multiple indicators according to the probability distribution of the time series data of the multiple indicators and before analyzing the causal connection relationship and connection direction between the indicators based on the clustering results, to obtain a discretized data set corresponding to each indicator, and obtain the probability distribution of each indicator based on the discretized data set corresponding to each indicator.

[0165] According to one or more embodiments of the present disclosure, when the causal structure learning unit analyzes the causal connection relationship and connection orientation between the indicators based on the clustering results and constructs the causal relationship network structure, it is used to:

[0166] Based on the discretized data sets corresponding to the indicators in the adjacency matrix, the causal connection relationship and connection direction between the indicators are determined, and a causal network structure is constructed.

[0167] According to one or more embodiments of the present disclosure, when determining the causal connection relationship and connection direction between indicators, the causal structure learning unit should:

[0168] A conditional independence test method is used to perform an independence test on each indicator in the adjacency matrix to determine the conditional independent indicator, and the causal connection relationship between the conditional independent indicator and other indicators is eliminated. The connection direction in the causal connection relationship between each indicator is determined according to the V-shaped structure and Meek's rule method to obtain a directed acyclic graph or a maximum ancestor graph, and the directed acyclic graph or the maximum ancestor graph is determined as the causal network structure.

[0169] According to one or more embodiments of the present disclosure, the causal structure learning unit determines the causal connection relationship and connection direction between the indicators based on the discretized data sets corresponding to the indicators in the adjacency matrix, and constructs the causal relationship network structure, and is used to:

[0170] Selecting the first discretized data set of each indicator in the current time window from the discretized data sets corresponding to each indicator in the adjacency matrix;

[0171] Based on the first discretized data set, the causal connection relationship and connection direction between the indicators are determined, and a causal network structure is constructed.

[0172] According to one or more embodiments of the present disclosure, the causal structure learning unit determines the causal connection relationship and connection direction between the indicators based on the discretized data sets corresponding to the indicators in the adjacency matrix, and constructs the causal relationship network structure, and is used to:

[0173] Selecting, from the discretized data sets corresponding to the indicators in the adjacency matrix, a second discretized data set of at least one first indicator in a previous time window and a third discretized data set of at least one second indicator in a current time window;

[0174] Based on the second discretized data set and the third discretized data set, the causal connection relationship and connection direction between each first indicator of the previous time window and each second indicator of the current time window are determined to construct a causal relationship network structure.

[0175] According to one or more embodiments of the present disclosure, when acquiring the conditional probability table of each indicator node in the causal relationship network structure, the conditional probability table learning unit is used to:

[0176] For any indicator node that has a parent indicator node, obtain the conditional probability of the indicator node when its parent indicator node takes each possible value, and obtain the conditional probability table of the indicator node; or

[0177] For any indicator node that does not have a parent indicator node, the probability distribution of the indicator node is obtained, and a conditional probability table of the indicator node is determined according to the probability distribution of the indicator node.

[0178] According to one or more embodiments of the present disclosure, when acquiring the probability distribution of the indicator node, the conditional probability table learning unit is used to:

[0179] The probability distribution of the indicator node is obtained according to the discretized data set corresponding to the indicator node.

[0180] According to one or more embodiments of the present disclosure, after acquiring the Bayesian belief network structure, the model generation unit is further used to:

[0181] Acquire the prior knowledge of the relationship between the inference condition and any two indicators; wherein the inference condition is the value of some indicator nodes in the Bayesian belief network structure;

[0182] According to the prior knowledge and the Bayesian belief network structure, a maximum probability solution that meets the reasoning conditions is obtained, and the values ​​of another part of the indicator nodes in the Bayesian belief network structure are determined based on the maximum probability solution.

[0183] According to one or more embodiments of the present disclosure, when acquiring the maximum probability solution that satisfies the reasoning condition, the model generation unit may adopt a publish / subscribe mode to asynchronously acquire the maximum probability solution that satisfies the reasoning condition.

[0184] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0185] The memory stores computer-executable instructions;

[0186] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the data processing method described in the first aspect and various possible designs of the first aspect.

[0187] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the data processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0188] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, comprising computer execution instructions. When a processor executes the computer execution instructions, the data processing method as described in the first aspect and various possible designs of the first aspect is implemented.

[0189] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0190] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0191] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A network service data processing method, It is characterized in that include: Acquire time series data of multiple indicators of causal relationships to be analyzed in the network service through a data acquisition system; the multiple indicators include at least two of the user concurrency of the network service, the server memory usage rate, and the CPU usage rate; Clustering the multiple indicators according to the probability distribution of the time series data of the multiple indicators through a data processing device, and representing the clustering result in an adjacency matrix, wherein the adjacency matrix is ​​used to represent the correlation between the indicators in the probability distribution, wherein the indicators of the same category are independent and identically distributed indicators; Discretize the time series data of each indicator to obtain the discretized data set corresponding to each indicator; Based on the discretized data sets corresponding to the indicators in the adjacency matrix, the causal connection relationship and the connection direction between the indicators are determined, and a causal relationship network structure is constructed, wherein the causal relationship network structure includes indicator nodes and directed edges connecting the indicator nodes, and the directed edges are used to represent the causal relationship between the connected indicator nodes; According to the time series data of the multiple indicators, a conditional probability table of each indicator node in the causal relationship network structure is obtained; According to the causal relationship network structure and the conditional probability table of each indicator node, a Bayesian belief network structure is obtained to represent the causal relationship between each indicator in the network service, and a decision plan for the network service is determined according to the causal relationship between each indicator in the network service, or the root cause of the fault in the network service is determined.

2. The method according to claim 1, It is characterized in that The clustering of the multiple indicators according to the probability distribution of the time series data of the multiple indicators includes: According to the time series data of multiple indicators, the distances between the indicators are obtained, and the correlations between the indicators in probability distribution are determined according to the distances between the indicators. According to the correlations between the indicators, an adjacency matrix is ​​obtained, and the adjacency matrix is ​​used as the clustering result of the multiple indicators.

3. The method according to claim 1, It is characterized in that Determining the causal connection relationship and connection direction between the indicators includes: A conditional independence test method is used to perform an independence test on each indicator in the adjacency matrix to determine the conditional independent indicator, and the causal connection relationship between the conditional independent indicator and other indicators is eliminated. The connection direction in the causal connection relationship between each indicator is determined according to the V-shaped structure and Meek's rule method to obtain a directed acyclic graph or a maximum ancestor graph, and the directed acyclic graph or the maximum ancestor graph is determined as the causal network structure.

4. The method according to claim 3, It is characterized in that The method of determining the causal connection relationship and connection direction between the indicators based on the discretized data set corresponding to each indicator in the adjacency matrix and constructing a causal relationship network structure includes: Selecting the first discretized data set of each indicator in the current time window from the discretized data sets corresponding to each indicator in the adjacency matrix; Based on the first discretized data set, the causal connection relationship and connection direction between the indicators are determined, and a causal network structure is constructed.

5. The method according to claim 3, It is characterized in that The method of determining the causal connection relationship and connection direction between the indicators based on the discretized data set corresponding to each indicator in the adjacency matrix and constructing a causal relationship network structure includes: Selecting, from the discretized data sets corresponding to the indicators in the adjacency matrix, a second discretized data set of at least one first indicator in a previous time window and a third discretized data set of at least one second indicator in a current time window; Based on the second discretized data set and the third discretized data set, the causal connection relationship and connection direction between each first indicator of the previous time window and each second indicator of the current time window are determined to construct a causal relationship network structure.

6. The method according to claim 1, It is characterized in that The step of obtaining a conditional probability table of each indicator node in the causal relationship network structure includes: For any indicator node that has a parent indicator node, obtain the conditional probability of the indicator node when its parent indicator node takes each possible value, and obtain the conditional probability table of the indicator node; or For any indicator node that does not have a parent indicator node, the probability distribution of the indicator node is obtained, and a conditional probability table of the indicator node is determined according to the probability distribution of the indicator node.

7. The method according to claim 1, It is characterized in that After obtaining the Bayesian belief network structure, the method further includes: Acquire the prior knowledge of the relationship between the inference condition and any two indicators; wherein the inference condition is the value of some indicator nodes in the Bayesian belief network structure; According to the prior knowledge and the Bayesian belief network structure, a maximum probability solution that meets the reasoning conditions is obtained, and the values ​​of another part of the indicator nodes in the Bayesian belief network structure are determined based on the maximum probability solution.

8. The method according to claim 7, It is characterized in that The obtaining of the maximum probability solution satisfying the reasoning condition comprises: The publish / subscribe mode is used to asynchronously obtain the maximum probability solution that meets the reasoning conditions.

9. A network service data processing device, It is characterized in that include: A time series data acquisition unit, used to acquire time series data of multiple indicators of causal relationships to be analyzed in network services through a data acquisition system; The multiple indicators include at least two of the user concurrency of the network service, the server memory usage, and the CPU usage; A clustering unit, used to cluster the multiple indicators according to the probability distribution of the time series data of the multiple indicators, and represent the clustering results in an adjacency matrix, wherein the adjacency matrix is ​​used to represent the correlation between the indicators in the probability distribution, wherein the indicators of the same category are independent and identically distributed indicators; The discretization unit is used to discretize the time series data of each indicator and obtain the discretized data set corresponding to each indicator; A causal structure learning unit, used to determine the causal connection relationship and connection direction between the indicators based on the discretized data sets corresponding to the indicators in the adjacency matrix, and to construct a causal relationship network structure, wherein the causal relationship network structure includes indicator nodes and directed edges connecting the indicator nodes, and the directed edges are used to represent the causal relationship between the connected indicator nodes; A conditional probability table learning unit, used to obtain a conditional probability table of each indicator node in the causal relationship network structure according to the time series data of the multiple indicators; The model generation unit is used to obtain a Bayesian belief network structure based on the causal relationship network structure and the conditional probability table of each indicator node to represent the causal relationship between each indicator in the network service, and determine the decision-making plan of the network service or the root cause of the fault in the network service based on the causal relationship between each indicator in the network service.

10. An electronic device, It is characterized in that include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, It is characterized in that The method comprises computer-executable instructions, and when a processor executes the computer-executable instructions, the method according to any one of claims 1 to 8 is implemented.