Operation and maintenance monitoring threshold generation method and system based on quantization of quantile and rule base

Abstract

The application provides an operation and maintenance monitoring threshold generation method and system based on quantile vectorization and a rule base, comprising: defining core indicators and threshold values thereof according to user settings; generating quantile segments according to user settings, and mapping the threshold values of the core indicators into the quantile segments according to a quantile segment mapping rule; and generating threshold values of non-core indicators based on the correlation between the quantile segments. Through the mapping rule of the quantile segments, the recommended threshold values of the remaining non-core indicators that have not been configured can be conveniently obtained from the threshold values of the core indicators that have been manually configured, the explainability, reliability and configuration efficiency are taken into account, and the cold start problem is solved.

Description

Technical Field

[0001] This invention relates to the field of operation and maintenance technology, and more specifically, to a method and system for generating operation and maintenance monitoring thresholds based on quantile vectorization and rule base. Background Technology

[0002] Modern information systems contain a vast number of monitoring metrics, a trend that has become even more pronounced with the advent of cloud computing and microservices. Taking an information system with one hundred nodes as an example, each networked node contains at least several categories of monitoring metrics, including network load (such as bit rate, packet count, concurrency, and number of new connections), network availability (such as connection success rate, number of resets, retransmission rate, and zero window), and performance (such as network latency, RTT, and service response time). The total number of monitoring metrics can reach thousands.

[0003] In existing technologies, monitoring systems based on machine learning models, manual threshold setting methods, and automatic parameter setting methods are commonly used.

[0004] Machine learning-based monitoring systems can circumvent the issue of indicator setting, but they lack interpretability and reliability. For example, patent document CN110782045A discloses a method and apparatus for generating dynamic thresholds in an operation and maintenance alarm system. Based on the SARIMA model, it models the values ​​of a certain operation and maintenance monitoring indicator at different time points to form a predictive model. A threshold space is generated through a preset confidence interval, thus forming a dynamic threshold. This modeling process is continuously adjusted and optimized as the indicator data increases, and the dynamic threshold also changes with the actual application environment. However, patent document CN110782045A completely ignores human experience and cannot solve the cold start problem.

[0005] Manual threshold setting methods are interpretable and reliable, but setting thresholds requires a certain technical background and domain experience, and different information systems need to be set independently, making manual setting costly. For example, patent document CN115001942A discloses a method and system for recommending operation and maintenance monitoring thresholds, which adjusts the configuration of alarm thresholds based on the specific performance of already configured alarm thresholds to achieve the goal of alarm threshold optimization. However, patent document CN115001942A relies on all alarm thresholds having already been manually configured before proceeding with subsequent work, and cannot automatically generate thresholds that have not been manually configured, nor can it solve the cold start problem.

[0006] Some automatic parameter setting methods are based on a large number of historical indicators, but they will encounter difficult cold start problems when facing new systems. For example, patent document CN110415494A provides a method for calculating equipment alarm thresholds based on historical operating data, which includes the following steps: Step 1, collecting historical operating data of the equipment; Step 2, preprocessing the historical operating data; Step 3, calculating statistical indicators of the preprocessed historical data, including: data average and data standard deviation; Step 4, calculating the alarm threshold Alm using the data average and data standard deviation; Step 5, monitoring alarms and iteratively updating the alarm threshold using the measured status parameter S of the equipment. However, this patent document CN110415494A is not applicable to new systems. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for generating operation and maintenance monitoring thresholds based on quantile vectorization and rule base.

[0008] A method for generating operation and maintenance monitoring thresholds based on quantile vectorization and rule base provided by the present invention includes:

[0009] First step: Define the core metric (core_metric) and its threshold based on the user's settings;

[0010] The second step is to generate quantile segments based on the user's settings, and map the thresholds of core indicators to quantile segments according to the quantile_mapping_rules.

[0011] The third step: Generate the threshold of the non-core metric comm_metric based on the correlation of quantile segments.

[0012] Preferably, in the second step, the intervals corresponding to the quantiles of the core indicator thresholds are vectorized;

[0013] The second step, based on the quantile vector calculation model and the indicator rule base, is obtained through the following steps:

[0014] Step B.1: Construct a sample set based on quantiles; Step B.1 includes:

[0015] Step B.1.1: For the index values ​​in the balanced index sample set ban_metric_sample_set, calculate the N quantile points for each index value, and divide the index values ​​into N+1 segments; according to the mapping rule quantile_mapping_rules from the index value to the quantile segment, convert the index vector into a vector composed of the index of the quantile segment, and obtain the sample set quantile_sample_set based on the quantile value. The elements in the sample set based on the quantile value are denoted as quantile_sample.

[0016] Step B.2: Construct a quantile vector calculation model; Step B.2 includes:

[0017] Step B.2.1: Construct the model input layer; Construct the input layer based on the product of the length of the element quantile_sample and the number of quantile segments. The input vector stored in the input layer is denoted as input_vec.

[0018] Step B.2.2: Construct the vector coding layer; the vector coding layer is a matrix, the number of rows R of the matrix is ​​the length of the input vector input_vec stored in the input layer, and the number of columns C of the matrix is ​​the expected length of the vector after quantile encoding; the matrix product of the input layer and the vector coding layer is the input of the next layer;

[0019] Step B.2.3: Construct a complete classification model; the classification model includes the following layers: input layer, vector encoding layer, fully connected layer, and activation function output;

[0020] Step B.3: Calculate the quantile segment vector; Step B.3 includes:

[0021] Step B.3.1: Obtain the sample set quantile_sample_set based on quantile values. Each element in the sample set has the structure <index point vector, class label>. Convert the index point vector into the input vector of the classification model, and use the class label as the output of the classification model. Train the classification model. After training, each row of the vector encoding layer corresponds to a quantile segment of an index. The quantile segment is denoted as bucket_i_j, and the vector corresponding to each quantile segment is denoted as embd_vec_i_j, where i represents the i-th index and j represents the quantile corresponding to the quantile segment.

[0022] Step B.4: Construct the correlation quantile segment relationships between pairs of indicators; if there exists an indicator metric_i, divided into M quantile segments, each quantile vector contained therein is denoted as embd_vec_i_m; if there exists an indicator metric_j, divided into N quantile segments, each quantile vector contained therein is denoted as embd_vec_j_n; for a given quantile vector embd_vec_i_α of m = α, find the quantile vector embd_vec_j_β with the highest similarity to it in indicator metric_j. The quantile segments corresponding to these two vectors, embd_vec_i_α and embd_vec_j_β, constitute the correlation quantile segment relationship, denoted as {bucket_i_α, bucket_j_β}; calculate and obtain the correlation quantile segment relationships between bucket_i_α and all other indicators except this indicator, and obtain the set relevant_set composed of correlation quantile segment relationships;

[0023] Step B.5: Construct the indicator rule base, which includes: quantile_mapping_rules mapping rules for quantile segments, and relevant_set, a set of quantile segment relationships consisting of the associated quantile segment relationships of each indicator category metric_cluster.

[0024] Preferably, the balanced metric sample set ban_metric_sample_set is obtained through the following steps:

[0025] Step A.1: Obtain the metric sample set metric_sample_set; Step A.1 includes:

[0026] Step A.1.1: Set the core metric threshold (thres) corresponding to the core metric (core_metric); For the core metric (core_metric), compare the metric value in the historical log with the core metric threshold (thres) to obtain anomaly time_set (anomaly_time_set) and normal time_set (normal_time_set) with time_point as the element; the value of the normal time point metric is a normal sample, and the value of the abnormal time point metric is an abnormal sample. The labels of normal samples and abnormal samples are different, and the labels are denoted as class labels;

[0027] Step A.1.2: Construct an indicator sequence from the values ​​of core and non-core indicators in the same indicator category metric_cluster at the same time point, denoted as indicator vector (vec);

[0028] Step A.1.3: Combine the corresponding indicator vector with the class label to form an indicator sample metric_sample, and combine multiple indicator samples to form an indicator sample set metric_sample_set;

[0029] Step A.2: Balance the metric sample set metric_sample_set so that the proportion of outlier samples falls within a preset numerical range, obtaining the balanced metric sample set ban_metric_sample_set; Step A.2 includes:

[0030] Step A.2.1: Randomly select one outlier sample from the metric_sample_set, denoted as s1;

[0031] Step A.2.2: Calculate the similarity between the outlier samples in the metric sample set metric_sample_set and the metric vector of s1, and select the p samples with the highest similarity, denoted as near_s;

[0032] Step A.2.3: Generate an anomaly vector vec based on the random anomaly sample s1 and the p most similar samples near_s:

[0033] vec = (s1.vec + near_s.vec) × r, where r is a random number between 0 and 1.

[0034] s1.vec represents the index vector vec of sample s1 (generated in step A.1.2);

[0035] near_s.vec represents the index vector vec of sample near_s (generated in step A.1.2);

[0036] Repeat steps A.2.1, A.2.2, and A.2.3 until the proportion of abnormal samples falls within the preset numerical range. The balanced sample set is then denoted as ban_metric_sample_set.

[0037] Preferably, in the third step, the threshold of the non-core indicator comm_metric is obtained by deriving the vectorization result; based on the indicator rule base, based on the set relevant_set composed of the related quantile relationships therein, the related quantile segments corresponding to bucket_i_α in each indicator are found, the quantile points corresponding to the related quantile segments are found, and the values ​​of each quantile point are the threshold.

[0038] Preferably, in the second step, the segment mapping rule is obtained based on the indicator data logs accumulated on the new system.

[0039] According to the present invention, an operation and maintenance monitoring threshold generation system based on quantile vectorization and rule base is provided, comprising:

[0040] The first module: Define the core metric (core_metric) and its threshold based on the user's settings.

[0041] The second module generates quantile segments based on user settings and maps the thresholds of core metrics to quantile segments according to the quantile_mapping_rules.

[0042] The third module generates the threshold for the non-core metric comm_metric based on the correlation of quantile segments.

[0043] Preferably, in the second module, the intervals corresponding to the quantiles of the core indicator thresholds are vectorized;

[0044] The second module M, based on the quantile vector calculation model and the indicator rule base, is obtained through the following modules included in the operation and maintenance monitoring threshold generation system based on quantile vectorization and rule base:

[0045] Module MB.1: Constructs a sample set based on quantiles; Module MB.1 includes:

[0046] Module MB.1.1: For the index values ​​in the balanced index sample set ban_metric_sample_set, calculate N quantile points for each index value, dividing the index values ​​into N+1 segments; according to the mapping rule quantile_mapping_rules from index value to quantile segment, convert the index vector into a vector composed of the index of the quantile segment, to obtain the sample set quantile_sample_set based on the quantile value, and the elements in the sample set based on the quantile value are denoted as quantile_sample;

[0047] Module MB.2: Constructs the quantile vector calculation model; Module MB.2 includes:

[0048] Module MB.2.1: Construct the model input layer; Construct the input layer based on the product of the length of the element quantile_sample and the number of quantile segments, and denote the input vector stored in the input layer as input_vec;

[0049] Module MB.2.2: Construct the vector coding layer; the vector coding layer is a matrix, the number of rows R of the matrix is ​​the length of the input vector input_vec stored in the input layer, and the number of columns C of the matrix is ​​the expected length of the vector after quantile encoding; the matrix product of the input layer and the vector coding layer is the input of the next layer;

[0050] Module MB.2.3: Constructs a complete classification model; the classification model includes the following layers: input layer, vector encoding layer, fully connected layer, and activation function output;

[0051] Module MB.3: Calculates the quantile segment vector; Module MB.3 includes:

[0052] Module MB.3.1: Obtain the sample set quantile_sample_set based on quantile values. Each element in the quantile_sample_set has the structure <index point vector, class label>. Convert the index point vector into the input vector of the classification model, and use the class label as the output of the classification model. Train the classification model. After training, each row of the vector encoding layer corresponds to a quantile segment of an index. The quantile segment is denoted as bucket_i_j, and the vector corresponding to each quantile segment is denoted as embd_vec_i_j, where i represents the i-th index and j represents the quantile corresponding to the quantile segment.

[0053] Module MB.4: Constructing the correlation quantile segment relationships between pairs of indicators; if there exists an indicator metric_i, divided into M quantile segments, each quantile vector contained therein is denoted as embd_vec_i_m; if there exists an indicator metric_j, divided into N quantile segments, each quantile vector contained therein is denoted as embd_vec_j_n; for a given quantile vector embd_vec_i_α of m = α, find the quantile vector embd_vec_j_β with the highest similarity to it in indicator metric_j, the quantile segments corresponding to these two vectors embd_vec_i_α and embd_vec_j_β form the correlation quantile segment relationship, denoted as {bucket_i_α, bucket_j_β}; calculate and obtain the correlation quantile segment relationships between bucket_i_α and all other indicators except for this indicator, and obtain the set relevant_set composed of correlation quantile segment relationships;

[0054] Module MB.5: Constructs the indicator rule base, which includes: quantile_mapping_rules mapping rules for quantile segments, and a set of quantile segment relationships consisting of relevant_set, which is a set of associated quantile segment relationships for each indicator category metric_cluster.

[0055] Preferably, the balanced indicator sample set ban_metric_sample_set is obtained through the following modules of the operation and maintenance monitoring threshold generation system based on quantile vectorization and rule base:

[0056] Module MA.1: Obtains the metric sample set metric_sample_set; Module MA.1 includes:

[0057] Module MA.1.1: Sets the core metric threshold threshold (thres) corresponding to the core metric (core_metric); for the core metric (core_metric), compares the metric value in the historical log with the core metric threshold (thres) to obtain anomaly time_set (anomaly_time_set) and normal time_set (normal_time_set) with time_point as the element; the value of the normal time point metric is a normal sample, and the value of the abnormal time point metric is an abnormal sample. The labels of normal samples and abnormal samples are different, and the labels are denoted as class labels;

[0058] Module MA.1.2: It constructs an indicator sequence, denoted as an indicator vector (vec), by taking the values ​​of core and non-core indicators in the same indicator category metric_cluster at the same time point.

[0059] Module MA.1.3: Combine the corresponding indicator vector with the class label to form an indicator sample metric_sample, and combine multiple indicator samples to form an indicator sample set metric_sample_set;

[0060] Module MA.2: Balances the metric sample set metric_sample_set so that the proportion of outlier samples falls within a preset numerical range, resulting in a balanced metric sample set ban_metric_sample_set; Module MA.2 includes:

[0061] Module MA.2.1: Randomly select one outlier sample from the metric_sample_set, denoted as s1;

[0062] Module MA.2.2: Calculate the similarity between the outlier samples in the metric sample set metric_sample_set and the metric vector of s1, and select the p samples with the highest similarity, denoted as near_s;

[0063] Module MA.2.3: Generates anomaly vector vec based on random anomaly sample s1 and p most similar samples near_s.

[0064] vec = (s1.vec + near_s.vec) × r, where r is a random number between 0 and 1.

[0065] s1.vec represents the index vector vec of sample s1 (generated in module MA.1.2);

[0066] near_s.vec represents the index vector vec of sample near_s (generated in module MA.1.2);

[0067] Repeatedly trigger the execution of modules MA.2.1, MA.2.2, and MA.2.3 until the proportion of abnormal samples falls within the preset numerical range. The balanced sample set is then denoted as ban_metric_sample_set.

[0068] Preferably, in the third module, the threshold of the non-core indicator comm_metric is obtained by deriving the vectorization result; based on the indicator rule base, based on the set relevant_set composed of the related quantile relationships therein, the related quantile segments corresponding to bucket_i_α in each indicator are found, the quantile points corresponding to the related quantile segments are found, and the values ​​of each quantile point are the threshold.

[0069] Preferably, in the second module, the segment mapping rule is obtained based on the indicator data logs accumulated on the new system.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] 1. This invention, through the mapping rules of quantile segments, can conveniently obtain the recommended thresholds of other unconfigured non-core indicators from the thresholds of core indicators that have been manually configured, thereby reducing the human cost of configuration.

[0072] 2. Since the core indicators of this invention are manually configured, and the rest are calculated using the manually configured core indicators, it balances interpretability, credibility, and configuration efficiency.

[0073] 3. This invention solves the cold start problem by using empirical quantile segment mapping or constructing quantile mapping rules based on a small number of logs. Attached Figure Description

[0074] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0075] Figure 1 This is a schematic diagram of the classification model.

[0076] Figure 2 A schematic diagram of the process steps to obtain a balanced sample set.

[0077] Figure 3 A flowchart illustrating the steps involved in obtaining the indicator rule base.

[0078] Figure 4 A schematic diagram of the process steps for generating the threshold. Detailed Implementation

[0079] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0080] This invention proposes a method for setting operation and maintenance monitoring thresholds based on quantile vectorization and rule base. This method vectorizes the intervals corresponding to the quantiles (statistical concepts) of the thresholds, and derives the thresholds of non-core indicators based on the manually set thresholds of core indicators through the vectorization results.

[0081] The solution of this invention can complete data preparation on an experimental system. When facing a new system, it can recommend the configuration of thresholds for other non-core indicators based on a small number of manually configured core thresholds, effectively solving the cold start problem. This invention has both the credibility and interpretability of manually set thresholds, and, based on vectorized quantile relationships, it enables the promotion and application of human knowledge across different indicators and systems, significantly reducing configuration costs.

[0082] The method for setting operation and maintenance monitoring thresholds based on quantile vectorization and rule base includes the following steps:

[0083] Step 1: Construct indicator classifications and "indicator-label" sample sets;

[0084] Step 2: Construct a rule base for indicators based on quantiles;

[0085] Step 3: Generate monitoring thresholds based on the indicator rule base;

[0086] Step 1 includes the following steps:

[0087] Step 1.1: Based on user instructions, construct metric categories and core metrics for each category. Within each category, all metrics except the core metrics are non-core metrics. Metrics reflecting the same type of problem should be grouped together. For example, but not limited to, metrics can be divided into three categories: network load (core metrics such as bit rate), network function availability (core metrics such as connection success rate), and performance (core metrics such as network latency). Each category can have more than one core metric, but it must be significantly less than the total number of metrics in that category. Denote each metric category as `metric_cluster`, and the set of core metrics within each category as `core_metric_set`. Each core metric is designated as a `core_metric`. For example, `metric_cluster_i` represents the i-th metric category, `core_metric_set_i` represents the i-th core metric set, and `core_metric_i` represents the i-th core metric. The metric includes the core metric `core_metric` and the non-core metric `comm_metric`.

[0088] Step 1.2: Set core indicator thresholds and process the experimental system log data based on these thresholds to obtain indicator samples. The experimental system refers to the simulation data generation system constructed to achieve the objectives of this invention. The indicator setting and processing process can be illustrated as follows:

[0089] Within the metric category metric_cluster_1, there is a core metric core_metric_1. Its alarm threshold is set by the user according to experience as thres_1. When the core metric core_metric_1 > the core metric threshold thre_1, it is an abnormal metric (it is also possible that the core metric core_metric_1 < the core metric threshold thre_1 is abnormal; this is just one example).

[0090] The historical logs corresponding to the core metric `core_metric_1` are processed using the above rules to obtain anomaly time set `anomaly_time_set` and normal time set `normal_time_set`. Each element in either set is a time point, denoted as `time_point`. For the k-th `time_point_k`, the value sequence of the corresponding non-core metric (denoted as `comm_metric`) can be obtained, for example...<comm_metric_1k,comm_metric_2k…comm_metric_nk> Where comm_metric_ik represents the i-th non-core metric corresponding to the k-th time point (time_point_k).

[0091] The values ​​of core and non-core metrics together constitute the metric sequence of metric_cluster_1 at time_point_k.<core_metric_1k,core_metric_2k,…core_metric_mk,comm_metric_1k,comm_metric_2k…comm_metric_nk> Where core_metric_ik represents the i-th core metric corresponding to the k-th time point (time_point_k).

[0092] If the labels for normal samples and abnormal samples are 0 and 1 respectively, and time_point_k is an abnormal time, then the index sample for time1 at a certain time is:

[0093] [<core_metric_1k,core_metric_2k,…core_metric_mk,comm_metric_1k,comm_metric_2k…comm_metric_nk> ,1]

[0094] The left element<core_metric_1k,core_metric_2k,…core_metric_mk,comm_metric_1k,comm_metric_2k…comm_metric_nk> The "metric vector" (vec) of the metric sample is called the "class label" of the metric sample. The set of sample points similar to the example above is the metric sample set corresponding to the metric category metric_cluster, denoted as metric_sample_set;

[0095] Step 1.3: Generate a balanced metric sample set. In the metric sample set `metric_sample_set`, the proportion of outlier samples (i.e., those with a class label of 1) may be too low, causing an imbalance between 0 and 1 samples. Use the following method to generate outlier samples and construct a balanced sample set:

[0096] Step 1.3.1: Randomly select one outlier sample from the metric_sample_set, denoted as s1;

[0097] Step 1.3.2: Calculate the similarity between the outlier samples in the metric_sample_set and the metric vector of s1. Use the cosine vector method to calculate the similarity and select the p samples with the highest similarity.

[0098] Step 1.3.3: Generate anomaly vector vec based on s1 and the p most similar samples (denoted as near_s):

[0099] vec = (s1.vec + near_s.vec) × r, where r is a random number between 0 and 1.

[0100] s1.vec represents the index vector vec of sample s1 (generated in step 1.2);

[0101] near_s.vec represents the index vector vec of sample near_s (generated in step 1.2);

[0102] Repeat steps 1.3.1-1.3.3 until the ratio of normal samples to abnormal samples is close, at which point the samples are considered balanced. A close ratio means falling within the range of 3:4 to 4:3, with possible values ​​of 3:4, 5:5, or 4:3. Denote the balanced sample set as ban_metric_sample_set.

[0103] Step 2 includes the following steps:

[0104] Step 2.1: Construct a quantile-based sample set. Based on the experimental system data, calculate N quantile points for each indicator. For example, if N is 9, the 9 quantile points are [0.1 quantile, 0.2 quantile, 0.3 quantile…0.9 quantile]. A quantile is a standard statistical concept; the calculation method refers to relevant statistical materials. The 0.5 quantile is the median. For example, according to the formula, based on the value of a certain indicator metric_1, we can obtain: Quantile_0.1 = 0.027, Quantile_0.2 = 0.12…Quantile_0.9 = 0.832. Quantile_x represents the x-quantile.

[0105] Based on these 9 quantiles, the values ​​of the metric_1 can be divided into 10 segments. For example, a value of 0.11 can be assigned to the segment corresponding to Quantile_0.1. If the segment is counted from 1, it corresponds to segment 2. The mapping rule from metric to quantile segment is denoted as quantile_mapping_rules. Following quantile_mapping_rules, the metric vector in the above example is converted into a vector based on the index of its respective quantile segment, for example:

[0106] [<1,5,3,8…7>,1]

[0107] [<2,3,2,7…5>,0]

[0108] [<5,8,6,2…8>,0]

[0109] The numbers in the left element <1,5,3,8…7> of [<1,5,3,8…7>,1] represent the 1st, 5th, 3rd, 8th, and 7th quantile segments, respectively.

[0110] The balanced sample set ban_metric_sample_set generated in step 1 is used to calculate the sample set based on quantile values ​​using the above method, denoted as quantile_sample_set, where each element in the sample set based on quantile values ​​is denoted as quantile_sample.

[0111] Step 2.2: Construct a quantile vector calculation model.

[0112] Step 2.2.1: Construct the model input layer. The input layer is constructed based on the product of the length of `quantile_sample` and the number of quantile segments. The input vector stored in the input layer is denoted as `input_vec`. If the index vector length of the input vector `quantile_sample` stored in the input layer is 20 (meaning there are 20 indices, each divided into 10 quantiles), then the length of the model's input vector is 20 × 10 = 200. If `quantile_sample` exists with an index vector of <5,2…>, then in the first 20 bits of the input vector, the 5th and 12th bits are set to 1, and the rest are set to 0, as shown below:

[0113] [0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0…];

[0114] Step 2.2.2: Construct the vector coding layer. The vector coding layer is a matrix. The number of rows R of the matrix is ​​the length of the input vector input_vec stored in the input layer, and the number of columns C is the expected length of the vector after quantile encoding. For example, if the expected length is 32, then C = 32. The matrix product of the input layer and the vector coding layer is the input of the next layer.

[0115] Step 2.2.3: Construct a complete classification model. Based on steps 2.2.1 and 2.2.2, construct a complete classification neural network model. The model layers have the following structure, for example, but not limited to: [input layer, vector encoding layer, fully connected layer 1, fully connected layer 2, fully connected layer 3, activation function output], as follows. Figure 1 As shown;

[0116] Step 2.3: Calculate the quantile segment vector. Each element in the quantile_sample_set obtained in Step 2.1 has the structure <index point vector, class label>. Convert its index point vector into the input vector of the classification model using the method described in Step 2.2.1, and use its class label as the output of the classification model. Train the classification model constructed in Step 2.2 based on the above data. After training, each row of the vector encoding layer corresponds to a quantile segment of an index. Denote the quantile segment as bucket_i_j, and the vector corresponding to each quantile segment as embd_vec_i_j, where i represents the i-th index, and j represents the quantile number corresponding to the quantile segment. If each indicator is divided into 10 quantile segments, the second row of the matrix corresponds to the vector of the second quantile segment of the first indicator, which can be denoted as bucket_1_0.1 and the corresponding vector is embd_vec_1_0.1; the 16th row of the matrix corresponds to the vector of the sixth quantile segment of the second indicator, which can be denoted as bucket_2_0.6 and the corresponding vector is embd_vec_2_0.6.

[0117] Step 2.4: Construct the correlation quantile segment relationships between pairs of metrics. If there exists a metric_i, divided into M quantile segments, each quantile vector is denoted as embd_vec_i_m; if there exists a metric_j, divided into N quantile segments, each quantile vector is denoted as embd_vec_j_n. For a given quantile vector embd_vec_i_α with m = α, the vector embd_vec_j_β with the highest similarity to it can be found in metric_j. Similarity calculation can use, but is not limited to, cosine similarity calculation. The quantile segments corresponding to the above two vectors constitute the correlation quantile segment relationship, denoted as {bucket_i_α, bucket_j_β}. bucket_i_α needs to be correlated with all other metrics except itself to obtain the correlation quantile segment relationship. The set of correlation quantile segment relationships is denoted as relevant_set. Calculate the relevant_set for each metric_cluster using the above method.

[0118] Step 2.5: Construct the metric rule base. The metric rule base consists of two parts: "quantile segment mapping" and "quantile segment relationship set". The empirical quantile segment mapping rules are quantile_mapping_rules (generated in step 2.1), and the quantile segment relationship set is the set composed of the relevant_set of each metric_cluster.

[0119] Step 3 includes the following steps:

[0120] Step 3.1: The core metrics for each metric_cluster are configured manually;

[0121] Step 3.2: Determine the quantile mapping rules. The quantile mapping rules can come from: 1) using the empirical quantile mapping rules generated in Step 2; 2) accumulating a small amount of indicator data logs on the new system, such as, but not limited to, 1-3 days of indicator operation logs, and calculating the quantile mapping rules for that system;

[0122] Step 3.3: Generate quantile segments for manually set values. Based on the quantile segment mapping rules determined in Step 3.2, map the manually set core indicator numbers to quantile segments, such as bucket_i_α;

[0123] Step 3.4: Generate thresholds based on associated quantile segments. Based on the rule base generated in Step 2, and using the relevant_set, find the associated quantile segments corresponding to bucket_i_α in each indicator: bucket_i_α1, bucket_i_α2…bucket_i_α n Find the quantile points corresponding to all the above-mentioned associated quantile segments, such as Quantile_1_0.2 = 0.032, Quantile_2_0.3 = 0.75, etc. The value of each quantile point is the generated threshold.

[0124] This invention also provides a quantile vectorization and rule base-based operation and maintenance monitoring threshold generation system. Those skilled in the art can implement the quantile vectorization and rule base-based operation and maintenance monitoring threshold generation system by executing the process steps of the method. That is, the method can be understood as a preferred embodiment of the quantile vectorization and rule base-based operation and maintenance monitoring threshold generation system. Specifically, according to this invention, a quantile vectorization and rule base-based operation and maintenance monitoring threshold generation system includes:

[0125] The first module: Define the core metric (core_metric) and its threshold based on the user's settings.

[0126] The second module generates quantile segments based on user settings and maps the thresholds of core indicators to quantile segments according to the quantile_mapping_rules. In the second module, the quantile segment mapping rules are obtained based on the indicator data logs accumulated on the new system.

[0127] The third module generates the threshold for the non-core metric *comm_metric* based on the correlation quantile relationships. In this module, the threshold for *comm_metric* is derived through vectorization results. Based on the metric rule base and the set *relevant_set* composed of correlation quantile relationships, the correlation quantile segments corresponding to *bucket_i_α* in each metric are found, and the quantile points corresponding to these segments are identified. The values ​​of each quantile point are the threshold values.

[0128] Preferably, in the second module, the intervals corresponding to the quantiles of the core indicator thresholds are vectorized;

[0129] The second module M, based on the quantile vector calculation model and the indicator rule base, is obtained through the following modules included in the operation and maintenance monitoring threshold generation system based on quantile vectorization and rule base:

[0130] Module MB.1: Constructs a sample set based on quantiles; Module MB.1 includes:

[0131] Module MB.1.1: For the index values ​​in the balanced index sample set ban_metric_sample_set, calculate N quantile points for each index value, dividing the index values ​​into N+1 segments; according to the mapping rule quantile_mapping_rules from index value to quantile segment, convert the index vector into a vector composed of the index of the quantile segment, to obtain the sample set quantile_sample_set based on the quantile value, and the elements in the sample set based on the quantile value are denoted as quantile_sample;

[0132] Module MB.2: Constructs the quantile vector calculation model; Module MB.2 includes:

[0133] Module MB.2.1: Construct the model input layer; Construct the input layer based on the product of the length of the element quantile_sample and the number of quantile segments, and denote the input vector stored in the input layer as input_vec;

[0134] Module MB.2.2: Construct the vector coding layer; the vector coding layer is a matrix, the number of rows R of the matrix is ​​the length of the input vector input_vec stored in the input layer, and the number of columns C of the matrix is ​​the expected length of the vector after quantile encoding; the matrix product of the input layer and the vector coding layer is the input of the next layer;

[0135] Module MB.2.3: Constructs a complete classification model; the classification model includes the following layers: input layer, vector encoding layer, fully connected layer, and activation function output;

[0136] Module MB.3: Calculates the quantile segment vector; Module MB.3 includes:

[0137] Module MB.3.1: Obtain the sample set quantile_sample_set based on quantile values. Each element in the quantile_sample_set has the structure <index point vector, class label>. Convert the index point vector into the input vector of the classification model, and use the class label as the output of the classification model. Train the classification model. After training, each row of the vector encoding layer corresponds to a quantile segment of an index. The quantile segment is denoted as bucket_i_j, and the vector corresponding to each quantile segment is denoted as embd_vec_i_j, where i represents the i-th index and j represents the quantile corresponding to the quantile segment.

[0138] Module MB.4: Constructing the correlation quantile segment relationships between pairs of indicators; if there exists an indicator metric_i, divided into M quantile segments, each quantile vector contained therein is denoted as embd_vec_i_m; if there exists an indicator metric_j, divided into N quantile segments, each quantile vector contained therein is denoted as embd_vec_j_n; for a given quantile vector embd_vec_i_α of m = α, find the quantile vector embd_vec_j_β with the highest similarity to it in indicator metric_j, the quantile segments corresponding to these two vectors embd_vec_i_α and embd_vec_j_β form the correlation quantile segment relationship, denoted as {bucket_i_α, bucket_j_β}; calculate and obtain the correlation quantile segment relationships between bucket_i_α and all other indicators except for this indicator, and obtain the set relevant_set composed of correlation quantile segment relationships;

[0139] Module MB.5: Constructs the indicator rule base, which includes: quantile_mapping_rules mapping rules for quantile segments, and a set of quantile segment relationships consisting of relevant_set, which is a set of associated quantile segment relationships for each indicator category metric_cluster.

[0140] Preferably, the balanced indicator sample set ban_metric_sample_set is obtained through the following modules of the operation and maintenance monitoring threshold generation system based on quantile vectorization and rule base:

[0141] Module MA.1: Obtains the metric sample set metric_sample_set; Module MA.1 includes:

[0142] Module MA.1.1: Sets the core metric threshold threshold (thres) corresponding to the core metric (core_metric); for the core metric (core_metric), compares the metric value in the historical log with the core metric threshold (thres) to obtain anomaly time_set (anomaly_time_set) and normal time_set (normal_time_set) with time_point as the element; the value of the normal time point metric is a normal sample, and the value of the abnormal time point metric is an abnormal sample. The labels of normal samples and abnormal samples are different, and the labels are denoted as class labels;

[0143] Module MA.1.2: It constructs an indicator sequence, denoted as an indicator vector (vec), by taking the values ​​of core and non-core indicators in the same indicator category metric_cluster at the same time point.

[0144] Module MA.1.3: Combine the corresponding indicator vector with the class label to form an indicator sample metric_sample, and combine multiple indicator samples to form an indicator sample set metric_sample_set;

[0145] Module MA.2: Balances the metric sample set metric_sample_set so that the proportion of outlier samples falls within a preset numerical range, resulting in a balanced metric sample set ban_metric_sample_set; Module MA.2 includes:

[0146] Module MA.2.1: Randomly select one outlier sample from the metric_sample_set, denoted as s1;

[0147] Module MA.2.2: Calculate the similarity between the outlier samples in the metric sample set metric_sample_set and the metric vector of s1, and select the p samples with the highest similarity, denoted as near_s;

[0148] Module MA.2.3: Generates anomaly vector vec based on random anomaly sample s1 and p most similar samples near_s.

[0149] vec = (s1.vec + near_s.vec) × r, where r is a random number between 0 and 1.

[0150] s1.vec represents the index vector vec of sample s1 (generated by module MA.1.2);

[0151] near_s.vec represents the index vector vec of sample near_s (generated by module MA.1.2);

[0152] Repeatedly trigger the execution of modules MA.2.1, MA.2.2, and MA.2.3 until the proportion of abnormal samples falls within the preset numerical range. The balanced sample set is then denoted as ban_metric_sample_set.

[0153] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0154] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A threshold generation method for operation and maintenance monitoring based on quantization of quantile and rule base, characterized in that, Comprise: The first step: according to the user's settings, define the core metric core_metric, define the threshold of the core metric core_metric; The second step: according to the user's settings, generate quantile segment, according to the quantile mapping rule quantile_mapping_rules, map the threshold of the core metric to the quantile segment; The third step: generate the threshold of the non-core metric comm_metric based on the associated quantile segment relationship; In the second step, the interval corresponding to the quantile number of the threshold of the core metric is vectorized; The second step is based on the quantile point vector calculation model and the index rule library, and is obtained by the following steps: Step B.1: construct a sample set based on quantile number; Step B.2: construct a quantile point vector calculation model; Step B.3: calculate the quantile segment vector; Step B.4: construct the associated quantile segment relationship between two indicators; if there is an index metric_i, it is divided into M quantile segments, and each quantile vector included is denoted as embd_vec_i_m; if there is an index metric_j, it is divided into N quantile segments, and each quantile vector included is denoted as embd_vec_j_n; for a given quantile vector embd_vec_i_α of m=α, find the quantile vector embd_vec_j_β with the highest similarity in the index metric_j, and the quantile segments corresponding to the two vectors embd_vec_i_α and embd_vec_j_β constitute the associated quantile segment relationship, denoted as {bucket_i_α, bucket_j_β}; bucket_i_α and all other indicators except the current indicator are calculated to obtain the associated quantile segment relationship, and the set of associated quantile segment relationships relevant_set is obtained; Step B.5: construct the index rule library, which includes: the mapping rule quantile_mapping_rules of the quantile segment, and the set of quantile segment relationships composed of the set relevant_set of the associated quantile segment relationships of each index cluster metric_cluster. 2.The threshold generation method based on quantile vectorization and rule base for operation and maintenance monitoring according to claim 1, characterized in that, The step B.1 includes: Step B.1.1: for the index value in the balanced index sample set ban_metric_sample_set, calculate the N quantile points Quantile of each index value, and divide the index value into N+1 segments; according to the mapping rule quantile_mapping_rules of the index metric value to the quantile segment, the index vector is converted into a vector composed of the serial number based on the quantile segment, and a sample set quantile_sample_set based on the sample representation of the quantile value is obtained, and the elements in the sample set quantile_sample_set based on the sample representation of the quantile value are denoted as quantile_sample; The step B.2 includes: Step B.2.1: constructing the input layer of the model; the input layer is constructed based on the product of the length of the element quantile_sample and the number of quantile segments, and the input vector stored in the input layer is denoted as input_vec; Step B.2.2: constructing the vector encoding layer; the vector encoding layer is a matrix, the number of rows R of the matrix is the length of the input vector input_vec stored in the input layer, and the number of columns C of the matrix is the expected vector length after quantile number encoding; the matrix product of the input layer and the vector encoding layer is the input of the next layer; Step B.2.3: constructing the complete classification model; the layers of the classification model include: the input layer, the vector encoding layer, the fully connected layer, and the activation function output; The step B.3 includes: Step B.3.1: obtaining the structure of each element in the sample set quantile_sample_set based on the sample representation based on the quantile value is <indicator point vector, class label>, the indicator point vector is converted to the input vector of the classification model, the class label is taken as the output of the classification model, the classification model is trained, after the training is completed, each row of the vector encoding layer corresponds to a quantile segment of an indicator; the quantile segment is denoted as bucket_i_j, and the vector corresponding to each quantile segment is denoted as embd_vec_i_j, wherein i represents the i-th indicator, and j represents the quantile number corresponding to the quantile segment. 3.The threshold generation method based on quantile vectorization and rule base for operation and maintenance monitoring according to claim 1, characterized in that, The balanced indicator sample set ban_metric_sample_set is obtained by the following steps: Step A.1: obtaining the indicator sample set metric_sample_set; the step A.1 includes: Step A.1.1: setting the core indicator threshold thres corresponding to the core indicator core_metric; for the core indicator core_metric, the indicator values in the historical log are compared with the core indicator threshold thres to obtain an abnormal time point set anomaly_time_set and a normal time point set normal_time_set with time points time_point as elements; the value of the normal time point indicator is a normal sample, the value of the abnormal time point indicator is an abnormal sample, the normal sample label and the abnormal sample label are different, and the label is denoted as class label; Step A.1.2: constructing an indicator sequence from the values of the core indicators and non-core indicators in the same indicator cluster metric_cluster at the same time point, denoted as indicator vector vec; Step A.1.3: constructing an indicator sample metric_sample from the corresponding indicator vector and the class label, and constructing an indicator sample set metric_sample_set from multiple indicator samples; Step A.2: balancing the samples of the indicator sample set metric_sample_set so that the proportion of abnormal samples falls within a preset numerical interval, to obtain the balanced indicator sample set ban_metric_sample_set; the step A.2 includes: Step A.2.1: Randomly select 1 abnormal sample from the metric sample set metric_sample_set, denoted as s1; Step A.2.2: Calculate the similarity of the index vector of the abnormal sample in the metric sample set metric_sample_set and s1, and take the p samples with the highest similarity from them, denoted as near_s; Step A.2.3: Based on the random abnormal sample s1 and the p samples with the highest similarity near_s, generate the abnormal sample index vector vec: vec = (s1.vec + near_s.vec) x r, r is a random number between 0 and 1 s1.vec represents the index vector vec of sample s1; near_s.vec represents the index vector vec of sample near_s; repeat step A.2.1, step A.2.2, step A.2.3 until the proportion of abnormal samples falls within the preset numerical interval, and record the balanced sample set as ban_metric_sample_set.

4. The threshold generation method based on quantile vectorization and rule base for operation and maintenance monitoring according to claim 3, characterized in that, In the third step, the threshold of the non-core indicator comm_metric is obtained by vectorizing the result; based on the index rule library, based on the set relevant_set composed of the associated quantile segment relationship, find the corresponding associated quantile segment of bucket_i_α in each indicator, find the quantile point corresponding to the associated quantile segment, and the numerical value of each quantile point is the threshold.

5. The threshold generation method based on quantile vectorization and rule base for operation and maintenance monitoring according to claim 1, characterized in that, In the second step, the quantile segment mapping rule is obtained based on the accumulated index data log on the new system. 6.A threshold generation system for operation and maintenance monitoring based on quantization of quantile and rule base, characterized in that, It includes: First module: according to the user's setting, define the core indicator core_metric, define the threshold of the core indicator core_metric; Second module: generate quantile segment according to user's setting, map the threshold of the core indicator to the quantile segment according to the quantile mapping rule quantile_mapping_rules; Third module: generate the threshold of the non-core indicator comm_metric based on the associated quantile segment relationship; In the second module, the interval corresponding to the quantile number of the threshold of the core indicator is vectorized; The second module is based on the quantile point vector calculation model and the index rule library, and the operation and maintenance monitoring threshold generation system based on the quantile vectorization and rule library further includes the following modules: Module MB.1: Construct a sample set based on quantile number; Module MB.2: Construct a quantile point vector calculation model; Module MB.3: Calculate the quantile segment vector; Module MB.4: Constructing the correlation bucket relationship between two indicators; if there is an indicator metric_i, it is divided into M buckets, and each bucket vector is denoted as embd_vec_i_m; if there is an indicator metric_j, it is divided into N buckets, and each bucket vector is denoted as embd_vec_j_n; for a given bucket vector embd_vec_i_α, find the bucket vector embd_vec_j_β in the indicator metric_j with the highest similarity, and the bucket corresponding to the two vectors embd_vec_i_α and embd_vec_j_β forms a correlation bucket relationship, denoted as {bucket_i_α, bucket_j_β}; bucket_i_α is calculated with all other indicators except the current indicator to obtain the correlation bucket relationship, and the set of correlation bucket relationships is denoted as relevant_set; Module MB.5: Constructing the indicator rule base, which includes: the mapping rule of the bucket quantile_mapping_rules, and the bucket relationship set composed of the set of correlation bucket relationships relevant_set of each indicator cluster metric_cluster.

7. The threshold generation system based on quantile vectorization and rule base according to claim 6, wherein, the module MB.1 includes: Module MB.1.1: For the indicator value in the balanced indicator sample set ban_metric_sample_set, calculate the N quantile points of each indicator value, and divide the indicator value into N+1 segments; according to the mapping rule of the indicator value to the bucket quantile_mapping_rules, convert the indicator vector into a vector based on the serial number of the bucket, and obtain the sample set quantile_sample_set based on the sample representation of the quantile value, wherein the element in the sample set quantile_sample_set is denoted as quantile_sample; the module MB.2 includes: Module MB.2.1: Constructing the model input layer; based on the product of the length of the element quantile_sample and the number of buckets, constructing the input layer, and the input vector stored in the input layer is denoted as input_vec; Module MB.2.2: Constructing the vector encoding layer; the vector encoding layer is a matrix, the number of rows R of the matrix is the length of the input vector input_vec stored in the input layer, and the number of columns C of the matrix is the expected length of the vector after quantile encoding; the matrix product of the input layer and the vector encoding layer is the input of the next layer; Module MB.2.3: Constructing the complete classification model; the layers of the classification model include: input layer, vector encoding layer, fully connected layer, and activation function output; the module MB.3 includes: Module MB.3.1: Obtain the structure of each element in the sample set quantile_sample_set based on the sample representation of the quantile value, convert the index point vector to the input vector of the classification model, take the class label as the output of the classification model, train the classification model, and after the training is completed, each row of the vector encoding layer corresponds to a quantile segment of an index; record the quantile segment as bucket_i_j, and record the vector corresponding to each quantile segment as embd_vec_i_j, where i represents the i-th index, and j represents the quantile number corresponding to the quantile segment. 8.The threshold generation system based on quantile vectorization and rule base for operation and maintenance monitoring according to claim 6, characterized in that, The balanced index sample set ban_metric_sample_set is obtained by the following modules included in the operation and maintenance monitoring threshold generation system based on quantile vectorization and rule base: Module MA.1: Obtain the index sample set metric_sample_set; the module MA.1 includes: Module MA.1.1: Set the core index threshold thres corresponding to the core index core_metric; for the core index core_metric, compare the index value in the historical log with the core index threshold thres to obtain an abnormal time point set anomaly_time_set and a normal time point set normal_time_set with time points time_point as elements; the value of the normal time point index is a normal sample, the value of the abnormal time point index is an abnormal sample, the normal sample label and the abnormal sample label are different, and the label is recorded as a class label; Module MA.1.2: The values of the core index and the non-core index in the same index cluster metric_cluster at the same time point are combined to form an index sequence, which is recorded as an index vector vec; Module MA.1.3: The corresponding index vector and the class label form an index sample metric_sample, and a plurality of index samples form an index sample set metric_sample_set; Module MA.2: Sample balancing is performed on the index sample set metric_sample_set so that the proportion of abnormal samples falls within a preset numerical interval, and a balanced index sample set ban_metric_sample_set is obtained; the module MA.2 includes: Module MA.2.1: Randomly select 1 abnormal sample from the index sample set metric_sample_set, and record it as s1; Module MA.2.2: Calculate the index vector similarity between the abnormal samples in the index sample set metric_sample_set and s1, and take the p samples with the highest similarity from them, and record them as near_s; Module MA.2.3: Generate an abnormal sample index vector vec based on the random abnormal sample s1 and the p samples with the highest similarity near_s: vec = (s1.vec + near_s.vec) x r, r is a random number between 0 and 1 s1.vec represents the index vector vec of sample s1; near_s.vec represents the index vector vec of sample near_s; The repeat triggering module MA.2.1, the module MA.2.2 and the module MA.2.3 are executed until the proportion of the abnormal sample falls in a preset numerical interval, and the balanced sample set is recorded as ban_metric_sample_set. 9.The threshold generation system based on quantile vectorization and rule base for operation and maintenance monitoring according to claim 8, characterized in that, In the third module, the threshold of the non-core index comm_metric is obtained through the vectorization result; based on the index rule library, based on the set relevant_set composed of the correlation quantile segment relationship, the corresponding correlation quantile segment of bucket_i_α in each index is found, the quantile point corresponding to the correlation quantile segment is found, and the numerical value of each quantile point is the threshold.

10. The threshold generation system for O&M monitoring based on quantile vectorization and rule base according to claim 6, characterized in that, In the second module, the quantile segment mapping rule is obtained based on the index data log accumulated on the new system.

Images