Periodic index determination method, device, equipment and computer readable storage medium

Abstract

The application discloses a periodic index determination method, device and equipment and a computer readable storage medium. The periodic index determination method comprises the following steps: collecting index data of a data center, and marking the collected index data to obtain marked index data; performing aggregation processing on all feature data in the marked index data to determine the deviation degree corresponding to each feature data, and adjusting each feature data according to each deviation degree to obtain a new data set; determining the test value corresponding to the new data set based on a preset time sequence algorithm, and performing logistic regression training according to the new data set to obtain an initial prediction value; and determining whether the index corresponding to the marked index data is a periodic index according to the test value and the initial prediction value. The application improves the accuracy and effectiveness of determining a periodic index.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a periodic index determination method, device, equipment and computer readable storage medium. BACKGROUND

[0002] With the rapid development of Internet technology, the system architecture is more and more complex, and the index relationship for measuring the good and bad of the system is also more and more complex. It is time-consuming and laborious to rely on human experience to sort out the mesh relationship of the index, and the system is updated and iterated frequently, so the progress of sorting cannot catch up with the real production change speed, which undoubtedly increases the difficulty of the staff maintaining the system. That is, the index relationship for measuring the good and bad of the system is more and more complex, and it is necessary to rely on human experience to sort out the mesh relationship of the index, which is time-consuming and laborious, and the system is updated and iterated frequently, so the progress of sorting cannot catch up with the real production change speed, so that the accuracy and effectiveness of the periodic index sorted out are low. SUMMARY

[0003] The main purpose of the present application is to provide a periodic index determination method, device, equipment and computer readable storage medium, which aims to solve the technical problem of how to improve the accuracy and effectiveness of determining the periodic index.

[0004] To achieve the above purpose, the present application provides a periodic index determination method, comprising the following steps:

[0005] Collecting index data of a data center, and labeling the collected index data to obtain labeled index data;

[0006] Aggregating all feature data in the labeled index data to determine the deviation degree corresponding to each feature data, and adjusting each feature data according to each deviation degree to obtain a new data set;

[0007] Determining the test value corresponding to the new data set based on a preset time series algorithm, and performing logistic regression training based on the new data set to obtain an initial prediction value;

[0008] Determining whether the index corresponding to the labeled index data is a periodic index according to the test value and the initial prediction value.

[0009] Optionally, the step of aggregating all feature data in the labeled index data to determine the deviation degree corresponding to each feature data comprises:

[0010] Extracting all feature data in the labeled index data, and aggregating and classifying each feature data to obtain all aggregated clusters;

[0011] Traverse each of the aggregated clusters, determine all the aggregated feature data in the traversed aggregated clusters, and calculate the deviation degree according to a preset deviation degree calculation formula and each of the aggregated feature data.

[0012] Optionally, the step of calculating the deviation degree according to the preset deviation degree calculation formula and each of the aggregated feature data comprises:

[0013] Traverse each of the aggregated feature data, and calculate all the average similarity values between the traversed aggregated feature data and each of the aggregated feature data.

[0014] Determine the number of aggregated cluster classes corresponding to each of the average similarity values, and input each of the average similarity values and the number of aggregated cluster classes into a preset deviation degree calculation formula for calculation to obtain a deviation degree.

[0015] Optionally, the step of calculating all the average similarity values between the traversed aggregated feature data and each of the aggregated feature data comprises:

[0016] Calculate the minimum metric distance and the maximum metric distance between a target aggregated feature data in each of the aggregated feature data and the traversed aggregated feature data.

[0017] Calculate an average metric distance according to the minimum metric distance and the maximum metric distance, and take the average metric distance as the average similarity value between the traversed aggregated feature data and the target aggregated feature data.

[0018] Optionally, the step of adjusting each of the feature data according to each of the deviation degrees to obtain a new data set comprises:

[0019] Determine the value range corresponding to each of the deviation degrees in sequence, determine the adjustment strategy of each of the deviation degrees according to each of the value ranges, adjust the feature data corresponding to each of the deviation degrees according to each of the adjustment strategies, and obtain a new data set.

[0020] Optionally, the step of determining whether the index corresponding to the labeled index data is a periodic index according to the test value and the initial prediction value comprises:

[0021] Determine a preset weight corresponding to the labeled index data, calculate a final prediction value according to the preset weight, the test value and the initial prediction value;

[0022] Determine whether the index corresponding to the labeled index data is a periodic index according to a detection result of detecting whether the final prediction value is greater than a preset value.

[0023] Optionally, the step of calculating a final prediction value according to the preset weight, the test value and the initial prediction value comprises:

[0024] calculating a first product between the preset weight and the initial prediction value, and calculating a second product between a second weight corresponding to the preset weight and the test value;

[0025] calculating a sum value of the first product and the second product, and taking the sum value as a final prediction value.

[0026] In addition, to achieve the above object, the present application also provides a periodic index determination device, comprising:

[0027] a collection unit, configured to collect index data of a data center, and label the collected index data to obtain labeled index data;

[0028] an aggregation unit, configured to perform aggregation processing on all feature data in the labeled index data to determine a deviation degree corresponding to each feature data, and adjust each feature data according to the deviation degree to obtain a new data set;

[0029] a training unit, configured to determine a test value corresponding to the new data set based on a preset time series algorithm, and perform logistic regression training according to the new data set to obtain an initial prediction value;

[0030] a determination unit, configured to determine whether an index corresponding to the labeled index data is a periodic index according to the test value and the initial prediction value.

[0031] In addition, to achieve the above object, the present application also provides a periodic index determination device, comprising a memory, a processor, and a periodic index determination program stored in the memory and executable on the processor, and the periodic index determination program implements the steps of the periodic index determination method as described above when executed by the processor.

[0032] In addition, to achieve the above object, the present application also provides a computer readable storage medium, and the computer readable storage medium stores a periodic index determination program, and the periodic index determination program implements the steps of the periodic index determination method as described above when executed by the processor.

[0033] The present application determines whether an index corresponding to the labeled index data is a periodic index by performing aggregation processing on all feature data in the collected labeled index data to obtain a deviation degree, adjusting each feature data according to each deviation degree to obtain a new data set, determining a test value and an initial prediction value of the new data set, and determining whether the index corresponding to the labeled index data is a periodic index according to the test value and the initial prediction value, thereby avoiding the phenomenon that the determination of the periodic index needs to rely on manual work in the prior art, leading to time-consuming and labor-consuming, and the accuracy and effectiveness of the combed periodic index is low, and improving the accuracy and effectiveness of the determination of the periodic index. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is a terminal structure schematic diagram of a hardware running environment related to the embodiment of the present application.

[0035] Figure 2 is a flow schematic diagram of the periodic index determination method of the first embodiment of the present application.

[0036] Figure 3 is a device unit schematic diagram of the periodic index determination device of the present application.

[0037] Figure 4 is a module flow schematic diagram in the periodic index determination method of the present application.

[0038] The object implementation, functional features and advantages of the present application will be further explained with reference to the accompanying drawings in combination with the embodiments. DETAILED DESCRIPTION

[0039] It should be understood that the specific embodiments described herein are merely intended to explain the present application and not to limit the present application.

[0040] As shown in Figure 1 , the terminal structure schematic diagram of a hardware running environment related to the embodiment of the present application. Figure 1

[0041] The terminal of the embodiment of the present application is a periodic index determination device.

[0042] As shown in Figure 1 , the terminal can include a processor 1001 such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection communication between these components. The user interface 1003 can include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 can further include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory, or a stable memory (non-volatile memory) such as a magnetic disk memory. The memory 1005 can optionally be a storage device independent of the aforementioned processor 1001.

[0043] ​Optionally, the terminal can further include a camera, RF (Radio Frequency) circuit, sensors, audio circuit, WiFi module, etc. The sensors can include, for example, light sensors, motion sensors, and other sensors. Specifically, the light sensors can include ambient light sensors and proximity sensors, wherein the ambient light sensors can adjust the brightness of the display screen according to the brightness of ambient light, and the proximity sensors can turn off the display screen and / or backlight when the terminal device is moved to the ear. Of course, the terminal device can also be configured with a gyroscope, barometer, hygrometer, thermometer, infrared sensor, and other sensors, which will not be described here.

[0044] Those skilled in the art can understand that Figure 1 The terminal structure shown in the above embodiments does not constitute a limitation on the terminal, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0045] As shown in Figure 1 The memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and a periodic index determination program.

[0046] In the terminal shown in Figure 1 The network interface 1004 is mainly used to connect to a background server and communicate data with the background server; the user interface 1003 is mainly used to connect to a client (user end) and communicate data with the client; and the processor 1001 can be used to call the periodic index determination program stored in the memory 1005 and perform the following operations:

[0047] Collecting index data of a data center and labeling the collected index data to obtain labeled index data;

[0048] Aggregating all feature data in the labeled index data to determine the deviation degree corresponding to each feature data, and adjusting each feature data according to each deviation degree to obtain a new data set;

[0049] Determining a test value corresponding to the new data set based on a preset time series algorithm, and performing logistic regression training on the new data set to obtain an initial prediction value;

[0050] Determining whether the index corresponding to the labeled index data is a periodic index according to the test value and the initial prediction value.

[0051] Referring to Figure 2 , the present application provides a periodic index determination method. In a first embodiment of the periodic index determination method, the periodic index determination method comprises the following steps:

[0052] Step S10, collect the index data of the data center, and label the collected index data to obtain labeled index data;

[0053] Since the current index analysis process is based on professional experience, the index is classified and labeled, and then statistical methods are used for feature extraction and batch calculation. The cost of manual labeling and sorting is high, and in the case of big data, the workload of manual labeling and sorting is large, which cannot meet the current demand. Therefore, in the present embodiment, the features of the index data are aggregated, and the deviation factor of the aggregated features is calculated according to the aggregation result, so as to calculate the deviation value (i.e. deviation degree) of the feature data according to the feature deviation factor, adjust the feature data according to the deviation value, obtain the adjusted data set (i.e. new data set), and perform initial training on the new data set to obtain the test value, such as the ADF (Augmented Dickey-Fuller, Unit Root Test) -Test (time) test value. At the same time, the initial prediction value is obtained by iterative training using the logistic regression model. The initial prediction value and the test value are combined to obtain the final prediction value, and the periodic index is determined according to the final prediction value. That is, in the present embodiment, the logistic regression model and the ADF-Test test value are combined to reasonably weight and adjust the model, realize the prediction of periodicity combined with the actual index attribute, and improve the accuracy of the prediction. And the present embodiment can avoid the phenomenon of strong dependence on professional experience, can extract business attributes from various business attribution systems, aggregate specific business categories, reduce manual labeling success, and improve the accuracy of periodic prediction through actual model application evaluation. That is, the periodic index determination method in the present embodiment can be as shown in Figure 4 The periodic index determination method in the present embodiment can be as shown in

[0054] Therefore, in the embodiment, data collection can be performed by the index data collection module first, that is, all business index data can be determined in the data center set in advance, all business index data can be sorted according to various business scenarios, and relatively active business index data can be selected for data sampling to obtain collected business index data. The collected business index data is subjected to data smoothing and denoising processing, and index fluctuation data caused by network fluctuation and regular abnormality is eliminated, so that the collected index data is obtained. The collected index data is labeled to obtain labeled index data. If the collected index data is a host performance type index, the label corresponding to the collected index data is an application layer. The labeled index data is index data with a label, and the number of labeled index data can be one or more.

[0055] In step S20, all feature data in the labeled index data is subjected to aggregation processing to determine a deviation degree corresponding to each feature data, and each feature data is adjusted according to the deviation degree to obtain a new data set.

[0056] When the labeled index data is obtained, all feature data in the labeled index data can be subjected to aggregation processing according to the index aggregation module, that is, the labeled index data is mapped to the interval [0-1] according to the data characteristics of the labeled index data, and principal component feature extraction (such as converting a group of variables that can have linear correlation into a group of linearly uncorrelated variables through orthogonal transformation) is performed on the mapped labeled index data to obtain all feature data. Each feature data is subjected to aggregation calculation, classified according to the aggregation dimension, and a plurality of aggregation clusters such as performance type aggregation cluster and business type aggregation cluster are obtained. The deviation degree of each aggregation cluster is calculated, that is, the deviation degree corresponding to each feature data. When calculating the deviation degree, each feature data needs to be subjected to feature dimension reduction processing to obtain reduced feature data, and then index analysis is performed on the feature data with a higher feature contribution rate to reduce the interference of high-dimensional data on model training. After obtaining each deviation degree, each feature data needs to be adjusted according to each deviation degree to obtain a new data set.

[0057] That is, each feature data is taken as a template data set D=x1, x2, …, x m , and a similarity measure function s, an aggregation cluster type number k, and a deviation degree a are determined, where c i =x i , i=1, 2, …, m, j=1, 2, …, m, and d(i, j) represents a measure distance.

[0058] The minimum measure distance formula is:

[0059]

[0060] The maximum measure distance formula is:

[0061]

[0062] The average metric distance formula is:

[0063]

[0064] That is, the minimum metric distance between each index is calculated according to the minimum metric distance formula, the maximum metric distance between each index is calculated according to the maximum metric distance formula, and the average metric distance is calculated according to the minimum metric distance and the maximum metric distance, so as to determine the average similarity s between each feature data according to the average metric distance t , t = 1, 2, … k; and the average similarity is input into the bias degree factor calculation formula to calculate the bias degree. That is:

[0065]

[0066] Then, the index bias between all indexes associated with the bias degree in the aggregated cluster is adjusted according to the bias degree. That is:

[0067] y i = x m - a t .

[0068] Where y is the adjusted index data, and the generated new data set is Θ = y1, y2, …, y

[0001] .

[0069] Step S30, determining the test value corresponding to the new data set based on the preset time series algorithm, and performing logistic regression training according to the new data set to obtain an initial prediction value;

[0070] When the new data set is obtained, the new data set also needs to be trained, that is, the time series index training set is extracted according to the index training module, that is, equal sampling can be performed according to the new data set and the labeling information of the labeled index data to obtain a training set and a validation set, wherein the proportion of the training set and the validation set is 8:2. Then, the training set is calculated according to the time series algorithm set in advance to obtain a test value. In this embodiment, the training set can be trained by an ARIMA (Autoregressive Integrated Moving Average model) model to obtain an ADF-Test test value γ. Wherein, 0 < γ < 1, and the closer γ is to 0, the more stable the determined index (such as the host index) corresponding to the labeled index data is. Wherein, the test value means to test whether the time series is stationary.

[0071] In the embodiment, the logical regression training is also needed according to the training set to obtain the initial prediction value, that is, the time series host performance data can be continuously mapped according to the logical regression algorithm, the training set data is regressed and trained to obtain a training model, and the output function of the training model is:

[0072]

[0073] wherein g(z) is the initial prediction value, and the output result of g(z) is a value between (0, 1). In step S40, whether the index corresponding to the labeled index data is a periodic index is determined according to the test value and the initial prediction value.

[0074] When the initial prediction value and the test value are obtained, the test value and the initial prediction value can be fused, that is, the test value is integrated into the logical regression algorithm for model training, and the function expression is:

[0075] f(z) = βg(z) + (1-β)γ

[0076] And since the output result of g(z) is a value between (0, 1), the closer the output value is to 1, the stronger the periodicity of the index. In the actual analysis of the index, there is a measure index, which does not have periodicity analysis value, so the weight β needs to be set in advance, and the prediction value f(z) is calculated according to the weight β. Whether the index corresponding to the labeled index data is a periodic index is determined according to the size of the prediction value f(z).

[0077] That is, in the embodiment, the index is collected from the data center first, such as collecting the index according to the day / week frequency; then the collected index is preprocessed, smoothed and denoised, the business label is extracted, and the primary index classification is performed by using business classification, such as host class, container class, and business attribute class. The feature aggregation is performed on the labeled index data, the deviation degree is calculated, the deviation of the index of each cluster class is calculated, the data is optimized, and a new data set is formed. The model is trained on the new data set to obtain the ADF-Test test value, and the data is integrated, the logical regression model is trained, the index characteristics with periodicity / non-periodicity are output, and are classified and stored. The visual analysis is performed on the classified index data, the abnormal data detection model is established for the index data with periodicity, so as to predict the next periodic data trend according to the abnormal data detection model, thereby discovering the abnormal point. For the index data without periodicity, the data is stored and reserved, and the iterative training is performed after new data is stored. If it is still determined to be non-periodic, it is marked as non-periodic, and if there is periodicity, the iterative training is continued.

[0078] In the embodiment, by aggregating all feature data in the collected labeled index data, the deviation degree is obtained, and each feature data is adjusted according to each deviation degree, a new data set is obtained, the test value and the initial prediction value of the new data set are determined, and it is determined whether the index corresponding to the labeled index data is a periodic index according to the test value and the initial prediction value, thereby avoiding the phenomenon that the existing technology needs to rely on manual determination of the periodic index, resulting in time-consuming and labor-consuming, and the accuracy and effectiveness of the sorted periodic index is low, and the accuracy and effectiveness of determining the periodic index is improved.

[0079] Further, based on the first embodiment of the present application, a second embodiment of the periodic index determination method of the present application is proposed. In the embodiment, the refinement of the step of aggregating all feature data in the labeled index data to determine the deviation degree corresponding to each feature data in the step S20 of the above embodiment includes:

[0080] Step a, extracting all feature data in the labeled index data, and aggregating and classifying each feature data to obtain all aggregated clusters;

[0081] In the embodiment, when the labeled index data is obtained, all feature data in the labeled index data needs to be extracted. The feature data is all features corresponding to the labeled index data. For example, a set of variables that may have linear correlation is converted into a set of linearly uncorrelated variables through orthogonal transformation. After obtaining all feature data, all feature data can be processed by dimension reduction, and then all feature data processed by dimension reduction is aggregated and calculated, classified according to the aggregation dimension, and a plurality of aggregated clusters such as performance type aggregated cluster, business type aggregated cluster, etc. are obtained.

[0082] Step b, traversing each aggregated cluster, determining all aggregated feature data in the traversed aggregated cluster, and calculating the deviation degree according to the preset deviation degree calculation formula and each aggregated feature data.

[0083] Each aggregated cluster is traversed in turn, all feature data contained in the traversed aggregated cluster is determined, and is used as aggregated feature data. Then, the deviation degree is calculated according to the preset deviation degree calculation formula and each aggregated feature data.

[0084] In the embodiment, by extracting all feature data in the labeled index data and performing aggregation and classification processing, all aggregated clusters are obtained, and then traversing and calculating all aggregated feature data in the traversed aggregated cluster according to the preset deviation degree calculation formula to obtain the deviation degree, so as to ensure the accuracy of the obtained deviation degree.

[0085] Specifically, the step of calculating the deviation degree according to the preset deviation degree calculation formula and each aggregated feature data includes:

[0086] Step c, calculate all average similarity values between the traversed aggregated feature data and each of the aggregated feature data;

[0087] In this embodiment, by traversing each aggregated feature data, and determining the traversed aggregated feature data, the average similarity values between the traversed aggregated feature data and all the aggregated feature data are calculated, and all the average similarity values are counted.

[0088] Step d, determine the aggregated cluster class number corresponding to each of the average similarity values, and input each of the average similarity values and the aggregated cluster class number into a preset deviation degree calculation formula for calculation to obtain a deviation degree.

[0089] The total number of all average similarity values is determined, and all the average similarity values and the total number are input into the deviation degree calculation formula set in advance for calculation to obtain the deviation degree. For example:

[0090]

[0091] Wherein, the aggregated cluster class number k, the deviation degree a, the average similarity value s t .

[0092] In this embodiment, by traversing each aggregated feature data, all average similarity values between the traversed aggregated feature data and each of the aggregated feature data are calculated, and the aggregated cluster class number corresponding to each of the average similarity values and the average similarity value are input into the deviation degree calculation formula for calculation to obtain the deviation degree, thereby ensuring the accuracy of the calculated deviation degree.

[0093] Specifically, the step of calculating all average similarity values between the traversed aggregated feature data and each of the aggregated feature data includes:

[0094] Step e, calculate the minimum metric distance and the maximum metric distance between the target aggregated feature data in each of the aggregated feature data and the traversed aggregated feature data;

[0095] In this embodiment, when calculating the average similarity value, the target aggregated feature data in each of the aggregated feature data (i.e. the data for calculation of the prepared and traversed aggregated feature data) can be determined first, then the minimum metric distance between the traversed aggregated feature data and the target aggregated feature data is calculated according to the minimum metric distance calculation formula set in advance, and the maximum metric distance between the traversed aggregated feature data and the target aggregated feature data is calculated according to the maximum metric distance formula set in advance.

[0096] Step f, calculating an average metric distance according to the minimum metric distance and the maximum metric distance, and taking the average metric distance as an average similarity value between the aggregated feature data of the traversal and the target aggregated feature data.

[0097] The minimum metric distance and the maximum metric distance are further input into a preset average metric distance formula to calculate an average metric distance. The average metric distance is taken as an average similarity value between the aggregated feature data of the traversal and the target aggregated feature data. In this embodiment, the same calculation is performed on each aggregated feature data to obtain an average similarity value between each two of the aggregated feature data.

[0098] In this embodiment, the minimum metric distance and the maximum metric distance are calculated, and the average metric distance is determined according to the minimum metric distance and the maximum metric distance and taken as the average similarity value, thereby ensuring the accuracy and effectiveness of the obtained average similarity value.

[0099] Further, the step of adjusting each feature data according to each bias degree to obtain a new data set comprises:

[0100] Step g, determining a value range corresponding to each bias degree in sequence, and determining an adjustment strategy of each bias degree according to each value range, adjusting the feature data corresponding to each bias degree according to each adjustment strategy to obtain a new data set.

[0101] In this embodiment, after obtaining the bias degree corresponding to each feature data, each bias degree can be detected in sequence to determine the value range of each bias degree. For example, if the bias degree is α, if 0 < α < 0.5, it is determined that the similarity between the indicators of the aggregated cluster is small, and the adjustment strategy can be fine-tuning the indicator distance, and fine-tuning the indicator distance has little effect on the overall distribution characteristics of the indicators, which can achieve the purpose of optimizing the training model. If 0.5 ≤ α ≤ 0.9, it is indicated that the similarity between the indicators is large, and the adjustment strategy can be to remove the interference of the similarity between the indicators by using the bias degree adjustment method. If 0.9 ≤ α < 1, it is determined that the similarity between the indicators is very high, and therefore the adjustment strategy can be to remove one of the indicators with very high similarity and retain the other for iterative operation until the clustering condition converges.

[0102] After determining the adjustment strategy corresponding to each bias degree, the feature data corresponding to each bias degree can be adjusted according to each adjustment strategy, and the adjusted feature data is taken as a new data set.

[0103] In the embodiment, the value range corresponding to each deviation degree is determined in sequence, and an adjustment strategy is determined based on each value range, so that each feature data is adjusted according to the adjustment strategy to obtain a new data set, thereby ensuring the accuracy of the obtained new data set.

[0104] Further, the step of determining whether the index corresponding to the labeled index data is a periodic index according to the test value and the initial prediction value comprises:

[0105] Step k, determining a preset weight corresponding to the labeled index data, and calculating a final prediction value according to the preset weight, the test value and the initial prediction value;

[0106] In the embodiment, the preset weight corresponding to the labeled index data is determined (the weight is set in advance by the user according to the demand, and the value range of the weight is between 0 and 1). Then, the preset weight β, the test value γ and the initial prediction value g(z) are input into the formula f(z) = βg(z) + (1-β)γ set in advance to calculate the final prediction value.

[0107] Step m, determining whether the index corresponding to the labeled index data is a periodic index according to the detection result of whether the final prediction value is greater than a preset value.

[0108] After obtaining the final prediction value, the final prediction value and the preset value (any value set in advance by the user) are compared. If the final prediction value is greater than the preset value, it is determined that the index corresponding to the labeled index data is a periodic index. If the final prediction value is less than or equal to the preset value, it is determined that the index corresponding to the labeled index data is a non-periodic index.

[0109] In the embodiment, the final prediction value is calculated according to the preset weight, the test value and the initial prediction value, and whether the index is a periodic index is determined according to the detection result of whether the final prediction value is greater than the preset value, thereby ensuring the effectiveness of the detected periodic index.

[0110] Specifically, the step of calculating the final prediction value according to the preset weight, the test value and the initial prediction value comprises:

[0111] Step n, calculating a first product between the preset weight and the initial prediction value, and calculating a second product between a second weight corresponding to the preset weight and the test value;

[0112] In the embodiment, when calculating the final prediction value, a first product of the preset weight β and the initial prediction value g(z) can be calculated first. Meanwhile, a second weight corresponding to the preset weight needs to be calculated, which is set to 1-β in the embodiment. Then, a second product of the second weight and the test value γ is calculated.

[0113] Step x, calculating a sum value of the first product and the second product, and taking the sum value as the final prediction value.

[0114] The sum value of the first product and the second product, i.e. βg(z) + (1-β)γ, is calculated again, and the sum value is directly taken as the final prediction value.

[0115] In the embodiment, the first product between the preset weight and the initial prediction value is calculated, and the second product between the second weight and the test value is calculated, and the sum value of the first product and the second product is taken as the final prediction value, so that the accuracy of the obtained final prediction value is guaranteed.

[0116] It should be noted that in the actual fault correlation analysis scenario, the data center platform can be used to extract host performance indicators, database performance indicators and interface time consumption data sets. The data is preprocessed according to the data smoothing and denoising method, and the business attribute label is labeled, and then the feature clustering is performed, the clustering deviation is calculated, and the optimized interface performance class and business class indicator data are obtained. The model is trained, the periodicity of the indicators is judged, and after the result verification and model iteration optimization, the periodicity of the indicators is 91.2%. After determining the periodicity, the business characteristics are associated, the index correlation analysis is performed, and the fault location and problem root location are assisted. For example, the host performance indicators and the interface indicators are both periodic indicators, the correlation degree of the two periodic indicators is judged according to the periodicity, the correlation coefficient is calculated, and when the host fails, the related indicators are determined according to the correlation coefficient, so as to assist the fault location.

[0117] In addition, with reference to Figure 3 , the embodiment of the present application also provides a periodicity indicator determination device, comprising:

[0118] The acquisition unit A10 is configured to acquire the indicator data of the data center, and label the acquired indicator data to obtain labeled indicator data.

[0119] The aggregation unit A20 is configured to aggregate all feature data in the labeled indicator data to determine the deviation degree corresponding to each feature data, and adjust each feature data according to the deviation degree to obtain a new data set.

[0120] The training unit A30 is configured to determine a test value corresponding to the new data set based on a preset time series algorithm, and perform a logistic regression training based on the new data set to obtain an initial prediction value.

[0121] The determination unit A40 is configured to determine whether the index corresponding to the labeled index data is a periodic index according to the test value and the initial prediction value.

[0122] Optionally, the aggregation unit A20 is configured to:

[0123] extract all feature data in the labeled index data, and perform an aggregation classification processing on each feature data to obtain all aggregation clusters;

[0124] traverse each aggregation cluster, determine all aggregation feature data in the traversed aggregation cluster, and calculate a deviation degree according to a preset deviation degree calculation formula and each aggregation feature data.

[0125] Optionally, the aggregation unit A20 is configured to:

[0126] traverse each aggregation feature data, calculate all average similarity values between the traversed aggregation feature data and each aggregation feature data;

[0127] determine a cluster class number corresponding to each average similarity value, and input each average similarity value and the cluster class number into a preset deviation degree calculation formula for calculation to obtain a deviation degree.

[0128] Optionally, the aggregation unit A20 is configured to:

[0129] calculate a minimum metric distance and a maximum metric distance between target aggregation feature data in each aggregation feature data and the traversed aggregation feature data;

[0130] calculate an average metric distance according to the minimum metric distance and the maximum metric distance, and take the average metric distance as an average similarity value between the traversed aggregation feature data and the target aggregation feature data.

[0131] Optionally, the aggregation unit A20 is configured to:

[0132] determine a value range corresponding to each deviation degree in sequence, determine an adjustment strategy of each deviation degree according to each value range, and adjust feature data corresponding to each deviation degree according to each adjustment strategy to obtain a new data set.

[0133] Optionally, the determination unit A40 is configured to:

[0134] determine a preset weight corresponding to the labeling index data, and calculate a final prediction value according to the preset weight, the test value and the initial prediction value;

[0135] determine whether the labeling index data corresponds to a periodic index according to a detection result of detecting whether the final prediction value is greater than a preset value.

[0136] Optionally, the determining unit A40 is configured to:

[0137] calculate a first product between the preset weight and the initial prediction value, and calculate a second product between a second weight corresponding to the preset weight and the test value;

[0138] calculate a sum value of the first product and the second product, and take the sum value as the final prediction value.

[0139] The steps implemented by each functional unit of the periodic index determination apparatus can refer to the steps of each embodiment of the periodic index determination method, which will not be described here again.

[0140] In addition, the present application also provides a periodic index determination apparatus, which comprises a memory, a processor and a periodic index determination program stored in the memory; the processor is configured to execute the periodic index determination program to implement the steps of each embodiment of the periodic index determination method.

[0141] The present application also provides a computer readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of each embodiment of the periodic index determination method.

[0142] The computer readable storage medium of the present application has basically the same implementation as each embodiment of the periodic index determination method, which will not be described here again.

[0143] It should be noted that, in this document, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or system. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article or system including the element.

[0144] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0145] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, can also be through hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application essentially or say the part of the prior art contribution can be embodied in the form of software products, the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disc, optical disc) as described above, including a number of instructions to make a terminal device (may be a mobile phone, computer, server, air conditioner, or network equipment, etc.) executes the method described in various embodiments of the present application.

[0146] The above is only the preferred embodiment of the present application, not therefore limit the patent scope of the present application, any equivalent structure or equivalent flow transformation made by using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A periodicity index determination method, characterized by, The periodic index determination method comprises the following steps: Collecting index data of a data center, and labeling the collected index data to obtain labeled index data; wherein the index data is host performance index, database performance index and interface time consumption, and the collected index data is labeled with a service attribute label; Extracting all feature data in the labeled index data, and performing aggregated classification processing on each feature data to obtain all aggregated clusters; wherein the labeled index data is mapped to the interval [0-1] according to the data characteristics of the labeled index data, and then principal component feature extraction is performed on the mapped labeled index data to obtain all feature data, the principal component feature extraction includes converting a group of variables that may have linear correlation into a group of linearly uncorrelated variables through orthogonal transformation; the aggregated cluster includes a performance type aggregated cluster and a service type aggregated cluster; Traverse each of the aggregated clusters, determine all the aggregated feature data in the traversed aggregated clusters, traverse each of the aggregated feature data, calculate the minimum metric distance and the maximum metric distance between the target aggregated feature data in each of the aggregated feature data and the traversed aggregated feature data; wherein the minimum metric distance formula is: The maximum metric distance formula is: Wherein each feature data is taken as a template data set D=x1, x2, …, x m And determine the similarity metric function s, the number of aggregated cluster classes k, the bias degree a, c i =x i , i=1, 2, …, m, j=1, 2, …, m, d(i, j) represents the metric distance; According to the minimum metric distance and the maximum metric distance, an average metric distance is calculated, and the average metric distance is taken as an average similarity value between the aggregate feature data of the traversal and the target aggregate feature data; wherein the average metric distance formula is: determining a number of the aggregated cluster classes corresponding to each of the average similarity values, and inputting each of the average similarity values and the number of the aggregated cluster classes into a preset deviation degree calculation formula to obtain a deviation degree; wherein the deviation degree calculation formula is: wherein the average similarity value s t ; Adjusting each feature data according to each deviation degree to obtain a new data set; Determining a test value corresponding to the new data set based on a preset time series algorithm, and performing logistic regression training according to the new data set to obtain an initial prediction value; Determining whether the index corresponding to the labeled index data is a periodic index according to the test value and the initial prediction value.

2. The periodicity indicator determination method of claim 1, wherein, The step of adjusting each feature data according to each deviation degree to obtain a new data set comprises: Determine the value range corresponding to each deviation degree in turn, and determine the adjustment strategy of each deviation degree according to each value range, adjust the feature data corresponding to each deviation degree according to each adjustment strategy to obtain a new data set.

3. The periodicity indicator determination method of any of claims 1-2, wherein, The step of determining whether the index corresponding to the labeled index data is a periodic index according to the test value and the initial prediction value comprises: Determine a preset weight corresponding to the labeled index data, calculate a final prediction value according to the preset weight, the test value and the initial prediction value; Determine whether the index corresponding to the labeled index data is a periodic index according to the detection result of whether the final prediction value is greater than a preset value.

4. The periodicity indicator determination method of claim 3, wherein, The step of calculating a final prediction value according to the preset weight, the test value and the initial prediction value comprises: Calculate the first product between the preset weight and the initial prediction value, and calculate the second product between the second weight corresponding to the preset weight and the test value; Calculate the sum value of the first product and the second product, and take the sum value as the final prediction value.

5. A periodicity index determining apparatus characterized by comprising: The periodic index determination device comprises: A collection unit configured to collect index data of a data center, and label the collected index data to obtain labeled index data; wherein the index data is host performance index, database performance index and interface time consumption, and the collected index data is labeled with a service attribute label; A polymeric unit is used to extract all feature data in the labeled index data and to perform a polymeric classification processing on each feature data to obtain all polymeric clusters; wherein the labeled index data is mapped to an interval of [0-1] according to a data characteristic of the labeled index data, and then principal component feature extraction is performed on the mapped labeled index data to obtain all feature data, the principal component feature extraction including converting a group of variables that can have linear correlation into a group of linearly uncorrelated variables through orthogonal transformation; the polymeric cluster includes a performance type polymeric cluster and a business type polymeric cluster; all polymeric feature data in the traversed polymeric cluster is determined by traversing each polymeric cluster, each polymeric feature data is traversed, and a minimum metric distance and a maximum metric distance between target polymeric feature data in each polymeric feature data and the traversed polymeric feature data are calculated; wherein the minimum metric distance formula is: The maximum metric distance formula is: wherein each feature data is taken as a template data set D=x1, x2, …, x m , and a similarity metric function s, a polymeric cluster type number k, a bias degree a, and c i =x i , i=1, 2, …, m, j=1, 2, …, m, and d(i,j) represents a metric distance; an average metric distance is calculated according to the minimum metric distance and the maximum metric distance, and the average metric distance is taken as an average similarity value between the traversed polymeric feature data and the target polymeric feature data; wherein the average metric distance formula is: A polymeric cluster type number corresponding to each average similarity value is determined, and each average similarity value and the polymeric cluster type number are input into a preset bias degree calculation formula to obtain a bias degree; wherein the bias degree calculation formula is: wherein the average similarity value s t ; each feature data is adjusted according to each bias degree to obtain a new data set; A training unit configured to determine a test value corresponding to the new data set based on a preset time series algorithm, and perform logistic regression training according to the new data set to obtain an initial prediction value; The determining unit is configured to determine whether the index corresponding to the labeling index data is a periodic index according to the check value and the initial prediction value.

6. A periodicity index determining device characterized by comprising: The periodic index determination device includes a memory, a processor, and a periodic index determination program stored on the memory and executable on the processor, and the periodic index determination program, when executed by the processor, implements the steps of the periodic index determination method of any one of claims 1 to 4.

7. A computer readable storage medium characterized by The computer readable storage medium stores a periodic index determination program, and the periodic index determination program, when executed by the processor, implements the steps of the periodic index determination method of any one of claims 1 to 4.

Images