Disk detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a disk detection method and device, electronic equipment and a storage medium. The method comprises the following steps: constructing an initial data set containing time delay information and throughput information; determining an abnormal value of the initial data set, removing the abnormal value, and generating a target data set; on the basis of the target data set, generating an online distillation model used for understanding a corresponding relation between time delay and throughput; when disk data of a target disk is obtained, inputting the disk data into the online distillation model, and outputting a slow disk event of the target disk based on the online distillation model; when the slow disk event meets the preset condition, the target disk is judged to be the slow disk, automatic detection of the slow disk is achieved, and the operation stability of the distributed storage system is improved.

Description

Technical Field

[0001] The present invention relates to the field of disk detection technology, and in particular to a disk detection method, a disk detection device, an electronic device and a computer-readable storage medium. Background Art

[0002] In distributed storage systems, due to disk aging, frequent I / O operations or other factors, the response time of some disks will increase significantly, resulting in a decrease in read and write performance. Such disks are called slow disks. The existence of slow disks will seriously affect the performance of the entire cluster, and may even cause serious consequences such as data corruption and service interruption. As the scale of distributed storage systems continues to expand, the number of disks is growing exponentially, and the detection and processing of slow disks has become increasingly complex. In addition, factors such as storage system load fluctuations and solid-state drive garbage collection may also cause short-term performance fluctuations, increasing the difficulty of slow disk identification. Therefore, how to accurately and efficiently detect slow disks from a large number of disks in real time has become one of the key challenges for the stable operation of distributed storage systems. Summary of the invention

[0003] The embodiments of the present invention provide a disk detection method, device, electronic device and computer-readable storage medium to overcome the above problems or at least partially solve the above problems.

[0004] The embodiment of the present invention discloses a disk detection method, comprising:

[0005] Construct an initial data set containing latency information and throughput information;

[0006] Determine outliers in the initial data set, remove the outliers, and generate a target data set;

[0007] generating an online distillation model for understanding the corresponding relationship between latency and throughput based on the target data set;

[0008] When the disk data of the target disk is obtained, the disk data is input into the online distillation model, and a slow disk event of the target disk is output based on the online distillation model;

[0009] When the slow disk event meets a preset condition, the target disk is determined to be a slow disk.

[0010] Optionally, the step of determining outliers of the initial data set includes:

[0011] Determine an upper delay quartile and a lower delay quartile of the delay information;

[0012] Determine a throughput upper quartile and a throughput lower quartile of the throughput information;

[0013] Calculate the delay interquartile range based on the delay upper quartile and the delay lower quartile;

[0014] Calculating a throughput interquartile range based on the throughput upper quartile and the throughput lower quartile;

[0015] Determine a delay threshold based on the delay interquartile range, and calculate a throughput threshold based on the throughput interquartile range;

[0016] Data whose delay is greater than the delay threshold and whose throughput is less than the throughput threshold is determined as abnormal values.

[0017] Optionally, the step of generating an online distillation model for understanding the corresponding relationship between latency and throughput based on the target data set includes:

[0018] Constructing a multi-branch model, wherein the multi-branch model includes a polynomial model as a student model, a decision tree model, and a Gaussian regression model;

[0019] Obtaining a polynomial model output, a decision tree model output, and a Gaussian regression model output of the polynomial model, the decision tree model, and the Gaussian regression model;

[0020] Determining an implicit teacher model output of an implicit teacher model using the polynomial model output, the decision tree model output, and the Gaussian regression model output;

[0021] Calculating a first loss function based on the implicit teacher model output;

[0022] Calculating a second loss function based on the implicit teacher model output, the polynomial model output, the decision tree model output, and the Gaussian regression model output;

[0023] An online distillation model is trained based on the first loss function and the second loss function.

[0024] Optionally, the step of determining the implicit teacher model output of the implicit teacher model by using the polynomial model output, the decision tree model output and the Gaussian regression model output comprises:

[0025] Inputting the polynomial model output, the decision tree model output and the Gaussian regression model output into Formula 1 to calculate the implicit teacher model output;

[0026] The formula 1 is:

[0027]

[0028] Where n refers to the number of student models, R si refers to the output of the i-th student model, Rt Refers to the implicit teacher model output after integration.

[0029] Optionally, the step of calculating a first loss function based on the implicit teacher model output comprises:

[0030] Inputting the implicit teacher model output into Formula 2 to calculate a first loss function;

[0031] The formula 2 is:

[0032] L1=||yR t || 2

[0033] Among them, y refers to the real data value in the training set, and L1 is the first loss function.

[0034] Optionally, the step of calculating a second loss function based on the implicit teacher model output, the polynomial model output, the decision tree model output and the Gaussian regression model output comprises:

[0035] Inputting the implicit teacher model output, the polynomial model output, the decision tree model output and the Gaussian regression model output into Formula 3 to calculate a second loss function;

[0036] The formula three is:

[0037]

[0038] Among them, L2 is the second loss function.

[0039] Optionally, the step of training the online distillation model based on the first loss function and the second loss function includes:

[0040] Determine a hyperparameter for the online distillation model, and input the hyperparameter, the first loss function, and the second loss function into Formula 4 to calculate an overall loss function;

[0041] The formula 4 is:

[0042] L=L1+θL2

[0043] Wherein, θ is the hyperparameter, and L is the overall loss function;

[0044] The overall loss function is used to train the online distillation model.

[0045] The embodiment of the present invention also discloses a disk detection device, comprising:

[0046] An initial data set construction module, used to construct an initial data set including delay information and throughput information;

[0047] A target data set generation module, used to determine the outliers of the initial data set, remove the outliers, and generate a target data set;

[0048] An online distillation model generation module, used to generate an online distillation model for understanding the corresponding relationship between latency and throughput based on the target data set;

[0049] A slow disk event output module, used for inputting the disk data into the online distillation model when obtaining the disk data of the target disk, and outputting the slow disk event of the target disk based on the online distillation model;

[0050] The target disk is a slow disk determining module is used to determine that the target disk is a slow disk when the slow disk event meets a preset condition.

[0051] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0052] The memory is used to store computer programs;

[0053] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.

[0054] The embodiment of the present invention further discloses a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, enables the processors to execute the method described in the embodiment of the present invention.

[0055] The embodiments of the present invention include the following advantages:

[0056] An embodiment of the present invention achieves the following beneficial effects by constructing an initial data set including latency information and throughput information; determining outliers of the initial data set, removing the outliers, and generating a target data set; generating an online distillation model for understanding the correspondence between latency and throughput based on the target data set; when disk data of a target disk is obtained, inputting the disk data into the online distillation model, and outputting a slow disk event of the target disk based on the online distillation model; and determining that the target disk is a slow disk when the slow disk event meets a preset condition.

[0057] High accuracy: The application of knowledge distillation improves the model's ability to fit complex data.

[0058] Strong generalization: The design of the multi-branch model enhances the model's adaptability to different scenarios.

[0059] Good robustness: The introduction of outlier processing improves the robustness of the model to noisy data.

[0060] This technical solution effectively improves the accuracy and generalization of slow disk detection by introducing knowledge distillation and multi-branch models. This method has strong practical value and can be applied to large-scale distributed storage systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 is a flowchart of a disk detection method provided in an embodiment of the present invention;

[0062] Figure 2 It is a flow chart of a method for removing outliers in a data set provided in an embodiment of the present invention;

[0063] Figure 3 is a flowchart of a disk detection method provided in an embodiment of the present invention;

[0064] Figure 4 is a flow chart of a model training method provided in an embodiment of the present invention;

[0065] Figure 5 is a structural schematic diagram of a model provided in an embodiment of the present invention;

[0066] Figure 6 is a structural block diagram of a disk detection device provided in an embodiment of the present invention;

[0067] Figure 7 is a hardware structure block diagram of an electronic device provided in an embodiment of the present invention;

[0068] Figure 8 is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0070] During the long-term operation of a distributed storage system, some disks may experience increased latency and slower reading and writing due to physical conditions or frequent reading and writing. These disks are called slow disks. Slow disks may cause the performance of the entire cluster to degrade, and in severe cases may even lead to data loss, service unavailability and other problems. As the deployment scale of distributed storage systems becomes larger and larger, the number of disks included can reach thousands or even tens of thousands, and the number of slow disks will also increase. How to quickly and effectively detect slow disks is a problem, because a sudden increase in the storage system's read and write volume and internal garbage collection of solid-state drives may cause similar situations to slow disks.

[0071] The existing slow disk detection is generally based on the two parameters of disk latency and throughput. That is, when the throughput is not large, the disk with large latency will be considered a slow disk. There are mainly the following methods for slow disk detection:

[0072] The first method is to set a static threshold to identify slow disks. That is, set two static thresholds, Lc and Tc, which represent the latency threshold and throughput threshold respectively. If the latency of the disk is greater than Lc and the throughput is less than Tc, it can be considered a slow disk. This threshold is generally given by experts based on experience and parameter fine-tuning. Although it is relatively simple in logic, the threshold is difficult to control and can easily affect the accuracy of slow disk detection.

[0073] The second method: Use machine learning methods to establish the relationship between latency and throughput to identify slow disks. Generally, a polynomial model is used for modeling, and the slow disk is identified based on the model. However, the polynomial model is relatively simple, with poor accuracy and generalization. Complex models have better effects, but they occupy more resources when performing slow disk detection, which will affect normal business operations.

[0074] Therefore, it is necessary to improve the existing slow disk detection method and model the disk latency and throughput more effectively on the basis of lower complexity to solve the problems of low accuracy and poor generalization of the existing method.

[0075] Reference Figure 1 , shows a flowchart of a disk detection method provided in an embodiment of the present invention, which may specifically include the following steps:

[0076] Step 101, constructing an initial data set including delay information and throughput information;

[0077] Step 102, determining outliers in the initial data set, removing the outliers, and generating a target data set;

[0078] Step 103: generating an online distillation model for understanding the corresponding relationship between latency and throughput based on the target data set;

[0079] Step 104: when the disk data of the target disk is obtained, the disk data is input into the online distillation model, and a slow disk event of the target disk is output based on the online distillation model;

[0080] Step 105: When the slow disk event meets a preset condition, the target disk is determined to be a slow disk.

[0081] In a specific implementation, an embodiment of the present invention can construct an initial data set containing latency information and throughput information; to collect the original performance data of the disk, provide a basis for subsequent model training, and thus establish a comprehensive data set that can more accurately reflect the performance characteristics of the disk.

[0082] In a specific implementation, an embodiment of the present invention can determine the outliers of the initial data set, remove the outliers, and generate a target data set; remove noise and outliers in the data, improve the quality of the data, and prevent abnormal data from interfering with model training, so as to improve the robustness of the model and make it better adapt to normal data distribution.

[0083] In a specific implementation, an embodiment of the present invention can generate an online distillation model for understanding the correspondence between latency and throughput based on the target data set; build a model that can accurately predict disk latency to provide a basis for subsequent slow disk judgment, so as to continuously optimize the model through online distillation so that it can adapt to the dynamic changes in disk performance.

[0084] In a specific implementation, an embodiment of the present invention can input the disk data of the target disk into the online distillation model when the disk data of the target disk is obtained, and output the slow disk event of the target disk based on the online distillation model; use the trained model to predict the new disk data to determine whether there is a slow disk event, thereby realizing real-time monitoring of disk performance and timely detecting slow disks.

[0085] In a specific implementation, an embodiment of the present invention can determine that the target disk is a slow disk when the slow disk event meets preset conditions; based on conditions such as the duration or severity of the slow disk event, it is ultimately determined whether the disk is a slow disk, providing a clear judgment standard to avoid misjudgment.

[0086] For example, the trained online distillation model output can be used to express the prediction value of the slow disk event. Specifically, the target disk throughput of the disk data of the target disk can be used as the input of the online distillation model, and the delay value can be obtained through the output of the online distillation model. If the delay of the disk exceeds the predicted delay, it can be considered as a slow disk event; after the determined slow disk event lasts for a certain period of time, the disk can be considered to be a slow disk.

[0087] For example:

[0088] The test is performed every half a minute. If the same disk has 8 slow disk events within five minutes (that is, more than 80% of the test results within five minutes are slow disk events), the disk can be considered a slow disk. The reason for this is that only continuous slow disk events can be judged as slow disks, because occasional peaks may be caused by acceptable performance changes.

[0089] According to an embodiment of the present invention, an initial data set including latency information and throughput information is constructed; outliers of the initial data set are determined, the outliers are removed, and a target data set is generated; an online distillation model for understanding the correspondence between latency and throughput is generated based on the target data set; when disk data of a target disk is obtained, the disk data is input into the online distillation model, and a slow disk event of the target disk is output based on the online distillation model; when the slow disk event meets a preset condition, the target disk is determined to be a slow disk, thereby achieving automatic detection of the slow disk and improving the operational stability of the distributed storage system.

[0090] Based on the above embodiment, a variant embodiment of the above embodiment is proposed. It should be noted that in order to make the description concise, only the differences from the above embodiment are described in the variant embodiment.

[0091] In an optional embodiment of the present invention, the step of determining the outliers of the initial data set comprises:

[0092] Determine an upper delay quartile and a lower delay quartile of the delay information;

[0093] Determine a throughput upper quartile and a throughput lower quartile of the throughput information;

[0094] Calculate the delay interquartile range based on the delay upper quartile and the delay lower quartile;

[0095] Calculating a throughput interquartile range based on the throughput upper quartile and the throughput lower quartile;

[0096] Determine a delay threshold based on the delay interquartile range, and calculate a throughput threshold based on the throughput interquartile range;

[0097] Data whose delay is greater than the delay threshold and whose throughput is less than the throughput threshold is determined as abnormal values.

[0098] refer to Figure 2 , Figure 2 It is a flow chart of a method for removing outliers in a data set provided in an embodiment of the present invention;

[0099] Exemplarily, determining the outliers of the initial data set may be performed through the following sub-steps.

[0100] S20: Arrange the constructed data set in ascending order of latency and throughput, and obtain the upper quartile L of the latency in the data set. Q3 and the lower quartile L Q1 , and the upper quartile of throughput T Q3 and the lower quartile T Q1,By calculating the quartiles of the data set, we can understand the distribution of the data and provide a reference for subsequent outlier judgment.

[0101] S21: Calculate the IQR (Interquartile Range) based on the upper and lower quartiles. The interquartile range of the delay L IQR The expression is L IQR =L Q3 -L Q1 , the interquartile range of throughput T IQR The expression is T IQR =T Q3 -T Q1 The interquartile range (IQR) can be used to indicate the spread of data distribution. By calculating the IQR, outliers can be more accurately determined.

[0102] S22: Obtain a threshold value based on the interquartile range and the upper quartile. The threshold value of the delay L = L Q3 +2L IQR , the throughput threshold is T = T Q3 +2T IQR Based on the interquartile range and the upper quartile, a reasonable threshold is determined to judge outliers.

[0103] S23: Remove the part of the data set with a latency greater than L and a throughput less than T. This part is an outlier. Remove the outliers in the data set to improve data quality and avoid interference with model training.

[0104] The embodiment of the present invention determines the upper quartile of the delay and the lower quartile of the delay information; determines the upper quartile of the throughput and the lower quartile of the throughput information; calculates the delay interquartile range based on the upper quartile of the delay and the lower quartile of the delay; calculates the throughput interquartile range based on the upper quartile of the throughput and the lower quartile of the throughput; determines the delay threshold based on the delay interquartile range, and calculates the throughput threshold based on the throughput interquartile range; determines data with a delay greater than the delay threshold and a throughput less than the throughput threshold as an outlier, refers to the static threshold method, and uses the quartiles in the data set to remove outliers. This method is simple and fast, and removing some obvious outliers is beneficial to subsequent training.

[0105] Optionally, the step of generating an online distillation model for understanding the corresponding relationship between latency and throughput based on the target data set includes:

[0106] Constructing a multi-branch model, wherein the multi-branch model includes a polynomial model as a student model, a decision tree model, and a Gaussian regression model;

[0107] Obtaining a polynomial model output, a decision tree model output, and a Gaussian regression model output of the polynomial model, the decision tree model, and the Gaussian regression model;

[0108] Determining an implicit teacher model output of an implicit teacher model using the polynomial model output, the decision tree model output, and the Gaussian regression model output;

[0109] Calculating a first loss function based on the implicit teacher model output;

[0110] Calculating a second loss function based on the implicit teacher model output, the polynomial model output, the decision tree model output, and the Gaussian regression model output;

[0111] An online distillation model is trained based on the first loss function and the second loss function.

[0112] In a specific implementation, the embodiment of the present invention can select a polynomial model, a decision tree model and a Gaussian regression model as a student model to construct a multi-branch model.

[0113] Polynomial model: This model assumes a polynomial relationship between latency and throughput and can fit nonlinear relationships.

[0114] Decision tree model: The decision tree model is good at handling classification problems. It can divide data according to the values ​​of different features to establish classification rules.

[0115] Gaussian regression model: The Gaussian regression model assumes that the output variable follows a Gaussian distribution and can be used to predict continuous values.

[0116] Multi-branch structure: These three models are connected in parallel to form a multi-branch model. Each branch model predicts the input data and obtains different outputs.

[0117] In a specific implementation, the embodiment of the present invention can obtain the polynomial model output, decision tree model output and Gaussian regression model output of the polynomial model, the decision tree model and the Gaussian regression model, so as to input the data to be predicted into each branch model, and obtain the polynomial model output, decision tree model output and Gaussian regression model output respectively. These outputs represent the prediction results of each model on the relationship between delay and throughput.

[0118] In a specific implementation, an embodiment of the present invention can use the polynomial model output, the decision tree model output and the Gaussian regression model output to determine the implicit teacher model output of the implicit teacher model; the outputs of the three branch models are fused in some form to obtain a comprehensive output, namely, the output of the implicit teacher model.

[0119] This fusion can be a simple average, weighted average, or a more complex fusion method, such as an attention mechanism.

[0120] The output of the implicit teacher model represents the joint prediction of multiple models on the relationship between latency and throughput.

[0121] In a specific implementation, the embodiment of the present invention can calculate a first loss function based on the output of the implicit teacher model; compare the output of the implicit teacher model with the true label (i.e., the known correspondence between latency and throughput) to calculate a loss function. This loss function is used to measure the prediction accuracy of the implicit teacher model.

[0122] In a specific implementation, the embodiment of the present invention can calculate the second loss function based on the output of the implicit teacher model, the output of the polynomial model, the output of the decision tree model and the output of the Gaussian regression model; in addition to comparing the difference between the implicit teacher model and the true label, the difference between the output of each branch model and the output of the implicit teacher model can also be calculated. This loss function is used to measure the consistency of each branch model with the implicit teacher model.

[0123] In a specific implementation, the embodiment of the present invention can train an online distillation model based on the first loss function and the second loss function. Joint optimization: By minimizing the first and second loss functions, the entire multi-branch model is jointly optimized. Online learning: When new data arrives, the parameters of the model are continuously updated to adapt to changes in data distribution.

[0124] In an embodiment of the present invention, a multi-branch model is constructed, wherein the multi-branch model includes a polynomial model, a decision tree model and a Gaussian regression model as student models; the polynomial model output, the decision tree model output and the Gaussian regression model output of the polynomial model, the decision tree model and the Gaussian regression model are obtained; the implicit teacher model output of the implicit teacher model is determined using the polynomial model output, the decision tree model output and the Gaussian regression model output; a first loss function is calculated based on the implicit teacher model output; a second loss function is calculated based on the implicit teacher model output, the polynomial model output, the decision tree model output and the Gaussian regression model output; and an online distillation model is trained based on the first loss function and the second loss function, thereby achieving the following beneficial effects.

[0125] Diversity: Multiple models can capture different characteristics of the data and improve the robustness of the model.

[0126] Complementarity: The advantages of different models complement each other and can improve the prediction accuracy of the model.

[0127] Implicit teacher: The implicit teacher model can guide the student model to learn more accurate knowledge.

[0128] Online Learning: Online learning can adapt the model to the changing data distribution.

[0129] Improve prediction accuracy: The collaboration of multiple models can improve the prediction accuracy of the relationship between latency and throughput.

[0130] Enhance model robustness: Different models have different sensitivities to noise and abnormal data. The combination of multiple models can reduce the model's sensitivity to noise.

[0131] Improve model interpretability: By analyzing the output of different models, the prediction results of the model can be explained.

[0132] This multi-branch model based on online distillation can effectively understand the correspondence between latency and throughput by combining the advantages of multiple models. This method has broad potential in practical applications.

[0133] Optionally, the step of determining the implicit teacher model output of the implicit teacher model by using the polynomial model output, the decision tree model output and the Gaussian regression model output comprises:

[0134] Inputting the polynomial model output, the decision tree model output and the Gaussian regression model output into Formula 1 to calculate the implicit teacher model output;

[0135] The formula 1 is:

[0136]

[0137] Where n refers to the number of student models, R si refers to the output of the i-th student model, R t Refers to the implicit teacher model output after integration.

[0138] Formula 1 indicates that the output Rt of the implicit teacher model is the average of the outputs Rsi of all student models.

[0139] R t : The output of the implicit teacher model, representing the final prediction of the relationship between latency and throughput.

[0140] n: The number of student models, in this case 3 (polynomial model, decision tree model, Gaussian regression model).

[0141] R si : The output of the i-th student model.

[0142] The calculation process is as follows:

[0143] Get the output of each student model:

[0144] The data to be predicted is input into each student model.

[0145] Each model outputs a predicted value, Rsi.

[0146] Calculate the average:

[0147] Add the outputs of all student models together.

[0148] Divide the result of the addition by the number of student models n.

[0149] The average value obtained is the output Rt of the implicit teacher model.

[0150] The reasons for taking the average are:

[0151] Reducing variance: By taking the average, the variance of a single model’s prediction can be reduced, improving the stability of the prediction results.

[0152] Combining the advantages of different models: Each model has its own strengths. By averaging, the advantages of different models can be combined to improve the overall prediction accuracy.

[0153] Implements simple integration: Averaging is a simple yet effective integration method that is easy to implement.

[0154] The significance of the implicit teacher model

[0155] Knowledge distillation: The implicit teacher model can be regarded as a "teacher" that integrates the knowledge of multiple student models to improve the generalization ability of the model.

[0156] Model integration: By averaging the outputs of multiple models, model integration is achieved, which improves the robustness of the model.

[0157] Provides more reliable predictions: The output of the implicit teacher model is generally more reliable than the output of a single model.

[0158] By averaging the outputs of multiple student models, a more reliable and robust implicit teacher model can be obtained. This method is simple and effective and is widely used in many machine learning tasks.

[0159] Optionally, the step of calculating a first loss function based on the implicit teacher model output comprises:

[0160] Inputting the implicit teacher model output into Formula 2 to calculate a first loss function;

[0161] The formula 2 is:

[0162] L1=||yR t || 2

[0163] Among them, y refers to the real data value in the training set, and L1 is the first loss function.

[0164] Indicates that the first loss function L1 is the sum of the true value y and the predicted value R t The squared difference between .

[0165] L1: The first loss function measures the gap between the model prediction value and the true value.

[0166] y: The true data value in the training set, which is the correct answer we expect the model to predict.

[0167] R t : The output of the implicit teacher model, that is, the model's predicted value for the input data.

[0168] ||yR t || 2 : Represents the square of the L2 norm of the vector, which is the square of the Euclidean distance.

[0169] The calculation process is as follows:

[0170] Calculate the difference: Subtract the predicted value Rt from the true value y to get an error vector.

[0171] Compute the sum of squares: Square each element of the error vector and add them together to get a scalar.

[0172] Get the loss value: This scalar is the value of our loss function L1.

[0173] The significance of the second loss function:

[0174] Measuring prediction error: The smaller the loss function, the closer the model's prediction result is to the true value, and the better the model performance is.

[0175] Optimization goal: During the model training process, we continuously adjust the model parameters by minimizing the loss function to make the model's prediction results more and more accurate.

[0176] The reason for using squared error is:

[0177] Penalize large errors: Squared error penalizes larger errors more, which helps the model pay more attention to samples with larger prediction errors.

[0178] Differentiability: The squared error function is continuously differentiable, which allows us to use optimization algorithms such as gradient descent to minimize the loss function.

[0179] Widely used: Squared error is one of the most commonly used loss functions in machine learning and has good mathematical properties.

[0180] Formula 2 defines a simple but very effective loss function. By minimizing this loss function, we can train a model with better performance.

[0181] Optionally, the step of calculating a second loss function based on the implicit teacher model output, the polynomial model output, the decision tree model output and the Gaussian regression model output comprises:

[0182] Inputting the implicit teacher model output, the polynomial model output, the decision tree model output and the Gaussian regression model output into Formula 3 to calculate a second loss function;

[0183]

[0184] Among them, L2 is the second loss function.

[0185] Formula 3 shows that the second loss function L2 is the output R of all student models si With the implicit teacher model output R t The sum of the squared differences between .

[0186] L2: The second loss function measures the difference between the student model and the teacher model.

[0187] R si : The output of the i-th student model.

[0188] R t : Output of the implicit teacher model.

[0189] ||R si -R t || 2 : The squared error between the output of the i-th student model and the output of the teacher model.

[0190] Σ: summation symbol, indicating the addition of the squared errors of all student models.

[0191] Calculation process:

[0192] Calculate the difference between each student model and the teacher model: For each student model, calculate the square of the difference between its output and the teacher model output.

[0193] Sum: Add the squared errors of all student models to get the total squared error.

[0194] Get the loss value: This total square error is the value of our second loss function L2.

[0195] The significance of the loss function is:

[0196] Measuring the consistency between the student model and the teacher model: The smaller the loss function L2, the closer the output of the student model is to the output of the teacher model, that is, the closer the knowledge learned by the student model is to the teacher model.

[0197] Knowledge distillation: By minimizing L2, the student model can be forced to learn the knowledge of the teacher model, thereby improving the performance of the student model.

[0198] The reasons for using this loss function are:

[0199] Knowledge Distillation: This loss function is a core concept in knowledge distillation, which can transfer the knowledge of a complex large model (teacher model) to a smaller model (student model).

[0200] Improve generalization ability: By forcing the student model to imitate the teacher model, the generalization ability of the student model can be improved.

[0201] Regularization effect: This loss function also plays a certain regularization role, which can prevent overfitting.

[0202] Formula 3 defines a loss function for measuring the difference between the student model and the teacher model. By minimizing this loss function, knowledge distillation can be achieved and the performance of the student model can be improved.

[0203] The difference between L2 and L1: L1 measures the difference between the teacher model and the true label, while L2 measures the difference between the student model and the teacher model.

[0204] Optionally, different weights can be assigned to different student models to reflect their importance.

[0205] Optionally, other types of loss functions can be used instead of squared error, such as cross entropy loss, etc.

[0206] Optionally, the step of training the online distillation model based on the first loss function and the second loss function includes:

[0207] Determine a hyperparameter for the online distillation model, and input the hyperparameter, the first loss function, and the second loss function into Formula 4 to calculate an overall loss function;

[0208] The formula 4 is:

[0209] L=L1+θL2

[0210] Wherein, θ is the hyperparameter, and L is the overall loss function;

[0211] The overall loss function is used to train the online distillation model.

[0212] Formula 4 gives an overall loss function that comprehensively considers losses in multiple aspects.

[0213] L: represents the loss of the entire model and is the objective function of model training.

[0214] L1: The first loss function measures the difference between the model prediction value and the true value.

[0215] L2: The second loss function measures the difference between the student model and the teacher model.

[0216] θ: A hyperparameter used to balance the influence of L1 and L2 in the overall loss.

[0217] The role of the overall loss function:

[0218] Comprehensive Optimization: By combining L1 and L2 into an overall loss function, we can simultaneously optimize the prediction accuracy and knowledge distillation effect of the model.

[0219] Hyperparameter θ: The role of the hyperparameter θ is to adjust the weight of L1 and L2 in the total loss.

[0220] When θ is larger, the model will pay more attention to the consistency between the student model and the teacher model, that is, knowledge distillation.

[0221] When θ is smaller, the model will focus more on prediction accuracy.

[0222] Training process:

[0223] Determine the hyperparameter θ: Select an appropriate θ value based on the characteristics of the specific task and dataset.

[0224] Hyperparameter θ: In machine learning models, hyperparameters are parameters that need to be set manually before training, rather than parameters learned during training. θ controls the influence of the distillation loss (L2) on the total loss (L) in our model.

[0225] Distillation related loss (L2): This part of the loss measures the difference between the output of the student model and the teacher model. Its function is to make the student model imitate the teacher model and thus learn the knowledge of the teacher model.

[0226] Total loss (L): The final optimization target of the model, which consists of two parts: L1 (measures the gap between the model prediction and the true label) and L2 (measures the gap between the student model and the teacher model).

[0227] The value range of θ: The value of θ can only be between 0 and 1, including 1.

[0228] The role of θ: The larger θ is, the more we value the consistency between the student model and the teacher model when optimizing the model. In other words, we hope that the student model can accurately imitate the teacher model.

[0229] Choosing θ: The value of θ is not fixed. We can adjust the value of θ through experiments, and then compare the performance of the model under different θ values, and finally choose a θ value that can make the model work best.

[0230] When θ = 0, it means that we do not consider the distillation loss at all, and the model only focuses on fitting the training data.

[0231] When θ=1, it means that we only consider the distillation loss. The model will try its best to imitate the teacher model, while relatively ignoring the fitting of the training data.

[0232] When 0<θ<1, it means that we consider both prediction accuracy and knowledge distillation, and the value of θ determines the balance between the two.

[0233] The reason for adjusting θ is:

[0234] Task requirements: Different tasks have different requirements for prediction accuracy and knowledge distillation.

[0235] Model structure: Models with different structures have different sensitivities to distillation.

[0236] Dataset characteristics: The size and quality of the dataset will affect the optimal value of θ.

[0237] Methods for choosing θ include:

[0238] Grid search: Within a preset range, take different values ​​of θ, train the models separately, then compare the performance of the models on the validation set and select the θ with the best performance.

[0239] Random Search: Randomly sample values ​​of θ for training and evaluation.

[0240] Bayesian Optimization: Use the Bayesian optimization algorithm to search for the optimal θ value more efficiently.

[0241] By adjusting the hyperparameter θ, we can control the balance between prediction accuracy and knowledge distillation, thereby obtaining a model that better meets actual needs.

[0242] Calculate the overall loss L: Substitute the values ​​of L1 and L2 into Formula 4 to calculate the total loss of the current model.

[0243] Update model parameters: Use optimization algorithms such as gradient descent to update the model parameters based on the calculated loss L so that the value of the loss function continues to decrease.

[0244] Iterative training: Repeat steps 2 and 3 until the model converges or the preset number of training times is reached.

[0245] Why do we need an overall loss function?

[0246] Balance between knowledge distillation and prediction accuracy: L1 ensures the prediction accuracy of the model, while L2 ensures that the student model can learn the knowledge of the teacher model.

[0247] Flexible Adjustment: By adjusting the hyperparameter θ, the balance between prediction accuracy and knowledge distillation of the model can be flexibly controlled.

[0248] Online distillation model training

[0249] Online learning: The training of online distillation models is usually based on online learning, that is, the model continuously updates its parameters when receiving new data.

[0250] Dynamic Adjustment: During the training process, the hyperparameter θ can be dynamically adjusted to adapt to changes in data distribution.

[0251] Summarize

[0252] By introducing an overall loss function, the two objectives of prediction accuracy and knowledge distillation are unified, and the online distillation model can be effectively trained. By adjusting the hyperparameter θ, the two objectives can be balanced to obtain a model with better performance.

[0253] Optionally, the embodiment of the present invention may further:

[0254] Hyperparameter tuning: The choice of θ has a significant impact on the performance of the model and requires careful tuning.

[0255] Other loss functions: In addition to L1 and L2, other loss functions can also be introduced, such as regularization terms.

[0256] Optimization algorithm: In addition to gradient descent, other optimization algorithms can also be used, such as Adam, RMSprop, etc.

[0257] In order to enable those skilled in the art to better understand the embodiment of the present invention, an example is used below to illustrate the embodiment of the present invention.

[0258] The embodiment of the present invention aims to provide a slow disk detection method based on knowledge distillation, aiming to solve the problems of low accuracy and poor generalization in existing slow disk detection methods.

[0259] refer to Figure 3 , Figure 3 is a flowchart of a disk detection method provided in an embodiment of the present invention;

[0260] The method proposed in the present invention is mainly divided into two stages: training and reasoning.

[0261] The training phase consists of the following steps:

[0262] Step S1: construct a data set of latency and throughput;

[0263] Step S2: Use quartiles to calculate the threshold and remove outliers in the data set;

[0264] Step S3: Build a knowledge distillation-based model and use it to train on the dataset with outliers removed to obtain a relationship model between latency and throughput.

[0265] The inference phase consists of the following steps:

[0266] Step S4: Use the trained model to perform inference and obtain slow disk events;

[0267] Step S5: When the slow disk event lasts for a certain period of time, the disk is considered to be a slow disk.

[0268] By implementing slow disk detection in the above manner, the following beneficial effects are achieved.

[0269] High accuracy: The application of knowledge distillation improves the model's ability to fit complex data.

[0270] Strong generalization: The design of the multi-branch model enhances the model's adaptability to different scenarios.

[0271] Good robustness: The introduction of outlier processing improves the robustness of the model to noisy data.

[0272] This technical solution effectively improves the accuracy and generalization of slow disk detection by introducing knowledge distillation and multi-branch models. This method has strong practical value and can be applied to large-scale distributed storage systems.

[0273] The embodiment of the present invention proposes a slow disk detection method based on knowledge distillation, which is divided into two stages: training and reasoning. In the training stage, firstly, a data set of latency and throughput is constructed, and outliers are removed; then, a multi-branch model is constructed using knowledge distillation using multiple machine learning models with lower complexity. Each branch is a student model. In the training stage, these models are integrated to construct an implicit teacher model, and the knowledge of the teacher model is transferred to the student model to assist in training. Finally, the model with the highest test index is selected from the student model for the final slow disk detection.

[0274] In the inference phase, the trained model is used to obtain the predicted latency based on the disk throughput, and then compared with the actual latency to determine whether there is a slow disk event. If the event lasts for a period of time, the disk can be considered to be a slow disk.

[0275] The method proposed in the present invention constructs an implicit teacher model through multiple student models, and transfers the knowledge of the teacher model to the student networks of each branch through distillation, which effectively improves the accuracy and generalization of slow disk detection.

[0276] After acquiring the disk data of the target disk, the embodiment of the present invention integrates multiple models with lower complexity (student models) through knowledge distillation to construct an implicit teacher model, and transfers the knowledge of the teacher model to the student model through knowledge distillation, thereby improving the performance of the model; and adopts a multi-branch model, through multiple student models, to enhance the model's adaptability to different data distributions and improve the generalization of the model.

[0277] In the embodiment of the present invention, the quartile method can be used to remove outliers, improve data quality, and enhance model training effects.

[0278] The latency prediction method can predict the disk latency and compare it with the actual latency to determine whether there is a slow disk event.

[0279] The embodiments of the present invention improve the accuracy and generalization of the model by applying knowledge distillation to slow disk detection, that is, through knowledge distillation, and solve the problems of low model accuracy and poor generalization in traditional methods.

[0280] The design of multi-branch models can effectively enhance the model's adaptability to different scenarios.

[0281] The introduction of outlier processing: improved data quality and enhanced the robustness of the model.

[0282] Technical processes may include:

[0283] Data preparation: Construct a data set that includes latency and throughput, and perform outlier processing.

[0284] Model training:

[0285] Construct multiple student models;

[0286] Integrate the student model to build an implicit teacher model;

[0287] Through knowledge distillation, the knowledge of the teacher model is transferred to the student model;

[0288] The student model with the best performance is selected as the final model.

[0289] Model Reasoning:

[0290] Input the disk throughput and predict its latency.

[0291] Compare the predicted latency with the actual latency to determine whether there is a slow disk event.

[0292] If the slow disk event lasts for a certain period of time, the disk is determined to be a slow disk.

[0293] By implementing slow disk detection in the above manner, the following beneficial effects are achieved.

[0294] High accuracy: The application of knowledge distillation improves the model's ability to fit complex data.

[0295] Strong generalization: The design of the multi-branch model enhances the model's adaptability to different scenarios.

[0296] Good robustness: The introduction of outlier processing improves the robustness of the model to noisy data.

[0297] This technical solution effectively improves the accuracy and generalization of slow disk detection by introducing knowledge distillation and multi-branch models. This method has strong practical value and can be applied to large-scale distributed storage systems.

[0298] Step S2 specifically includes the following steps:

[0299] S20: Arrange the constructed data set in ascending order according to latency and throughput, and obtain an upper quartile LQ3 and a lower quartile LQ1 of latency, and an upper quartile TQ3 and a lower quartile TQ1 of throughput in the data set.

[0300] S21: Calculate the IQR (Interquartile Range) based on the upper quartile and the lower quartile. The interquartile range LIQR of latency is expressed as LIQR=LQ3-LQ1, and the interquartile range TIQR of throughput is expressed as TIQR=TQ3-TQ1.

[0301] S22: A threshold is obtained according to the interquartile range and the upper quartile. The threshold of latency is L=LQ3+2LIQR, and the threshold of throughput is T=TQ3+2TIQR.

[0302] S23: Remove the part of the data set with a delay greater than L and a throughput less than T. This part is an outlier.

[0303] In step S2, the static threshold method is used to remove outliers using the quartiles in the data set. This method is simple and fast, and removing some obvious outliers is beneficial to subsequent training.

[0304] See also Figure 4-Figure 5 , Figure 4 is a flow chart of a model training method provided in an embodiment of the present invention; Figure 5 is a structural schematic diagram of a model provided in an embodiment of the present invention;

[0305] Step S3 specifically includes the following steps:

[0306] Step S30: Select a polynomial model, a decision tree model, and a Gaussian regression model as student models to construct a multi-branch model;

[0307] Step S31: Integrate the outputs of the three student models as an implicit teacher model; due to the prediction value of the model's output delay, the integration method used here is the averaging method, which averages the output prediction results as the output of the teacher model. Therefore, the expression of the teacher model output is as follows:

[0308] Among them, n refers to the number of student models, here it is 3. Rsi refers to the output of the i-th student model, and Rt refers to the integrated teacher output.

[0309] Step S32: The loss function is divided into two parts, one is the loss calculated by the integrated output and the training set, and the other is the loss calculated by the teacher model and the student model.

[0310] Since the output of the regression model is a continuous value, the KL divergence in the traditional knowledge distillation method cannot be used. Therefore, the root mean square error is used in the knowledge distillation part. The loss value expression of the first part is as follows:

[0311] L1=||yR t || 2

[0312] Here, y refers to the true data value in the training set.

[0313] The loss value expression of the second part is as follows:

[0314]

[0315] The overall loss function L is expressed as follows:

[0316] L=L1+θL2

[0317] Among them, θ is a hyperparameter with a value range of (0,1]. The larger the proportion, the greater the effect of distillation-related loss. The best parameter can be selected based on the training results.

[0318] Step S33: Test the trained model and select the one with the highest score among several branches for the final slow disk detection.

[0319] The overall structure of the model involved in step S3 is as follows Figure 5 As shown in the figure. Multiple student models are used to build an implicit teacher model, where each student model is a machine learning model with low complexity. Compared with the traditional knowledge distillation method, the step of training a teacher model with high complexity is omitted, saving the training cost. When inference is performed, only one branch is used, which consumes less system resources and does not affect the normal business of the cluster.

[0320] In step S4, the trained model is used to obtain a predicted value, that is, the throughput of the input disk, and the latency value can be obtained. If the latency of the disk exceeds the predicted latency, it can be considered a slow disk event.

[0321] In step S5, when the slow disk event determined in step S4 lasts for a certain period of time, the disk can be considered to be a slow disk. The specific process is described below:

[0322] The test is performed every half a minute. If the same disk has 8 slow disk events within five minutes (that is, more than 80% of the test results within five minutes are slow disk events), the disk can be considered a slow disk. The reason for this is that only continuous slow disk events can be judged as slow disks, because occasional peaks may be caused by acceptable performance changes.

[0323] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0324] Reference Figure 6 The structure block diagram of a disk detection device provided in an embodiment of the present invention is shown, which may specifically include the following modules:

[0325] An initial data set construction module 601 is used to construct an initial data set including delay information and throughput information;

[0326] A target data set generation module 602 is used to determine outliers in the initial data set, remove the outliers, and generate a target data set;

[0327] An online distillation model generating module 603, configured to generate an online distillation model for understanding the corresponding relationship between latency and throughput based on the target data set;

[0328] A slow disk event output module 604 is used to input the disk data of the target disk into the online distillation model when the disk data of the target disk is acquired, and output the slow disk event of the target disk based on the online distillation model;

[0329] The target disk is a slow disk determining module 605 is used to determine that the target disk is a slow disk when the slow disk event meets a preset condition.

[0330] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0331] In addition, an embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0332] Memory 703, used for storing computer programs;

[0333] The processor 701 is used to implement any disk detection method described in the above embodiments when executing the program stored in the memory 703:

[0334] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0335] The communication interface is used for communication between the above terminal and other devices.

[0336] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0337] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0338] like Figure 8 As shown, in another embodiment provided by the present invention, a computer-readable storage medium 801 is further provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the disk detection method described in the above embodiment.

[0339] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

[0340] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the embodiments of the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0341] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0342] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0343] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0344] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0345] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0346] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A disk detection method, characterized in that: include: Construct an initial data set containing latency information and throughput information; Determine outliers in the initial data set, remove the outliers, and generate a target data set; generating an online distillation model for understanding the corresponding relationship between latency and throughput based on the target data set; When the disk data of the target disk is obtained, the disk data is input into the online distillation model, and a slow disk event of the target disk is output based on the online distillation model; When the slow disk event meets a preset condition, the target disk is determined to be a slow disk.

2. The method according to claim 1, characterized in that The step of determining outliers of the initial data set comprises: Determine an upper delay quartile and a lower delay quartile of the delay information; Determine a throughput upper quartile and a throughput lower quartile of the throughput information; Calculate the delay interquartile range based on the delay upper quartile and the delay lower quartile; Calculating a throughput interquartile range based on the throughput upper quartile and the throughput lower quartile; Determine a delay threshold based on the delay interquartile range, and calculate a throughput threshold based on the throughput interquartile range; Data whose delay is greater than the delay threshold and whose throughput is less than the throughput threshold is determined as abnormal values.

3. The method according to claim 1, characterized in that The step of generating an online distillation model for understanding the corresponding relationship between latency and throughput based on the target data set includes: Constructing a multi-branch model, wherein the multi-branch model includes a polynomial model as a student model, a decision tree model, and a Gaussian regression model; Obtaining a polynomial model output, a decision tree model output, and a Gaussian regression model output of the polynomial model, the decision tree model, and the Gaussian regression model; Determining an implicit teacher model output of an implicit teacher model using the polynomial model output, the decision tree model output, and the Gaussian regression model output; Calculating a first loss function based on the implicit teacher model output; Calculating a second loss function based on the implicit teacher model output, the polynomial model output, the decision tree model output, and the Gaussian regression model output; An online distillation model is trained based on the first loss function and the second loss function.

4. The method according to claim 3, characterized in that The step of determining the implicit teacher model output of the implicit teacher model by using the polynomial model output, the decision tree model output and the Gaussian regression model output comprises: Inputting the polynomial model output, the decision tree model output and the Gaussian regression model output into Formula 1 to calculate the implicit teacher model output; The formula 1 is: Where n refers to the number of student models, R si refers to the output of the i-th student model, R t Refers to the implicit teacher model output after integration.

5. The method according to claim 4, characterized in that The step of calculating the first loss function based on the implicit teacher model output comprises: Inputting the implicit teacher model output into Formula 2 to calculate a first loss function; The formula 2 is: L1=||yR t || 2 Among them, y refers to the real data value in the training set, and L1 is the first loss function.

6. The method according to claim 5, characterized in that The step of calculating the second loss function based on the implicit teacher model output, the polynomial model output, the decision tree model output and the Gaussian regression model output comprises: Inputting the implicit teacher model output, the polynomial model output, the decision tree model output and the Gaussian regression model output into Formula 3 to calculate a second loss function; The formula three is: Among them, L2 is the second loss function.

7. The method according to claim 6, characterized in that The step of training the online distillation model based on the first loss function and the second loss function comprises: Determine a hyperparameter for the online distillation model, and input the hyperparameter, the first loss function, and the second loss function into Formula 4 to calculate an overall loss function; The formula 4 is: L=L1+θL2 Wherein, θ is the hyperparameter, and L is the overall loss function; The overall loss function is used to train the online distillation model.

8. A disk detection device, characterized in that: include: An initial data set construction module, used to construct an initial data set including delay information and throughput information; A target data set generation module, used to determine the outliers of the initial data set, remove the outliers, and generate a target data set; An online distillation model generation module, used to generate an online distillation model for understanding the corresponding relationship between latency and throughput based on the target data set; A slow disk event output module, used for inputting the disk data into the online distillation model when obtaining the disk data of the target disk, and outputting the slow disk event of the target disk based on the online distillation model; The target disk is a slow disk determining module is used to determine that the target disk is a slow disk when the slow disk event meets a preset condition.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 7.

Images