Self-adaptive threshold calculation method suitable for equipment operation state anomaly detection

Abstract

The invention discloses a self-adaptive threshold calculation method suitable for equipment operation state anomaly detection, which combines artificial experience, distinguishes categories of equipment and data items, distinguishes rules of original data based on statistical results, pre-processes the original data, describes the rules of the original data through statistical values of variable coefficients, and improves the accuracy of the original data. Determining whether a plurality of numerical intervals are clustered; automatically screening data in a normal range; and after data exception processing, judging whether the existing data volume meets the minimum data volume requirement of the calculation threshold, and if so, calculating the self-adaptive threshold by using the mean value and the standard deviation. According to the method, the defects of traditional threshold value calculation are overcome, the defect that the threshold value is calculated only after box diagram denoising is carried out on the data is overcome, the calculated threshold value is more reasonable, the false alarm rate of a single-parameter over-limit early warning model is reduced, the workload of manually adjusting the threshold value is reduced, and the application efficiency of an intelligent operation and maintenance platform is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment online data application and analysis, and in particular to an adaptive threshold calculation method suitable for detecting abnormal operation status of equipment. Background Art

[0002] At present, large-scale metallurgical enterprises have accessed various types of equipment through the equipment remote intelligent operation and maintenance platform, generating millions of data items. In order to maximize the value of data aggregation, the basic goal is to use the data from the operation and maintenance platform to determine whether the equipment is operating normally, identify abnormal values, abnormal equipment status, or abnormalities in the production process, and give alarms to remind equipment managers to perform predictive maintenance based on equipment status, or to handle abnormalities in the production process in a timely manner to reduce losses caused by equipment failure and degradation. The initial application of equipment data is usually to set an early warning model for a single parameter exceeding the limit, and first of all, it is necessary to set a suitable threshold for exceeding the limit.

[0003] The conventional method for calculating the threshold is to select a section of data, filter it through a box plot, and then calculate the threshold using the mean and standard deviation. This method has been used on the operation and maintenance platform. For the ideal case where the data is stable overall, the calculated threshold is consistent with manual experience and can be used directly after confirmation by industry experts. For non-ideal cases where the data is unstable or the selected value is low, the threshold calculated using this method is different from the threshold set by manual experience. In actual production, there are still many false alarms triggered by inappropriate threshold settings. The main reason is that the selected data has abnormal values, missing data, normal operation of the equipment and low values, which cause the statistical calculation results to deviate from the reasonable range of the threshold setting. Therefore, a more reasonable threshold can be calculated by manually screening a relatively stable section of data.

[0004] When filtering data based on box plots, the following situations cannot be effectively handled: (1) Missing data If the data extracted when calculating the threshold is missing, you need to first determine how much data is missing, then decide whether to calculate the threshold, and at the same time give the maximum time of the missing data and restore the data in time; (2) The value remains unchanged for a long time After excluding parameters such as switch quantity and state quantity, the long-term unchanged value is also a manifestation of data anomaly. It is necessary to filter it out in advance, calculate the threshold, and restore the data in time; (3) Abnormally large or small values If the values ​​of some data items are obviously too large or too small, and the proportion of them is high, they cannot be filtered out using a box plot, which will cause the threshold calculated by the conventional method to be too high or too low; (4) Small amount of data After the above preprocessing, the data is excluded from missing data, long-term unchanged values, abnormally large or abnormally small values. If the remaining data is small, the threshold calculated with a small amount of data has no reference significance. The minimum amount of data required to calculate the threshold needs to be specified. The threshold is not calculated if the number of points is less than a certain number. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide an adaptive threshold calculation method suitable for abnormal detection of equipment operating status. This method overcomes the defects of traditional threshold calculation and makes up for the deficiency of calculating the threshold only after box plot denoising of the data, so that the calculated threshold is more reasonable, the false alarm rate of the single parameter over-limit warning model is reduced, the workload of manual adjustment of the threshold is reduced, and the application efficiency of the intelligent operation and maintenance platform is improved.

[0006] To solve the above technical problems, the present invention is applicable to an adaptive threshold calculation method for detecting abnormal operation status of equipment, which comprises the following steps: Step 1: Classify devices, distinguish device management domains, device categories, and data fields, and solidify the experience of manually setting thresholds into automatic calculation logic; Step 2: Data analysis: Analyze the patterns of the original data and distinguish numerical anomalies, data missing, and numerical segment changes based on statistical results; Step 3: Data preprocessing: remove noise from the original data, describe the law of the original data through the statistical value of the coefficient of variation, and determine whether it is clustered into multiple value intervals; Step 4: Data anomaly processing. Check whether there is any missing data. If the amount of missing data is within an acceptable range, continue to judge whether the value remains unchanged, filter out the cases where the value remains unchanged, the value is abnormally large, and the value is abnormally small. Then judge whether the existing data volume meets the minimum data volume requirement for calculating the threshold. If the data volume meets the minimum data volume requirement for calculating the threshold, use the mean and standard deviation to calculate the adaptive threshold.

[0007] Furthermore, in the step three, whether the data is clustered is determined by the statistical value of the coefficient of variation. If clustered, the clustering method is used to find two clusters of data with low and high values. If an upper limit needs to be set, the threshold is calculated using a cluster of data with a high value, and then the threshold is calculated using all the data before clustering, and the maximum value of the two is taken as the final adaptive threshold; if a lower limit needs to be set, the threshold is calculated using a cluster of data with a low value, and then compared with the threshold calculated for all the data, and the minimum value of the two is taken as the final adaptive threshold; if not clustered, the adaptive threshold is directly calculated using all the data after denoising.

[0008] Furthermore, before calculating the adaptive threshold in step 4, it is first determined whether the automatic calculation logic of the manually set threshold has a definite mathematical formula. If so, the threshold is calculated according to the automatic calculation logic. If not, the adaptive threshold is calculated after filtering the original data using a box plot.

[0009] Furthermore, the adaptive threshold calculation performs classified threshold calculation according to the device management domain, device category and data field.

[0010] Furthermore, the calculated adaptive threshold is updated regularly, and historical data is periodically selected to calculate a new adaptive threshold, which is compared with the original threshold. If the difference is large, the threshold setting personnel is reminded to manually determine whether the new adaptive threshold is reasonable and whether the threshold should be updated. Since the adaptive threshold calculation method for detecting abnormal operation status of equipment of the present invention adopts the above-mentioned technical scheme, that is, the method combines manual experience, distinguishes the categories of equipment and data items, distinguishes the regularity of original data based on statistical results, pre-processes the original data, describes the regularity of original data through the statistical value of coefficient of variation, determines whether to cluster into multiple numerical intervals; automatically filters data within the normal range; determines whether the existing data volume meets the minimum data volume requirement for calculating the threshold after processing the data abnormality, and if so, uses the mean and standard deviation to calculate the adaptive threshold. The method overcomes the defects of traditional threshold calculation, makes up for the deficiency of calculating the threshold only after box plot denoising of the data, makes the calculated threshold more reasonable, reduces the false alarm rate of the single parameter over-limit warning model, reduces the workload of manually adjusting the threshold, and improves the application efficiency of the intelligent operation and maintenance platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments: Figure 1 It is a flowchart of a method for calculating an adaptive threshold value applicable to abnormal detection of equipment operation status of the present invention; Figure 2 This is a schematic diagram of the Kmeans clustering results of the original data in this method; Figure 3 This is a schematic diagram of the statistical results of the coefficient of variation per unit time of the original data in this method; Figure 4 A schematic diagram showing the comparison between the adaptive threshold calculated by the present method and the threshold calculated by the conventional method; Figure 5 This is a schematic diagram comparing different calculation methods for motor bearing temperature thresholds; Figure 6 This is a schematic diagram comparing different calculation methods for pump bearing temperature thresholds; Figure 7 This is a schematic diagram comparing different threshold calculation methods when the vibration value changes intermittently; Figure 8 It is a schematic diagram comparing different threshold calculation methods when the vibration value trend increases; Fig. 9 This is a schematic diagram comparing different threshold calculation methods when there is a lot of missing motor current data; Fig.10 This is a schematic diagram comparing different threshold calculation methods when the motor current is relatively stable; Fig.11 This is a schematic diagram comparing different threshold calculation methods for regulating the liquid level in the pool; Fig.12 A schematic diagram comparing different calculation methods for lubricating oil tank level thresholds; Fig.13 This is a schematic diagram comparing different calculation methods of axis displacement thresholds; Fig.14 This is a schematic diagram comparing different threshold calculation methods for oil pressure in a lean oil station; Fig.15 This is a schematic diagram comparing different calculation methods for the threshold value of the rolling mill lubricating oil pressure. DETAILED DESCRIPTION

[0012] Implementation example Figure 1 As shown, the adaptive threshold calculation method for detecting abnormal operation status of equipment according to the present invention comprises the following steps: Step 1: Classify devices, distinguish device management domains, device categories, and data fields, and solidify the experience of manually setting thresholds into automatic calculation logic; Step 2: Data analysis: Analyze the patterns of the original data and distinguish numerical anomalies, data missing, and numerical segment changes based on statistical results; Step 3: Data preprocessing: remove noise from the original data, describe the law of the original data through the statistical value of the coefficient of variation, and determine whether it is clustered into multiple value intervals; Step 4: Data anomaly processing. Check whether there is any missing data. If the amount of missing data is within an acceptable range, continue to judge whether the value remains unchanged, filter out the cases where the value remains unchanged, the value is abnormally large, and the value is abnormally small. Then judge whether the existing data volume meets the minimum data volume requirement for calculating the threshold. If the data volume meets the minimum data volume requirement for calculating the threshold, use the mean and standard deviation to calculate the adaptive threshold.

[0013] Preferably, in step three, whether the data is clustered is determined by the statistical value of the coefficient of variation. If clustered, a clustering method is used to find two clusters of data with low and high values. If an upper limit needs to be set, a cluster of data with a high value is used to calculate the threshold, and then all the data before clustering are used to calculate the threshold, and the maximum value of the two is taken as the final adaptive threshold; if a lower limit needs to be set, a cluster of data with a low value is used to calculate the threshold, and then compared with the threshold calculated for all the data, and the minimum value of the two is taken as the final adaptive threshold; if not clustered, all the data after denoising are directly used to calculate the adaptive threshold.

[0014] Preferably, before calculating the adaptive threshold in step 4, it is first determined whether the automatic calculation logic of the manually set threshold has a definite mathematical formula. If so, the threshold is calculated according to the automatic calculation logic. If not, the adaptive threshold is calculated after filtering the original data using a box plot.

[0015] Preferably, the adaptive threshold calculation performs classified threshold calculation according to the device management domain, device category and data field.

[0016] Preferably, the calculated adaptive threshold is updated regularly, and historical data is periodically selected to calculate a new adaptive threshold, which is compared with the original threshold. If the difference is large, the threshold setting personnel is reminded to manually determine whether the new adaptive threshold is reasonable and whether to update the threshold.

[0017] In actual application, this method selects 600 data items from the operation and maintenance platform for 4 months. The threshold of each data item is calculated once a month. The background automatic operation is realized through the data interface, covering six data areas, including vibration, temperature, current, pressure, liquid level and displacement.

[0018] After reading the historical data, perform abnormal judgment and abnormal processing on the data. The specific implementation steps are as follows: (1) Due to the complexity of the industrial production environment, the stability of the data link is affected, resulting in long-term or intermittent data loss. There may be a large amount of missing data in the extracted historical data. By setting the minimum amount of data to decide whether to calculate the threshold, and giving the maximum number of days of data loss, it is convenient for the data operation and maintenance team to investigate and restore the cause of data loss. The adjustable parameter n is open to the user. The unit of n is day. If there is no data for n days, the threshold is not calculated during this period. The default value is n=1.

[0019] (2) The values ​​of some data items are abnormally large or small. For example, the bearing temperature is negative and exceeds the range and is displayed as -9999. The upper and lower limits are specified based on common sense and expert experience to filter out data points with abnormal values. The adjustable parameters up_threshold and down_threshold are open to users. Based on known expert experience and range, recommended upper and lower limits are provided for different equipment categories and data fields.

[0020] (3) Data points whose values ​​remain unchanged due to sensor anomalies, equipment downtime, etc. also need to be filtered out to avoid interference with the calculation of the threshold. First, the threshold for the value of the value remains unchanged is set to m days. The time for which the value of the extracted data remains unchanged is counted, and the data whose value remains unchanged for more than m days is filtered out. The adjustable parameter m is open to the user. The unit of m is day. If the value remains unchanged for m days, the threshold is not calculated during this period. The default value is m=1.

[0021] (4) After the preprocessing steps above, the amount of data after processing needs to be determined before calculating the threshold. If the amount of data is too small, the threshold is not calculated. The adjustable parameter k is open to the user. If the amount of data is less than k, the threshold is not calculated. The default value is k = 500.

[0022] After the above data anomaly judgment and processing, the data is filtered by box plot, and the threshold is calculated by selecting two data ranges and using the same calculation formula.

[0023] The first method is to use the coefficient of variation per unit time as the criterion for the Kmeans clustering algorithm. The coefficient of variation is a statistical indicator for measuring the degree of data fluctuation. The Kmeans clustering algorithm is an iterative clustering analysis algorithm. Whether to cluster and how many clusters to cluster into is determined by statistical analysis of a large amount of data. For 600 data items in six data fields, the coefficient of variation per unit time is statistically calculated and it is found that the value of the coefficient of variation is greater than 1.5 (see Figure 3 ), the data fluctuation changes significantly. At this time, the Kmeans clustering algorithm is used to cluster the data into two categories, and then the cluster with the larger value is used (see Figure 2 The standard value of the calculated threshold (data in the middle gray area) = mean + N times the standard deviation, threshold = R × the standard value of the threshold. The threshold calculated for this part of the data is defined as the first threshold, where N is the standard deviation coefficient, R is the threshold scaling coefficient, and the adjustable parameters N and R are open to the user.

[0024] The second method: use all filtered data to calculate the threshold, and use the same formula as the first method to determine the second threshold.

[0025] After calculating the first threshold and the second threshold respectively, the maximum value of the first threshold and the second threshold is selected as the final adaptive threshold. At the same time, a known expert threshold set according to manual experience is provided as a reference. If the value of the data taken is too low, resulting in a large difference between the calculated threshold and the expert threshold, the user is asked to confirm whether to adjust it to the expert threshold.

[0026] Take the threshold calculation results for September as an example. Figure 4 As shown in the figure, we can see that the threshold calculated by the conventional method is 443. The main reason is that the values ​​around September 15, 2023 are obviously abnormal, which cannot be filtered out by using the box plot filter, resulting in a large calculated standard deviation, which makes the calculated threshold too high. The adaptive threshold calculated by this method filters out abnormal situations such as values ​​that are too low and values ​​that remain unchanged based on manual experience before threshold calculation. The calculated threshold is around 100, which is close to the expert threshold set by manual experience and is more reasonable than the threshold calculated by the conventional method.

[0027] In order to verify the applicability of this method in multiple data fields, the thresholds are calculated in multiple data fields such as temperature, vibration, and voltage. The threshold calculation results of different methods are shown in Figures 5 to 15 .

[0028] like Figure 5 and Figure 6 As shown in the figure, the data pattern is that the values ​​change in segments. The lower values ​​may correspond to the state of equipment shutdown. When there is a lack of working condition parameters to judge shutdown, it is usually believed that the higher values ​​correspond to the running state of the equipment, and the higher value is used to calculate the threshold. In some periods, data suspected of shutdown state is obtained, and the calculated threshold is low. Exceeding this threshold does not reflect the abnormality of the equipment running state. If it is necessary to monitor the abnormality of equipment in non-operating state, a larger scaling factor can be multiplied on the calculated lower threshold to alarm before reaching the threshold of the running state.

[0029] like Figure 7 and Figure 8 As shown, the data pattern is that the values ​​fluctuate between high and low, and the overall trend rises slightly after a period of time. After preprocessing, the data is clustered, and the data near the larger value is used to calculate the threshold. This can give a higher threshold when the vibration rises as a whole, reducing false alarms that exceed the fixed threshold. Conventional methods calculate a larger standard deviation on data with large numerical differences, and then calculate a relatively large threshold. Equipment abnormalities may occur before the threshold is reached, and there is a risk of underreporting. This method selects data with smaller numerical differences through data preprocessing, and the calculated threshold is closer to the threshold given after manually observing the data, which can more accurately alert equipment abnormalities.

[0030] like Fig. 9 As shown, the data pattern is that there are more missing data, so the threshold is not calculated. The previous valid threshold is used, and the maximum duration of missing data is given.

[0031] like Fig.10 As shown, the data pattern is that the overall trend is stable, the values ​​are relatively concentrated in a range, and are significantly different from the expert threshold. The equipment status manager determines whether to modify it to the expert threshold.

[0032] Fig.11 and Fig.15 A schematic diagram comparing different threshold calculation methods for liquid level, displacement, and oil pressure is shown.

[0033] By comparing the adaptive threshold calculation and conventional threshold calculation of 600 data items in six data fields, it can be seen that the threshold calculated by the conventional method will not cause false alarms when the data is relatively stable, but there is a possibility of false alarms under some unstable and discontinuous data patterns. This method can calculate a more reasonable threshold after preprocessing the original data, thereby effectively avoiding false alarms or missed alarms caused by thresholds that are too large or too small.

[0034] This method is applicable to various data application and analysis technology fields of the equipment remote intelligent operation and maintenance platform. Its algorithm can also be extended to other online monitoring systems, and can provide efficient adaptive threshold calculation tools for intelligent operation and maintenance personnel.

Claims

1. An adaptive threshold calculation method suitable for detecting abnormal operation status of equipment, characterized in that The steps include: Step 1: Classify devices, distinguish device management domains, device categories, and data fields, and solidify the experience of manually setting thresholds into automatic calculation logic; Step 2: Data analysis: Analyze the patterns of the original data and distinguish numerical anomalies, data missing, and numerical segment changes based on statistical results; Step 3: Data preprocessing: remove noise from the original data, describe the law of the original data through the statistical value of the coefficient of variation, and determine whether it is clustered into multiple value intervals; Step 4: Data anomaly processing. Check whether there is any missing data. If the amount of missing data is within an acceptable range, continue to judge whether the value remains unchanged, filter out the cases where the value remains unchanged, the value is abnormally large, and the value is abnormally small. Then judge whether the existing data volume meets the minimum data volume requirement for calculating the threshold. If the data volume meets the minimum data volume requirement for calculating the threshold, use the mean and standard deviation to calculate the adaptive threshold.

2. The adaptive threshold calculation method for detecting abnormal operation status of equipment according to claim 1 is characterized in that: In the step 3, whether the data is clustered is determined by the statistical value of the coefficient of variation. If clustered, the clustering method is used to find two clusters of data with low and high values. If an upper limit value needs to be set, the threshold is calculated using the cluster of data with high values, and then the threshold is calculated using all the data before clustering, and the maximum value of the two is taken as the final adaptive threshold; if a lower limit value needs to be set, the threshold is calculated using the cluster of data with low values, and then compared with the threshold calculated for all the data, and the minimum value of the two is taken as the final adaptive threshold; If clustering is not performed, all the data after denoising is directly used to calculate the adaptive threshold.

3. The adaptive threshold calculation method for detecting abnormal operation status of equipment according to claim 1 or 2, characterized in that: Before calculating the adaptive threshold in step 4, it is first determined whether the automatic calculation logic of the manually set threshold has a definite mathematical formula. If so, the threshold is calculated according to the automatic calculation logic. If not, the adaptive threshold is calculated after filtering the original data using a box plot.

4. The adaptive threshold calculation method for detecting abnormal operation status of equipment according to claim 3, characterized in that: Adaptive threshold calculation performs classified threshold calculation according to device management domain, device category and data field.

5. The adaptive threshold calculation method for detecting abnormal operation status of equipment according to claim 3, characterized in that: The calculated adaptive threshold is updated regularly, and historical data is periodically selected to calculate a new adaptive threshold, which is compared with the original threshold. If the difference is large, the threshold setting personnel is reminded to manually determine whether the new adaptive threshold is reasonable and whether the threshold should be updated.

Images