Method and apparatus for generating dynamic thresholds for equipment early warning based on statistical models

Abstract

This application provides a method and apparatus for generating dynamic thresholds for device early warning based on a statistical model. The method includes: acquiring historical data from the most recent (n+1) days of operation of multiple devices, where n is a natural number greater than 1; processing the historical data from the most recent (n+1) days to generate device objects for the multiple devices; processing the device objects based on a statistical model to generate fitting calibration values ​​for result objects for the multiple devices; and using the result objects for the multiple devices to generate dynamic thresholds for early warning for each of the multiple devices. This method improves the accuracy of dynamic threshold prediction; encapsulating the mathematical models trained on multiple devices separately within the result objects facilitates network transmission and reduces the storage space occupied by the mathematical models.

Description

Technical Field

[0001] This invention relates to the field of equipment early warning technology, and more specifically, to a method and apparatus for generating dynamic thresholds for equipment early warning based on statistical models. Background Technology

[0002] Currently, anomaly detection technology is widely used in various industrial sectors, especially in the context of the Industrial Internet, where it is commonly used for the detection, early warning, and alarm functions of massive amounts of equipment. This is because, in the production management activities of manufacturing enterprises, real-time monitoring of the operating status of equipment or systems, and timely issuance of early warnings and alarms for abnormal states, along with prompt responses, are necessary measures to avoid production stoppages or business interruptions.

[0003] Most common anomaly detection technologies currently rely on pre-set static thresholds and pre-defined warning rules to provide early warnings or alarms for a large number of devices. However, static thresholds cannot adapt to normal data fluctuations caused by environmental factors, seasonal temperatures, local climate, and other influences on industrial equipment, leading to false alarms, such as false alerts or false alarms.

[0004] Therefore, it is necessary to use dynamic thresholds to provide early warnings or alarms for devices in order to effectively reduce or avoid false alarms. Summary of the Invention

[0005] To address the above issues, this application provides a method and apparatus for generating dynamic thresholds for equipment early warning based on a statistical model, thereby using dynamic thresholds for equipment early warning to effectively avoid the existing problem of false alarms for equipment.

[0006] In a first aspect, the present invention provides a method for generating dynamic thresholds for device early warning based on a statistical model, comprising:

[0007] Retrieve historical data for the most recent (n+1) days from the operation of multiple devices, where n is a natural number greater than 1;

[0008] Process the historical data of the most recent (n+1) days to generate device objects for the multiple devices;

[0009] Based on a statistical model, the device objects are processed to generate result objects for the multiple devices;

[0010] Using the result objects for the multiple devices, generate dynamic thresholds for device early warning for each of the multiple devices.

[0011] Further, the process of processing the historical data of the most recent (n+1) days to generate device objects for the multiple devices includes:

[0012] Preprocess the historical data of the most recent (n+1) days to generate a time series for each device with a time span of (p+1) days, where p is no greater than n;

[0013] The identifier corresponding to each device and the time series for each device are encapsulated in a device object for the multiple devices.

[0014] Further, the result object includes fitted calibration values ​​for each device; the step of processing the device object based on a statistical model to generate a result object for the multiple devices includes:

[0015] Extract time series for each device from the device object, and divide the time series into training samples and fitting samples;

[0016] Based on the cubic exponential smoothing statistical model, the training samples are processed to determine the cubic exponential smoothing mathematical model for each device.

[0017] Based on the cubic exponential smoothing mathematical model for each device, the training samples and the fitted samples are processed to determine the fitted adjustment value for each device.

[0018] The identifier corresponding to each device, along with the cubic exponential smoothing mathematical model and the fitted adjustment value for each device, are encapsulated in a result object for the multiple devices.

[0019] Further, dividing the time series into training samples and fitted samples includes:

[0020] The sampling points of the first q days in the time series with a time span of (p+1) days are used as training samples, and the sampling points of the last (p-q+1) days in the time series with a time span of (p+1) days are used as fitting samples.

[0021] Furthermore, the step of processing the training samples based on the cubic exponential smoothing statistical model to determine the cubic exponential smoothing mathematical model for each device includes:

[0022] Determine the initial values ​​of the ternary smoothing parameters in the cubic exponential smoothing statistical model;

[0023] Determine the initial values ​​for the smoothed value sequence, trend component value sequence, and seasonal component value sequence in the triple exponential smoothing statistical model;

[0024] Based on the training samples, the initial values ​​of the ternary smoothing parameters, the initial values ​​of the smoothed value sequence, the trend component value sequence, and the seasonal component value sequence, the optimal estimated values ​​of the ternary smoothing parameters, the smoothed value sequence, and the seasonal component value sequence in the triple exponential smoothing statistical model are determined using the least squares method.

[0025] Determine the cubic exponential smoothing mathematical model for each device, described by the optimal estimates of the respective ternary smoothing parameters, the smoothed value sequence, the trend component value sequence, and the seasonal component value sequence, and the most recent data point.

[0026] Further, the step of processing the training samples and the fitted samples according to the cubic exponential smoothing mathematical model for each device to determine the fitted adjustment value for each device includes:

[0027] Determine the initial values ​​for the fit calibration values ​​for each device;

[0028] Based on the training samples, the cubic exponential smoothing mathematical model, and the initial values ​​of the fitted calibration values, predict the calibration samples for each device;

[0029] Based on the calibration samples and the fitted samples, the fitted calibration values ​​for each device are determined using the least squares method.

[0030] Furthermore, the dynamic threshold is used to issue warnings for the device at multiple time periods within at least one specified natural day.

[0031] Secondly, the present invention provides an apparatus for generating dynamic thresholds for device early warning based on a statistical model, comprising:

[0032] The acquisition unit is used to acquire historical data of the most recent (n+1) days of operation of multiple devices, where n is a natural number greater than 1;

[0033] The device object processing unit is used to process the historical data of the most recent (n+1) days and generate device objects for the multiple devices;

[0034] The result object generation unit is used to process the device objects based on a statistical model and generate result objects for the multiple devices.

[0035] The dynamic threshold generation unit is used to generate dynamic thresholds for device early warning for each of the multiple devices using the result objects for the multiple devices.

[0036] Thirdly, the present invention provides a computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program implementing the steps of the method described in the first aspect.

[0037] Fourthly, the present invention provides a device early warning method, comprising: generating a dynamic threshold according to any of the methods described in the first aspect; and using the dynamic threshold to issue early warnings for multiple devices respectively.

[0038] These and other aspects of this application will become more apparent in the description of the following embodiments(s). Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating a method for generating dynamic thresholds for device early warning based on a statistical model, according to an embodiment of the present invention.

[0041] Figure 2 This is a schematic diagram of the device for generating dynamic thresholds for device early warning based on a statistical model, according to an embodiment of the present invention.

[0042] Figure 3 This is a schematic diagram illustrating the composition of a system for generating dynamic thresholds for device early warning based on a statistical model, according to an embodiment of the present invention.

[0043] Figure 4 The flowchart illustrates another method for generating dynamic thresholds for device early warning based on a statistical model, according to an embodiment of the present invention.

[0044] Figure 5 This is a performance comparison diagram of the method for generating dynamic thresholds for device early warning based on a statistical model according to an embodiment of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] In order to accurately describe the technical content of this application and to accurately understand this application, the following explanations or definitions of the terms used in this specification are given before describing the specific embodiments.

[0047] A time series is a set of observations or records of the same variable arranged in chronological order of their occurrence.

[0048] Sampling is the process of representing a continuously occurring analog quantity using discrete data points arranged in chronological order. For example, by sampling the amplitude values ​​of a continuous signal on a time axis at fixed or variable time intervals, the continuous signal becomes a discrete pulse sequence. The time interval between samples is called the sampling period (in seconds or s), and its reciprocal is called the sampling frequency (in samples per second or samples / s, or Hertz (Hz)). Continuous sampling, arranging the values ​​of each sample point in chronological order, also forms a time series.

[0049] Autoregressive Moving Average (ARMA) is an important method for processing time series data.

[0050] Exponential smoothing, or exponential moving average, is a moving average with exponentially decreasing weights. The weights of each data point decrease exponentially over time, with data points more recent to the present having higher weights.

[0051] The root mean square error (RMSE) is the square root of the ratio of the sum of squares of the deviations between the observed values ​​and the true values ​​to the number of observations.

[0052] The open-source library statsmodels is a Python library package for statistical analysis, such as testing the linear significance of a model when the mathematical model is unknown, hypothesis testing, regression analysis, and time series analysis. The Holt-Winters function in statsmodels can only generate a mathematical model for a single device's time series data at a time. In the application scenarios of the Industrial Internet with massive numbers of devices, these devices will correspond to a massive number of models. These massive models not only lead to maintenance difficulties but also hinder network transmission and interaction between network nodes.

[0053] When using deep learning to train mathematical models, a large amount of data needs to be acquired and a large amount of computing resources are required, such as NPUs and GPUs. After consuming a lot of computing time, the trained mathematical model can finally converge.

[0054] like Figure 1 As shown, the method for generating dynamic thresholds for device early warning based on a statistical model according to an embodiment of the present invention includes:

[0055] S10: Obtain historical data for the most recent (n+1) days of operation from multiple devices, where n is a natural number greater than 1;

[0056] S20: Process the historical data of the most recent (n+1) days to generate device objects for the multiple devices;

[0057] S30: Based on the statistical model, process the device objects and generate result objects for the multiple devices;

[0058] S40: Using the result objects for the plurality of devices, generate dynamic thresholds for device early warning for each of the plurality of devices.

[0059] In some embodiments, steps S10 to S30 are executed periodically at a first time interval, such as a week, a calendar month, or a quarter. In this case, in step S10, the historical data obtained for the most recent (n+1) days has n that is not less than 7, not less than 28, or not less than 120.

[0060] In some embodiments, after step S30, step S40 is executed periodically at a second time interval, such as one calendar day or two calendar days. Accordingly, the dynamic thresholds generated in step S40 for each of the plurality of devices are used to issue warnings for the devices at multiple time periods within a specified calendar day or two calendar days (such as every half hour in a 24-hour period of a calendar day).

[0061] In this way, dynamic thresholds for device early warning can be generated periodically or aperiodically using the result objects at shorter time intervals, and historical data of device operation can be acquired periodically or aperiodically at longer time intervals, generating device objects and mathematical model objects for multiple devices.

[0062] In step S10, historical data can be obtained from a data warehouse managed by a cloud server via a web service, or from an edge data warehouse, such as a data warehouse deployed at the edge of a smart factory using a large number of smart sensing devices set up based on the Industrial Internet.

[0063] In step S10, multiple devices can be multiple devices that are geographically close to each other, or multiple nodes at the same level or different levels in the Industrial Internet.

[0064] In some embodiments, processing the historical data of the most recent (n+1) days to generate device objects for the plurality of devices includes: preprocessing the historical data of the most recent (n+1) days to generate a time series for each device with a time span of (p+1) days, where p is not greater than n; and encapsulating the identifier corresponding to each device and the time series for each device in a device object for the plurality of devices.

[0065] Naturally, corresponding to the authenticity and validity of historical data, the time spans of the time series generated from the most recent (n+1) days of historical data from different devices after preprocessing may vary. However, as long as the minimum number of days is met, the time series for that device is considered valid and can be encapsulated into the corresponding device object for use as training data for mathematical models. During training, the time span corresponding to each time series can be quickly determined in advance. Specifically, preprocessing the most recent (n+1) days of historical data for each device to generate a time series with a time span of (p+1) days for each device can be referred to step S32 described later.

[0066] The above approach encapsulates the identifier for each device and the time series for each device within a device object for all devices. This can be achieved by creating a device class with the device identifier and the time series for each device within a Python integrated development environment (IDE), and then instantiating this device class to obtain multiple device objects for different devices, or a single device object for all devices. Alternatively, it can be done using the deserialization operation of Python's pickle module to create a Python-supported device object from a file (such as an Excel file) that stores the device identifier and the device's time series.

[0067] In some embodiments, the result object includes fitted calibration values ​​for each device; the step of processing the device object based on a statistical model to generate a result object for the multiple devices includes:

[0068] Extract time series for each device from the device object, and divide the time series into training samples and fitting samples;

[0069] Based on the cubic exponential smoothing statistical model, the training samples are processed to determine the cubic exponential smoothing mathematical model for each device.

[0070] Based on the cubic exponential smoothing mathematical model for each device, the training samples and the fitted samples are processed to determine the fitted adjustment value for each device.

[0071] The identifier corresponding to each device, along with the cubic exponential smoothing mathematical model and the fitted adjustment value for each device, are encapsulated in a result object for the multiple devices.

[0072] Subsequently, through the serialization operation of the pickle module supported by Python, the result objects of multiple devices in the Python program can be saved to a file. This allows for permanent storage without the need for a Python integrated development environment, and convenient exchange between local storage, databases, and different storage media.

[0073] In some embodiments, dividing the time series into training samples and fitting samples includes: using the sampling points of the first q days in the time series with a time span of (p+1) days as training samples, and using the sampling points of the last (p-q+1) days in the time series with a time span of (p+1) days as fitting samples, where q is a natural number less than or equal to p.

[0074] Based on the requirements for model training accuracy and fitting calibration accuracy, the time series can be flexibly divided into training samples and fitting samples that have a temporal relationship. Here, by default, the time series is divided according to the number of sampling points within each natural day as the smallest unit; that is, the time spans corresponding to the training samples and fitting samples are integer multiples of the number of natural days. Furthermore, the time span corresponding to the training samples is usually larger than the time span corresponding to the fitting samples; for example, the time span corresponding to the training samples is p, and the time span corresponding to the fitting samples is 1.

[0075] In some embodiments, processing the training samples based on the cubic exponential smoothing statistical model to determine the cubic exponential smoothing mathematical model for each device includes:

[0076] Determine the initial values ​​of the ternary smoothing parameters in the cubic exponential smoothing statistical model;

[0077] Determine the initial values ​​for the smoothed value sequence, trend component value sequence, and seasonal component value sequence in the triple exponential smoothing statistical model;

[0078] Based on the training samples, the initial values ​​of the ternary smoothing parameters, the initial values ​​of the smoothed value sequence, the trend component value sequence, and the seasonal component value sequence, the optimal estimated values ​​of the ternary smoothing parameters, the smoothed value sequence, and the seasonal component value sequence in the triple exponential smoothing statistical model are determined using the least squares method.

[0079] Determine the cubic exponential smoothing mathematical model for each device, described by the optimal estimates of the respective ternary smoothing parameters, the smoothed value sequence, the trend component value sequence, and the seasonal component value sequence, and the most recent data point.

[0080] In some embodiments, the initial values ​​of the ternary smoothing parameters in the triple exponential smoothing statistical model can be determined with reference to the description in section 3.2 below.

[0081] In some embodiments, the initial values ​​of the smoothed value sequence, trend component value sequence, and seasonal component value sequence in the triple exponential smoothing statistical model can be determined with reference to the description in section 3.2 below.

[0082] In some embodiments, based on the training samples, the initial values ​​of the ternary smoothing parameters, the initial values ​​of the smoothed value sequence, the trend component value sequence, and the seasonal component value sequence, the optimal estimated values ​​of the ternary smoothing parameters in the triple exponential smoothing statistical model are determined using the least squares method. The smoothed value sequence, the trend component value sequence, and the seasonal component value sequence can be referred to the description in sections 3.2) and 3.3) below.

[0083] In some embodiments, the Holt-Winters function supported by the Python language can be used for secondary development to implement the cubic exponential smoothing statistical model, such as the myHolt-Winters function. In this case, the training samples, the initial values ​​of the ternary smoothing parameters, the initial values ​​of the smoothed value sequence, the trend component value sequence, and the seasonal component value sequence, as well as the number of sampling points k corresponding to the inherent cycle period of the seasonal component value sequence, are the input parameter values ​​required for the myHolt-Winters function to run.

[0084] In some embodiments, the determination of the cubic exponential smoothing mathematical model for each device, described by the optimal estimates of the respective ternary smoothing parameters, the smoothing value sequence, the trend component value sequence, and the seasonal component value sequence, can be made with reference to the description in section 3.4 below.

[0085] Thus, by removing historical training data and intermediate data generated during the calculation process when generating result objects, the storage space occupied by mathematical models corresponding to multiple devices is further reduced. The output result object files are smaller; for example, compared to the result object files generated directly using ARMA in statsmodels, storage space can be saved by 90%. For instance, a mathematical model for motor bearing temperature, when saved as a .pkl file using Python's pickle module, is approximately 3.14kb in size, compared to a 10.5MB file generated using the ARMA model in statsmodels, saving at least 90% of storage space.

[0086] In some embodiments, the step of processing the training samples and the fitted samples according to a cubic exponential smoothing mathematical model for each device to determine the fitted calibration value for each device includes:

[0087] Determine the initial values ​​for the fit calibration values ​​for each device;

[0088] Based on the training samples, the cubic exponential smoothing mathematical model, and the initial values ​​of the fitted calibration values, predict the calibration samples for each device;

[0089] Based on the calibration samples and the fitted samples, the fitted calibration values ​​for each device are determined using the least squares method.

[0090] Thus, compared to deep learning, the method of this invention is based on a cubic exponential smoothing statistical model, which can train mathematical models for multiple devices with a smaller amount of data. The function based on the cubic exponential smoothing statistical model is implemented in batch processing using the secondary development source code myHolt-Winters function, which can simultaneously receive the time series of multiple devices as input to the function, and encapsulate the result objects obtained by training with the time series of multiple devices as the output of the function. The result objects for the multiple devices are saved to a file, reducing the storage space occupied by the mathematical models corresponding to multiple devices.

[0091] Figure 3 The diagram illustrates a structural block diagram of an equipment early warning system deployed using a method for generating dynamic thresholds based on statistical models according to an embodiment of the present invention. This embodiment's equipment early warning method generates dynamic thresholds based on any of the foregoing described methods for generating dynamic thresholds for equipment early warning using statistical models, and then uses these dynamic thresholds to issue early warnings for multiple devices.

[0092] The system includes an equipment database 1, a mathematical model 2, and early warning rules 3. Equipment database 1, for example, runs on a database management device and is used to store massive amounts of historical data generated by a vast number of devices in the industrial field during operation, as well as periodically updated dynamic thresholds. The periodically updated dynamic thresholds are used to issue equipment early warnings based on the early warning rules 3 described below.

[0093] After preprocessing the historical data of the devices, mathematical model 2 obtains training samples and fitted samples. Based on the training samples of each device, a mathematical model for each device is trained using a cubic exponential smoothing statistical model, and the final mathematical model is generated after adjustment. Based on the fitted samples, the aforementioned cubic exponential smoothing statistical model is used for prediction to obtain a predicted value sequence for each device. The predicted value sequences corresponding to multiple devices are stored in device database 1, which can be accessed via network interconnection to obtain the predicted value sequences corresponding to each device for device early warning.

[0094] Early warning rule 3 retrieves the predicted value sequence corresponding to the device identifier from the device database. Based on the retrieved current time, and using pre-set early warning rules, it selects the predicted value corresponding to the current time from the predicted value sequence as a dynamic threshold for device early warning. For example, if the time span corresponding to the device's predicted value sequence is one calendar day and the corresponding update interval is 30 minutes, the device's predicted value sequence includes 48 values. The number of minutes between the current time and 00:00:00 of the current day is calculated, and this number of minutes is divided by 30. After removing the remainder from the quotient, the value 'a' is obtained. The (a+1)th value in the predicted value sequence is the dynamic threshold corresponding to the current time. For example, if the current time is 2022-4-11 00:15:00, then the number of minutes between the current time and 00:00:00 on the same day is 15. Dividing this number of minutes by 30 and removing the remainder from the quotient results in a value of 0. The (0+1)th value in the predicted value sequence is the dynamic threshold corresponding to the current time. The first value in the predicted value sequence is then used as the dynamic threshold for the current time to trigger an equipment warning. This is done with each 30-minute period as a timeframe. That is, when the current time is between 2022-4-11 00:00:00 and 2022-4-11 00:30:00, the first value in the predicted value sequence is used as the dynamic threshold.

[0095] If the current time is 2022-4-11 00:35:00, then the number of minutes between this current time and 00:00:00 on the same day is 35. Dividing this number of minutes by 30 and removing the remainder from the quotient yields a value of 1. The (1+1)th value in the predicted value sequence is the dynamic threshold corresponding to the current time. The first value in the predicted value sequence is then taken as the threshold for the current time period. Furthermore, each time period is still defined as 30 minutes. That is, when the current time is between 2022-4-11 00:31:00 and 2022-4-11 01:00:00, the second value in the predicted value sequence is used as the dynamic threshold for device warnings. The dynamic thresholds for other times on the same day are determined as described above and will not be repeated here. Thus, within a natural day, the dynamic threshold for each moment or time period is a value from the aforementioned predicted value sequence.

[0096] Thus, the resulting early warning rule is: if the temperature value of device A at the current time is greater than the corresponding dynamic threshold, an alarm will be triggered for device A. For example... Figure 4 As shown, the method of this embodiment of the invention, when periodically updating the mathematical model with a first time interval of one natural day and periodically generating dynamic thresholds with a second time interval of one natural day, includes the following steps S31 to S35.

[0097] Step S31: At a fixed time of 00:00:00 every day (e.g., set a rule to trigger automatically), retrieve the historical data of multiple devices for the most recent (n+1) days from the device database, where the value of n is configured by the user.

[0098] Historical data can include sampling of data with different physical meanings (such as state variables or index variables) generated by different devices or the same device during operation. It can be a time series arranged chronologically according to the sampling time. These data can correspond to the same sampling moment on the time axis and have the same sampling period. Different time series can have the same number of sampling points, i.e., the same time span. For example, the temperature and speed of a single motor can be collected during long-term continuous operation, forming two historical data sets for that motor, corresponding to a temperature time series and a speed time series, respectively. Alternatively, the temperature and speed of multiple motors with the same or different power ratings on the same production line can be collected at the same sampling frequency during long-term continuous operation, forming multiple temperature and speed time series, which serve as the historical data for each motor on that production line.

[0099] In step S32 described later, the historical data of multiple devices for the most recent (n+1) days is preprocessed into historical data of multiple devices for the most recent (p+1) days or (n+1) days.

[0100] In step S33, data from the previous n days or p days (i.e., the days before the current day) are used as training samples; in step S34, data from the previous day (i.e., the day before the current day) (which is more recent in time) are used as fitting samples for calculating the fitting adjustment value j.

[0101] Step S32: Process the acquired historical data. For example, process the historical data of multiple devices for the most recent (n+1) days to obtain device objects corresponding to multiple devices.

[0102] The following example illustrates the specific preprocessing operation, using the preprocessing of historical data from the most recent (n+1) days for a single device.

[0103] 2.1) Divide the acquired data by day, i.e. calendar day. Corresponding to the historical data of the device for the most recent (n+1) days obtained in step S31 above, divide the data to obtain the data for each of the (n+1) days, and add the corresponding timestamp for each day.

[0104] 2.2) Perform duplicate data processing, discard processing, or padding processing on the data for each day.

[0105] First, divide each natural day's 24 hours into m time periods and m time points according to a specified time interval, and determine m corresponding timestamps. For example, if the specified time interval is 30 minutes, this results in 48 time periods and 48 time points, with 48 corresponding timestamps determined.

[0106] Typically, the acquired data contains data that is time-repeated. The reasons for this duplication usually include factors such as the sampling period and network transmission. For example, due to fluctuations in the sampling period, network transmission latency, and network transmission quality, for a time series with a sampling period of 30 seconds, under normal sampling and transmission, the first data point of the day might be acquired at 00:00:30. However, due to network issues, the data might be received by the device database at 00:00:31 or another time. For instance, some device data might have a sampling period in the millisecond range (such as the vibration of a motor bearing), meaning multiple vibration values ​​are collected per second; while other device data might have a sampling period in the second range (such as the temperature of a motor bearing), meaning only one temperature value is collected per second. When these different types of data are entered into the database at specified time intervals, if the sampling period is shorter than the specified time interval, multiple data points will appear within that specified time interval, resulting in duplication relative to that specified time interval. Network transmission factors mainly involve data reporting delays caused by network issues. For example, if the previously collected data is delayed in reporting, it might be reported at the same time as the currently collected data, thus creating time-repeated data.

[0107] Therefore, based on the m time periods, time points, or timestamps, duplicate data can be removed one by one, i.e., deduplication. Deduplication methods can include selecting the earliest or latest data in time from the data that are duplicated in time, discarding the other data, or selecting the maximum, minimum, median, etc., or taking the algebraic average or ensemble average of multiple data that are duplicated in time.

[0108] If, after deduplication, the number of data points for a particular day is less than the preset m data points, it means that at least one data point for that day's defined time period is missing (e.g., data will not reach the device database if packet loss occurs). In this case, the data for that day is discarded. For example, if, at a specified time interval of 30 minutes, after deduplication of a day's data, the number of data points is less than 48, then the data for that day is discarded and will not be added to the time series in subsequent steps. This is because if the number of data points for a particular day is too small or insufficient, the data for that day is no longer meaningful for training the model or adjusting the fit values; therefore, the data for that day is discarded, which is the aforementioned discarding process.

[0109] Furthermore, when packet loss occurs, data will not reach the device database, resulting in empty data points for the corresponding time period or moment. Therefore, this can be remedied by using the first data point of the day as the value of 00:00:00 (aligning to the first time period), and then using the (i+1)th data point of the day as the value of (00:00 + 30 * i:00) (aligning to the (i+1)th time period), until the last data point is used as the value of 23:59:59. This results in a total of m data points, such as 48 data points, each with its own corresponding timestamp, which is achieved through pre- and post-padding or time-series alignment.

[0110] Thus, by supplementing the beginning and end of the data and aligning the time sequence, the data for each day is processed into a time series containing a preset number of m data points. Here, m must be an integer multiple of k so that in step S35 described later, when resampling is performed according to a preset sampling period, such as 30 minutes, the resampled time series will contain k = 48 data points that are staggered in time for each natural day.

[0111] 2.3) After performing duplicate data processing, discarding data processing, or padding processing on the beginning and end of the data for each day, the final number of days corresponding to the data is obtained (p+1). At this time, p is a natural number less than or equal to n.

[0112] 2.4) If n is greater than p, it means that at least one day of historical data has been discarded. For the timestamp corresponding to the discarded data, the gap is filled with data from the previous day, and subsequent (later in time) data are shifted forward in time, ultimately ensuring that the remaining data from the most recent (p+1) days form a continuous time series. Alternatively, the gap is filled with data from the next day, and preceding (earlier in time) data are shifted backward in time, ultimately ensuring that the remaining data from the most recent (p+1) days form a continuous time series.

[0113] 2.5) Resample the daily data using a longer sampling period to obtain resampled data, thereby reducing the amount of data. For example, increasing the sampling period from 30 seconds to 30 minutes reduces the number of data points to 1 / 60 of the original number.

[0114] In industrial internet scenarios, factories or production lines contain tens of thousands of devices, and monitoring their operational status generates at least hundreds of millions of sampling points daily. If raw data is used for model training, the sheer volume of data means that training a single model could take several hours or even tens of hours, a computationally burdensome task for device prediction. Therefore, resampling using longer sampling periods reduces the amount of data, shortens model training time, and decreases the consumption of computational resources.

[0115] 2.6) The data from the previous multiple days, i.e. (p+1) days, after resampling, are concatenated into a time series with a time span of (p+1) days in chronological order.

[0116] 2.7) Encapsulate the above time series with a time span of (p+1) days and the identifier corresponding to the single device (such as the unique identifier id) into a device object that runs in a Python program.

[0117] For the preprocessing of historical data from multiple devices over the most recent (n+1) days, it can be implemented as described in sections 2.1) to 2.7 above, and will not be repeated here.

[0118] The historical data of the multiple devices for the most recent (n+1) days are processed into device objects running in a Python program. The device objects corresponding to the multiple devices are then uniformly encapsulated into the device object used in step 33 described later, and can be conveniently distributed over the network.

[0119] Step S33: Using the data processed in step S32, i.e., the device objects, train mathematical models corresponding to multiple devices respectively. Specifically, this includes:

[0120] 3.1) Traverse the encapsulated device objects in step S32, sequentially obtain the device objects corresponding to each device, and extract the time series x = {x} corresponding to the device with a time span of (p+1) days from the device objects. t}, where 0≦t≦m*(p+1)-1, and m is the number of time periods corresponding to a natural day or the number of data points after resampling.

[0121] 3.2) Using the triple exponential smoothing statistical model, the time series x = {x} t}, train the mathematical model corresponding to each device.

[0122] Specifically, when using the triple exponential smoothing statistical model as the mathematical model, the following formulas (1.1) to (1.4) are included:

[0123] s t =α(x t +P t-k )+(1-α)(s t-1 +T t-1 (1.1)

[0124] T t =β(s) t -s t-1 )+(1-β)T t-1 (1.2)

[0125] P t =γ(x t -s t )+(1-γ)P t-k (1.3)

[0126] x t+h =s t +hT t +P t-k+g (1.4)

[0127] The smoothed value sequence is denoted as s = {s t The trend component value sequence is denoted as T = {T}. t The seasonal component value sequence is denoted as P = {P}. tIn this time series, the seasonal component value sequence has an inherent cyclic period, within which the corresponding number of sampling points is k. In the above time series, the subscript t indicates the current time or the t-th data point; t-1 indicates the time 1 before the current time or the (t-1)-th data point; tk indicates the k times before the current time or the tk-th data point; t-k+g indicates the k times before the current time or the t-k+g-th data point; t+h represents the h times after the current time or the t+h-th value; and the variable g is the remainder of h divided by k, that is, the remainder after dividing h and k, meaning g is less than or equal to h and less than or equal to k.

[0128] Variables α, β, and γ are smoothing parameters, taking values ​​between 0 and 1, which are the aforementioned ternary smoothing parameters. Variable k corresponds to the inherent cycle of the seasonal component value sequence. When the cycle is one natural day and the data sampling period is 30 minutes, k is 48. Variable h indicates the h-th data point predicted from the current time t, 0≦h≦m*(p+1). That is, the mathematical model trained using historical data of (p+1) natural days can predict the data of each device's operation within a maximum of (p+1) natural days. The initial values ​​of the smoothed value sequence s, the trend component value sequence T, and the seasonal component value sequence P have little impact on the triple exponential smoothing statistical model. Therefore, for simplicity, s0 = x0, T0 = x1 - x0, and P0 = 0 are usually taken.

[0129] Given a time series x = {x} with a time span of (p+1) days. t Data from the previous n days or the previous (p+1) days that are further back in time are used as training samples; data from the previous day that is further back in time (i.e., the day before the current day) are used as fitting samples to calculate the fitting calibration value j.

[0130] The optimal estimates of the smoothing parameters α, β, and γ are determined using the least squares estimation method based on the training samples. Specifically, the loss function is constructed using the `fmin_l_bfgs_b` function from the `scipy.optimize` package provided by Python, according to the following equation (1.5):

[0131]

[0132] Where, x #Let y be the sequence of predicted values ​​predicted by the cubic exponential smoothing statistical model, and q be the number of data points in the training sample or the predicted value sequence, q = k*p. When the cycle period is one natural day and the data sampling period is 30 minutes, q = 48*p. The loss function can be implemented in code, such as using Python. For example, the initial values ​​of the smoothing parameters α, β, and γ, such as the array [0.3, 0.1, 0.1], and their value ranges, such as the array [(0,1), (0,1), (0,1)], can be passed to the fmin_l_bfgs_b function supported by the Python package to obtain the optimized values ​​of α, β, and γ, which are the optimal estimates of α, β, and γ in the least squares sense.

[0133] 3.3) Substitute the optimized values ​​of α, β, and γ determined in step 3.2) into formulas (1.1) to (1.4) of the cubic exponential smoothing statistical model to calculate the smoothed value sequence s = {s} corresponding to the specified time span r. t}, Trend component value sequence T={T t}, the seasonal component value sequence P = {P t}、Predicted value sequence y * In this case, r is an integer multiple of the number of data points k in the aforementioned cycle.

[0134] 3.4) The smoothed value sequence s = {s} determined in step 3.3) above. t The last value of}, the trend component value sequence T={T t The last value of}, the seasonal component value sequence P = {P t The values ​​in the last cycle of}, i.e. the most recent data point mentioned above, the optimized values ​​of α, β, and γ determined in step 3.2), the default fitting calibration value j=1 and the identifier corresponding to each device, are encapsulated into an initial result object for the aforementioned device.

[0135] That is, the time series x with a time span of (p+1) days corresponding to the device extracted in step 3.1) above, and the smoothed value sequence s = {s} calculated in step 3.3) above are removed. t}, Trend component value sequence T={T t}, the seasonal component value sequence P = {P t Remove the data after the most recent data point from the file. This reduces the size of the final mathematical model file.

[0136] Repeat steps 3.2) to 3.4) above to obtain the initial result objects for each device after encapsulation.

[0137] 3.5) Encapsulate the initial result objects of multiple devices into a result object group as a preliminary mathematical model.

[0138] Step S34: Calculate the fitting calibration value j for each device using the fitted samples. For example, use the least squares estimation method to determine the optimal estimated value of the fitting calibration value for each device. Specifically, this includes:

[0139] Using the mathematical models of each device determined in step S33 above, the data of each device for the previous day (i.e., the day before the current day) are predicted to obtain the prediction results. Each data value in the prediction results is obtained by multiplying the value calculated using the triple exponential smoothing statistical model by the default fitting adjustment value j.

[0140] The optimal estimate of the fitted calibration value j is determined using the least squares estimation method based on the fitted samples. Specifically, the loss function is constructed using the `fmin_l_bfgs_b` function from the `scipy.optimize` package provided by Python, according to the following equation (1.6):

[0141]

[0142] Where, x ^ For the fitted sample, y ^ The prediction result data is obtained by using the mathematical model of the equipment determined in step S33 above to predict the data of the equipment from the previous day (i.e., the day before the current day). N is the number of data points in the fitted sample or prediction result data, and N is the number of data points k in the cycle corresponding to the aforementioned seasonal component value sequence. When the cycle is 1 natural day and the data sampling period is 30 minutes, N is 48. The above loss function is implemented in code, such as using Python, to obtain the optimal estimate of the fitted calibration value j. The optimal estimate of the fitted calibration value j is updated to the corresponding result object, that is, the fitted calibration value j corresponding to the equipment is updated in the encapsulated result object group.

[0143] Repeat the above steps until the fit calibration value j for all devices is updated. This will reduce the error in the prediction results and improve the prediction accuracy.

[0144] Step S35: Use the previously trained mathematical model object group to predict data for at least one specified future calendar day of the device. Specifically, this includes:

[0145] Iterate through multiple devices to obtain their corresponding identifiers. Input the device identifier and a specified prediction period (e.g., one calendar day) into the mathematical model to obtain a sequence of predicted values ​​x for each device over the next specified calendar day. h .

[0146] Step S36: The predicted value sequence x obtained in step S35 h Each value in the prediction is multiplied by the fitting adjustment value j determined in step S34 above to obtain the final prediction result, which is stored in the device database as a dynamic threshold for use by the early warning rules.

[0147] The triple exponential smoothing statistical model (e.g., implemented as myHolt-Winters), Holt-Winters in statsmodels, and ARMA in statsmodels are used to process the test data (including historical data from two devices) stored in a local Excel file to obtain prediction results. Performance comparison experiments are then conducted on prediction accuracy or error, training time, and the file size of the mathematical model, etc. Figure 5 As shown.

[0148] like Figure 5 As shown, the smaller the Mean Square Error (MSE) value, the closer the predicted value is to the actual value. It can be seen that the cubic exponential smoothing statistical model of this invention, while achieving high accuracy with relatively small loss, shortens the training time and reduces the file size of the mathematical model.

[0149] Thus, the method for generating dynamic thresholds for device early warning based on statistical models in this embodiment of the invention encapsulates the mathematical models trained on multiple devices as result objects within a single result object, facilitating network transmission; by removing historical data of the devices and intermediate data generated during calculation, the result object retains only the most recent data points of each time series (the latest value or multiple values ​​in time), which helps reduce the storage space occupied by the mathematical model; and by introducing fitting calibration values, the accuracy of dynamic threshold prediction is further improved, and errors are reduced.

[0150] In some embodiments, such as Figure 2 As shown, the apparatus for generating dynamic thresholds for device early warning based on a statistical model according to an embodiment of the present invention includes:

[0151] Acquisition unit 100 is used to acquire historical data of the most recent (n+1) days of operation of multiple devices, where n is a natural number greater than 1;

[0152] The device object processing unit 200 is used to process the historical data of the most recent (n+1) days and generate device objects for the multiple devices;

[0153] Result object generation unit 300 is used to process the device object based on a statistical model and generate result objects for the multiple devices;

[0154] The dynamic threshold generation unit 400 is used to generate dynamic thresholds for device early warning for each of the plurality of devices using result objects for the plurality of devices.

[0155] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0156] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0158] The above description is illustrative only and not restrictive of the present invention. Those skilled in the art will understand that many modifications, variations or equivalents can be made without departing from the spirit and scope defined by the claims, and all such modifications, variations or equivalents will fall within the protection scope of the present invention.

Claims

1. A method for generating dynamic thresholds for equipment early warning based on a statistical model, characterized in that, include: Retrieve historical data for the most recent (n+1) days of operation from multiple devices, where n is a natural number greater than 1; Process the historical data of the most recent (n+1) days to generate device objects for the multiple devices; Based on a statistical model, the device objects are processed to generate result objects for the multiple devices; Using the result objects for the multiple devices, generate dynamic thresholds for device early warning for each of the multiple devices; The result object includes fitted calibration values ​​for each device; the step of processing the device object based on a statistical model to generate a result object for the multiple devices includes: Extract time series data for each device from the device object, and divide the time series data into training samples and fitted samples; process the training samples based on the cubic exponential smoothing statistical model to determine the cubic exponential smoothing mathematical model for each device; process the training samples and the fitted samples according to the cubic exponential smoothing mathematical model for each device to determine the fitted adjustment value for each device; encapsulate the identifier corresponding to each device, the cubic exponential smoothing mathematical model for each device, and the fitted adjustment value in a result object for the multiple devices. The step of processing the training samples based on the cubic exponential smoothing statistical model to determine the cubic exponential smoothing mathematical model for each device includes: Determine the initial values ​​of the ternary smoothing parameters in the cubic exponential smoothing statistical model; determine the initial values ​​of the smoothed value sequence, trend component value sequence, and seasonal component value sequence in the cubic exponential smoothing statistical model; based on the training samples, the initial values ​​of the ternary smoothing parameters, and the initial values ​​of the smoothed value sequence, trend component value sequence, and seasonal component value sequence, determine the optimal estimates of the ternary smoothing parameters, the smoothed value sequence, the trend component value sequence, and the seasonal component value sequence in the cubic exponential smoothing statistical model using the least squares method; determine the cubic exponential smoothing mathematical model for each device described by the most recent data points of the optimal estimates of the ternary smoothing parameters, the smoothed value sequence, the trend component value sequence, and the seasonal component value sequence. The step of processing the training samples and the fitted samples according to the cubic exponential smoothing mathematical model for each device to determine the fitted calibration value for each device includes: Determine the initial value of the fitted calibration value for each device; predict the calibration sample for each device based on the training sample, the cubic exponential smoothing mathematical model, and the initial value of the fitted calibration value; determine the fitted calibration value for each device based on the calibration sample and the fitted sample using the least squares method.

2. The method according to claim 1, characterized in that, The process of processing the historical data of the most recent (n+1) days to generate device objects for the multiple devices includes: Preprocess the historical data of the most recent (n+1) days to generate a time series for each device with a time span of (p+1) days, where p is no greater than n; The identifier corresponding to each device and the time series for each device are encapsulated in a device object for the multiple devices.

3. The method according to claim 1, characterized in that, The step of dividing the time series into training samples and fitting samples includes: The sampling points of the first q days in the time series with a time span of (p+1) days are used as training samples, and the sampling points of the last (p-q+1) days in the time series with a time span of (p+1) days are used as fitting samples.

4. The method according to any one of claims 1 to 3, characterized in that, The dynamic threshold is used to issue warnings for the device at multiple time periods within at least one specified natural day.

5. A device for generating dynamic thresholds for equipment early warning based on a statistical model, characterized in that, include: The acquisition unit is used to acquire historical data of the most recent (n+1) days of operation of multiple devices, where n is a natural number greater than 1; The device object processing unit is used to process the historical data of the most recent (n+1) days and generate device objects for the multiple devices; The result object generation unit is used to process the device objects based on a statistical model and generate result objects for the multiple devices. A dynamic threshold generation unit is used to generate dynamic thresholds for device early warning for each of the multiple devices using result objects for the multiple devices. The result object includes fitted calibration values ​​for each device; the step of processing the device object based on a statistical model to generate a result object for the multiple devices includes: Extract time series data for each device from the device object, and divide the time series data into training samples and fitted samples; process the training samples based on the cubic exponential smoothing statistical model to determine the cubic exponential smoothing mathematical model for each device; process the training samples and the fitted samples according to the cubic exponential smoothing mathematical model for each device to determine the fitted adjustment value for each device; encapsulate the identifier corresponding to each device, the cubic exponential smoothing mathematical model for each device, and the fitted adjustment value in a result object for the multiple devices. The step of processing the training samples based on the cubic exponential smoothing statistical model to determine the cubic exponential smoothing mathematical model for each device includes: Determine the initial values ​​of the ternary smoothing parameters in the cubic exponential smoothing statistical model; determine the initial values ​​of the smoothed value sequence, trend component value sequence, and seasonal component value sequence in the cubic exponential smoothing statistical model; based on the training samples, the initial values ​​of the ternary smoothing parameters, and the initial values ​​of the smoothed value sequence, trend component value sequence, and seasonal component value sequence, determine the optimal estimates of the ternary smoothing parameters, the smoothed value sequence, the trend component value sequence, and the seasonal component value sequence in the cubic exponential smoothing statistical model using the least squares method; determine the cubic exponential smoothing mathematical model for each device described by the most recent data points of the optimal estimates of the ternary smoothing parameters, the smoothed value sequence, the trend component value sequence, and the seasonal component value sequence. The step of processing the training samples and the fitted samples according to the cubic exponential smoothing mathematical model for each device to determine the fitted calibration value for each device includes: Determine the initial value of the fitted calibration value for each device; predict the calibration sample for each device based on the training sample, the cubic exponential smoothing mathematical model, and the initial value of the fitted calibration value; determine the fitted calibration value for each device based on the calibration sample and the fitted sample using the least squares method.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A method for early warning of equipment, characterized in that, include: The method for generating dynamic thresholds for equipment early warning based on a statistical model according to any one of claims 1 to 4 generates dynamic thresholds; The dynamic threshold is used to issue warnings for multiple devices.

Images