Industrial internet of things time series adaptive anomaly detection method and system based on concept drift segmentation calibration

Abstract

The present application relates to an industrial internet of things time series adaptive anomaly detection method and system based on concept drift segmentation calibration, comprising: for the multivariate time series data stream collected by the industrial internet of things system, the basic anomaly detection layer outputs the original anomaly score corresponding to each time; the drift detection layer outputs the drift point sequence; according to the drift point sequence, the continuous data stream is cut, and the risk calculation and alarm control are completed in each relatively stable data segment. The present application realizes the integration of segmented calibration, stage anomaly proportion perception and dynamic alarm control online anomaly detection in the continuous flow environment through the processing flow of "drift driven segmentation - segment score calibration - contamination rate estimation - adaptive risk control - online alarm judgment", which can better adapt to the scene where concept drift and anomaly proportion change coexist in the non-stationary time series of industrial internet of things, and improve the stability, consistency and interpretability of online anomaly detection results.

Description

Technical Field

[0001] This invention belongs to the field of online anomaly detection technology for industrial IoT time series, and particularly relates to an adaptive anomaly detection method and system for industrial IoT time series based on concept drift segmentation calibration. Background Technology

[0002] In the context of the Industrial Internet of Things (IIoT), real-time, accurate, and stable anomaly monitoring of equipment operating status is crucial for ensuring the safe operation of industrial systems, improving operational efficiency, and enabling predictive maintenance. In applications such as smart manufacturing, process industries, and equipment maintenance, industrial sites typically utilize numerous sensors to continuously collect multi-source time-series data, including equipment operating status, process parameters, and environmental variables, forming a continuously arriving data stream. By performing online anomaly detection on this type of time-series data, it is possible to promptly identify equipment failure symptoms, deviations from operating conditions, and operational anomalies, thereby providing technical support for equipment health monitoring, fault early warning, and predictive maintenance. Therefore, online anomaly detection of time-series data in the IIoT has become a significant research area in the field of industrial intelligent monitoring.

[0003] Existing online anomaly detection methods for industrial time series typically begin by processing the input data using a basic detection model, outputting corresponding anomaly scores, and then issuing alarm results based on preset judgment rules. Regarding the mapping process from anomaly scores to alarm results, existing research mainly employs the following types of methods: Firstly, using fixed thresholds, empirical rules, or fixed quantiles to directly determine anomalies based on anomaly scores is a relatively simple method. However, it usually assumes that anomaly scores at different stages of operation can be compared according to the same standard, and that the anomaly ratio remains relatively stable over long-term operation. Therefore, when there are significant differences in anomaly rates at different stages, it is easy to encounter problems such as more false alarms in some stages and more false alarms in others. Secondly, by using sliding window, local statistics, or online update strategies, the judgment boundary is modified according to the distribution changes of recent abnormal scores. Although this type of method can track score fluctuations to a certain extent, its processing object is still mainly the original abnormal scores. The statistical distribution of the original abnormal scores is often inconsistent in different operating stages. Therefore, even if the judgment boundary changes over time, it is difficult to ensure that the alarm results have consistent judgment criteria in different stages. Third, methods such as score calibration, online risk control, or similar conformal prediction are used to transform abnormal scores into more statistically significant judgment criteria. These methods improve the statistical interpretability of detection results to some extent. However, existing solutions usually focus on processing the overall data stream or local time windows and fail to fully incorporate the stage switching characteristics caused by concept drift. Therefore, in non-stationary industrial time series scenarios, it is still difficult to simultaneously solve the problem of unstable alarms caused by the difficulty in directly comparing abnormal scores between different operating stages and the dynamic changes in the abnormal proportion.

[0004] Because industrial time series are affected by factors such as operating condition switching, load changes, equipment aging, control strategy adjustments and external environmental disturbances, their statistical distribution will continue to evolve. The proportion of abnormal samples in different operating stages is usually not consistent. Therefore, existing methods are still difficult to meet the application requirements of cross-stage judgment consistency, online alarm stability and result interpretability at the same time. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an adaptive anomaly detection method and system for industrial IoT time series based on concept drift segmentation calibration. This method utilizes concept drift identification results as the basis for data segment switching to segment the continuous time series data stream. Within each relatively stable data segment, the original anomaly scores output by the basic detection model are statistically calibrated and converted into calibration statistics with unified risk semantics. Simultaneously, combined with the segment contamination rate estimation, the anomaly ratio of the current stage is adaptively modeled, and the online alarm decision is dynamically adjusted accordingly. This improves the stability, consistency, and interpretability of online anomaly detection for industrial IoT time series under different operating stages.

[0006] The technical problem solved by this invention is achieved through the following technical solution: An adaptive anomaly detection method for industrial IoT time series based on concept drift segmentation calibration, the method comprising the following steps: S1. For the multivariate time series data stream collected by the industrial IoT system, the basic anomaly detection layer outputs the original anomaly score corresponding to each moment. S2, the drift detection layer outputs a drift point sequence; S3. Segment the continuous data stream according to the drift point sequence, and sequentially perform intra-segment score calibration, contamination rate estimation, adaptive risk control and online alarm determination within each relatively stable data segment obtained by the drift points, and finally output the calibration risk sequence and abnormal alarm result.

[0007] Moreover, when S2 detects a new drift point, it considers that the data stream has entered a new operating phase and switches to a new data segment. During the segment switch, the current segment reference cache and the phase statistics status are reset synchronously, thereby ensuring that subsequent calibration and alarm decisions are always based on a relatively consistent data environment within the current segment.

[0008] Furthermore, S3 uses the reference score set of the current segment to perform intra-segment calibration on the original abnormal scores and converts them into a statistically significant intra-segment risk quantity. When the intra-segment risk quantity is small, it indicates that the current sample is more inconsistent with the historical reference set of this segment and is more likely to correspond to an abnormal event. Subsequently, based on the historical risk sequence of the current segment, the abnormal contamination rate of the segment is estimated online to characterize the proportion of abnormal events in the current stage. After obtaining the estimated pollution rate within the segment, a dynamic alarm threshold is generated by combining the basic threshold and the current stage statistical status. At the same time, a threshold stabilization mechanism is introduced to constrain the threshold update process. Based on the relationship between the risk level within the segment of the current sample and the dynamic alarm threshold, the online alarm result is output, and the original anomaly score, calibration result and related statistics corresponding to the current sample are written back to the current segment cache for continued updating and calculation in subsequent time steps.

[0009] An industrial IoT time series adaptive anomaly detection system based on concept drift segmentation calibration is provided to implement the aforementioned detection method. The system includes: The basic anomaly detection layer is used to receive industrial time-series data streams and output raw anomaly scores. A drift detection layer is used to output a sequence of drift points; The segmented calibration alarm layer is used to segment the continuous data stream according to the drift point sequence. Within each relatively stable data segment obtained by the drift points, the segment score calibration, contamination rate estimation, adaptive risk control and online alarm determination are performed in sequence, and finally the calibration risk sequence and abnormal alarm results are output.

[0010] The positive effects that this invention can produce are: 1. This invention achieves integrated online anomaly detection in a continuous streaming environment by employing a processing flow of "drift-driven segmentation—intra-segment score calibration—contamination rate estimation—adaptive risk control—online alarm determination," enabling segmented calibration, stage anomaly ratio sensing, and dynamic alarm control. Compared to existing methods that directly rely on raw anomaly scores for unified determination, this invention better adapts to scenarios where concept drift and anomaly ratio changes coexist in non-stationary time series of the Industrial Internet of Things (IIoT), improving the stability, consistency, and interpretability of online anomaly detection results.

[0011] 2. This invention segments the continuous data stream based on the concept drift result, so that alarm decisions are made within relatively stable data segments, avoiding cross-stage mismatch problems caused by directly sharing unified judgment rules in different operation stages.

[0012] 3. This invention performs statistical calibration on the original anomaly scores within a segment and combines this with the estimation of the anomaly ratio within the segment to achieve dynamic alarm control, thereby improving the consistency of anomaly judgment and the interpretability of alarm results at different stages.

[0013] 4. This invention adopts a post-control method, which does not change the basic anomaly detection model structure, is compatible with a variety of detectors that can output anomaly scores, and improves the stability and engineering applicability of online anomaly detection in non-stationary environments. Attached Figure Description

[0014] Figure 1 This is a framework diagram of the anomaly detection system of the present invention; Figure 2 This is a schematic diagram of the drift drive segmentation and segment state switching of the present invention; Figure 3 This is a flowchart of the anomaly score calibration and contamination rate estimation within the present invention. Figure 4 This is a flowchart of the adaptive risk control and online alarm determination process of the present invention. Detailed Implementation

[0015] The present invention will be further described in detail below through specific embodiments. The following embodiments are merely descriptive and not limiting, and should not be used to limit the scope of protection of the present invention.

[0016] like Figure 1 As shown, an industrial IoT time series adaptive anomaly detection system based on concept drift segmented calibration is innovative in that it includes a basic anomaly detection layer, a drift detection layer, and a segmented calibration alarm layer. The basic anomaly detection layer receives industrial time series data streams and outputs raw anomaly scores; the drift detection layer outputs a drift point sequence; and the segmented calibration alarm layer completes score calibration, contamination rate estimation, adaptive risk control, and online alarm determination within relatively stable data segments. The basic anomaly detection layer is preferably a reconstructive anomaly detection model or a predictive anomaly detection model, but is not limited to these; the drift detection layer is not limited to a specific algorithm, as long as it can output the stage switching time in the data stream, it can be used as the segmented input of this invention.

[0017] An innovative method for adaptive anomaly detection in industrial IoT time series based on concept drift segmentation calibration is described, with the following steps: S1. The multivariate time series data stream collected by the industrial IoT system is represented as follows: (1) in: Represents a multivariate time series data stream. Indicates time Sensor observation vectors, The time duration is represented by F, and the sensor observation vector is represented by F. Dimensions.

[0018] The detection system inputs the time-series data stream into the basic anomaly detection layer M to obtain the raw anomaly scores for each time step: (2) in: Indicates time The original outlier score, The larger the value, the greater the deviation of the current sample from the normal pattern; S2, such as Figure 1 and Figure 2 As shown, the drift detection layer outputs a sequence of drift points: (3) in: Indicates the time of the detected drift point. Indicates the number of drift points; The continuous data stream is divided into multiple relatively stable data segments based on the drift point sequence: (4) When the time step reaches a new drift point, the system considers the data stream to have entered a new operational phase and switches to a new data segment; at the same time, the system synchronously resets the current segment reference cache. The corresponding pollution rate estimate and related historical statistics ensure that subsequent calibration and alarm control are always based on a relatively stable data environment within the current segment.

[0019] S3, such as Figure 3 As shown, in the current data segment Internally, the system does not directly rely on the original anomaly score. Instead of making alarm decisions, it utilizes the current segment's historical reference set. Intra-segment calibration was performed to obtain statistically significant intra-segment risk quantities. Preferably, the reference set for the current segment is set as follows: (5) Then time intra-segment risk Represented as: (6) in: Indicates an indicator function; Indicates the number of samples in the current segment reference set; when When the value is smaller, it indicates that the current sample is more inconsistent with the historical samples in this segment and is more likely to correspond to an abnormal event. Through this step, the original anomaly scores that cannot be directly compared under different operating stages are mapped to a unified risk representation space.

[0020] S4, such as Figure 3 As shown, the risk level within the segment is obtained. Then, based on the historical risk sequence of the current segment, the abnormal pollution rate of this segment is estimated online to characterize the proportion of abnormal events in the current operation phase. Preferably, a basic reference threshold is set as follows: The indicator variable for low-risk events is defined as follows: (7) in: This indicates that the current sample has entered the set of low-risk events; This indicates that the current sample has not entered the set. Furthermore, the estimated pollution rate for the current segment... The update is performed using a recursive method: (8) in: The updated coefficients are used to control the impact of new observations on the current estimation results. Through this method, the system can model the proportion of anomalies in a given segment online based on the frequency of low-risk events within the current segment. It should be noted that the pollution rate estimation method is not limited to the above recursive form; it can also be achieved using moving averages, window statistics, or other methods that can characterize the changing trend of the proportion of anomalies in a given segment online.

[0021] S5, such as Figure 4 As shown, after obtaining the estimated pollution rate of the current segment, the system constructs a pollution rate driving term based on the base threshold and the reference pollution rate. Let the reference pollution rate be... The pollution rate-driven threshold is then expressed as: (9) in: This is the pollution rate adjustment coefficient; when Higher than the reference pollution rate When this occurs, it indicates that the current stage has a relatively high anomaly rate, and the system can adjust the alarm threshold within the segment accordingly to enhance anomaly detection capabilities; when Below the reference pollution rate If the alarm threshold is too low at this time, it indicates that the current abnormality rate is low. The system can then tighten the alarm threshold accordingly to suppress false alarms.

[0022] S6, such as Figure 4As shown, to avoid excessively rapid threshold changes in the early stages of a new data segment due to a small sample size and unstable statistics, the system further introduces a threshold stability term to constrain the threshold update process within the segment; let the current segment end time be... The number of samples is Low-risk event sequence The empirical variance is Then the threshold stability term Represented as: (10) in: The confidence parameter is used; when the number of samples in the current segment is small, the stability term is large, which can provide stronger constraints on the threshold change; as the number of samples in the segment gradually increases, the statistical results tend to stabilize, and the stability term gradually decreases, so that the threshold can more fully reflect the actual changes in the current stage.

[0023] S7, such as Figure 4 As shown, the system combines the pollution rate driving term with the threshold stabilization term to obtain the dynamic alarm threshold for the current segment. The dynamic threshold is expressed as: (11) in: This indicates that the threshold is projected onto a preset interval. Inside; This is the adjustment coefficient for the stability term. Through this step, the system can comprehensively consider the changes in the proportion of anomalies in the current stage and the statistical stability within the segment, and dynamically generate alarm thresholds applicable to the current data segment.

[0024] S8, such as Figure 4 As shown, the risk level within the segment is obtained. and dynamic threshold Then, the system outputs online anomaly alarm results based on the following judgment rules: (12) in: 1 indicates that the current sample has triggered an anomaly alarm. This indicates that no alarm will be triggered for the current sample. After completing the judgment for the current sample, the system writes the original anomaly score, intra-segment risk, and related statistics corresponding to the sample back to the current segment cache. This is used for risk calculation, contamination rate updates, and threshold adjustments for subsequent samples.

[0025] S9. The system continues to execute S1~S8 step by step according to time steps; when a new drift point is detected, the system re-enters the following sequence. Figure 2 The process of switching to a new segment and resetting the status is shown, and repeated. Figure 3 and Figure 4The process shown includes segment score calibration, contamination rate estimation, dynamic threshold generation, and online alarm judgment, thereby achieving integrated online anomaly detection in a continuous streaming environment, which includes drift-driven segmentation, unified risk representation within segments, stage anomaly ratio perception, and dynamic alarm control.

[0026] Although embodiments of the invention have been disclosed for illustrative purposes, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the scope of the invention is not limited to the contents disclosed in the embodiments.

Claims

1. An adaptive anomaly detection method for industrial IoT time series based on concept drift segmentation calibration, characterized in that: The steps of the method are as follows: S1. For the multivariate time series data stream collected by the industrial IoT system, the basic anomaly detection layer outputs the original anomaly score corresponding to each moment. S2, the drift detection layer outputs a drift point sequence; S3. Segment the continuous data stream according to the drift point sequence, and sequentially perform intra-segment score calibration, contamination rate estimation, adaptive risk control and online alarm determination within each relatively stable data segment obtained by the drift points, and finally output the calibration risk sequence and abnormal alarm result.

2. The adaptive anomaly detection method for industrial IoT time series based on concept drift segmentation calibration according to claim 1, characterized in that: When S2 detects a new drift point, it considers that the data stream has entered a new operating phase and switches to a new data segment. During the segment switch, the current segment reference cache and the phase statistics status are reset synchronously to ensure that subsequent calibration and alarm decisions are always based on a relatively consistent data environment within the current segment.

3. The adaptive anomaly detection method for industrial IoT time series based on concept drift segmentation calibration according to claim 2, characterized in that: S3 uses the reference score set of the current segment to perform intra-segment calibration on the original abnormal scores and converts them into a statistically significant intra-segment risk quantity. When the intra-segment risk quantity is small, it indicates that the current sample is more inconsistent with the historical reference set of this segment and is more likely to correspond to an abnormal event. Subsequently, the abnormal pollution rate of the segment is estimated online based on the historical risk sequence of the current segment to characterize the proportion of abnormal events in the current stage; After obtaining the estimated pollution rate within the segment, a dynamic alarm threshold is generated by combining the basic threshold and the current stage statistical status. At the same time, a threshold stabilization mechanism is introduced to constrain the threshold update process. Based on the relationship between the risk level within the segment of the current sample and the dynamic alarm threshold, the online alarm result is output, and the original anomaly score, calibration result and related statistics corresponding to the current sample are written back to the current segment cache for continued updating and calculation in subsequent time steps.

4. An industrial IoT time series adaptive anomaly detection system based on concept drift segmentation calibration, characterized in that: The system, used to perform the detection method as described in any one of claims 1 to 3, comprises: The basic anomaly detection layer is used to receive industrial time-series data streams and output raw anomaly scores. A drift detection layer is used to output a sequence of drift points; The segmented calibration alarm layer is used to segment the continuous data stream according to the drift point sequence. Within each relatively stable data segment obtained by the drift points, the segment score calibration, contamination rate estimation, adaptive risk control and online alarm determination are performed in sequence, and finally the calibration risk sequence and abnormal alarm results are output.

Images