Methods, equipment, and media for high-frequency time-series data anomaly detection based on integrated strategies

Abstract

This application discloses a method, device, and medium for high-frequency time-series data anomaly detection based on an ensemble strategy, relating to the field of industrial internet technology. The method includes: standardizing high-frequency time-series data to obtain structured data; segmenting the structured data using a sliding window to determine high-frequency time-series data segments; performing data quality checks on the high-frequency time-series data segments to obtain high-frequency time-series data segments to be inspected; determining a first anomaly score sequence based on the high-frequency time-series data segments to be inspected through unsupervised anomaly identification using anomaly sensitivity; determining a second anomaly score sequence based on the high-frequency time-series data segments to be inspected through data volatility analysis; and obtaining anomaly monitoring data for the high-frequency time-series data through meta-classification analysis using ensemble learning based on the first and second anomaly score sequences. This application solves the technical problem that high-frequency time-series data anomaly detection cannot adapt to complex industrial environments through the above method.

Description

Technical Field

[0001] This application relates to the field of industrial internet technology, and in particular to a method, device and medium for detecting anomalies in high-frequency time-series data based on an integration strategy. Background Technology

[0002] In today's industrial environment, the acquisition process of high-frequency sensor signals is highly susceptible to various external and internal interferences. These interference sources include, but are not limited to, external shocks, errors or instabilities that may occur during data transmission, complex and ever-changing field operating environments, and electromagnetic interference, which can lead to frequent data loss and various anomalies in the acquired high-frequency time-series data.

[0003] In existing technologies, high-frequency time-series data anomaly detection methods are often designed for specific operating conditions or static environments, while the operating conditions of equipment in IoT industrial monitoring frequently change. Due to the extremely complex data environment for anomaly detection in high-frequency time-series data under the Industrial Internet, existing algorithms have shortcomings. First, the performance of anomaly detection algorithms drops sharply under different loads, speeds, and other operating conditions, leading to false alarms or missed alarms. Second, the sheer volume of data generated by high-frequency sensors severely limits the computational efficiency and effectiveness of directly applying unsupervised machine learning algorithms for anomaly detection. Third, the waveform of monitored data can change due to actual equipment failures, making it difficult for detection algorithms to distinguish between truly normal data and legitimate data pattern changes indicating equipment failure. Fourth, compared to the vast majority of normal data segments, the number of data segments containing genuine anomalies is extremely small, making existing unsupervised algorithms unable to meet the requirements of complex industrial applications. Summary of the Invention

[0004] This application provides a method, device, and medium for detecting anomalies in high-frequency time-series data based on an integrated strategy, which solves the technical problem that existing high-frequency time-series data anomaly detection methods cannot adapt to complex industrial environments.

[0005] In a first aspect, embodiments of this application provide a method for detecting anomalies in high-frequency time-series data based on an ensemble strategy. The method comprises: acquiring high-frequency time-series data and performing data standardization processing on the high-frequency time-series data to obtain structured data; performing sliding window segmentation on the structured data to determine high-frequency time-series segment data; performing data quality checks on the high-frequency time-series segment data to obtain high-frequency time-series segment data to be inspected; determining a first anomaly score sequence based on the high-frequency time-series segment data to be inspected through unsupervised anomaly identification using anomaly sensitivity; determining a second anomaly score sequence based on the high-frequency time-series segment data to be inspected through data volatility analysis; and obtaining anomaly monitoring data for the high-frequency time-series data through meta-classification analysis using ensemble learning based on the first and second anomaly score sequences.

[0006] In one implementation of this application, high-frequency time-series data is subjected to data standardization processing to obtain structured data. Specifically, this includes: parsing the high-frequency time-series data into a time-series array to obtain a numerical time-series array; and converting the data format of the numerical time-series array to obtain structured data.

[0007] In one implementation of this application, structured data is segmented by a sliding window to determine high-frequency time-series data segments. Specifically, this includes: obtaining sliding window parameters, and performing non-overlapping window processing on the data stream of high-frequency time-series data based on the sliding window parameters to obtain a non-overlapping sliding window; and segmenting the structured data by a sliding window according to the non-overlapping sliding window to determine high-frequency time-series data segments.

[0008] In one implementation of this application, data quality checks are performed on high-frequency time-series segment data to obtain high-frequency time-series segment data to be checked. Specifically, this includes: detecting the missing data rate of the high-frequency time-series segment data to obtain the missing data segment rate; and determining the high-frequency time-series segment data to be checked based on the missing data segment rate and a missing rate threshold.

[0009] In one implementation of this application, based on the high-frequency time-series segment data to be inspected, a first anomaly score sequence is determined through unsupervised anomaly identification with anomaly sensitivity. Specifically, this includes: extracting key features in the time domain from the high-frequency time-series segment data to be inspected to determine a key feature sequence; performing unsupervised anomaly detection on the key feature sequence using multiple parallel algorithms to obtain anomaly feature points; calculating the anomaly score corresponding to the anomaly feature point and normalizing the anomaly score to determine the first anomaly score sequence.

[0010] In one implementation of this application, a second abnormal score sequence is determined based on the high-frequency time series data to be inspected through data volatility analysis. Specifically, this includes: setting volatility monitoring parameters; wherein the volatility monitoring parameters include: detection window length, normal range limit coefficient, and volatility detection evaluation index; comparing the volatility of adjacent time windows based on the detection window length and volatility detection evaluation index, and selecting time points where the volatility of adjacent time windows is greater than the normal range limit coefficient to determine volatility offset points; and calculating the abnormal score corresponding to the volatility offset point to determine the second abnormal score sequence.

[0011] In one implementation of this application, after determining the second abnormal score sequence based on the high-frequency time series data to be inspected through data volatility analysis, the method further includes: extracting contextual meta-features from the high-frequency time series data to be inspected to obtain meta-feature vectors.

[0012] In one implementation of this application, anomaly monitoring data for high-frequency time-series data is obtained through meta-classification analysis of ensemble learning based on a first anomaly score sequence and a second anomaly score sequence. Specifically, this includes: configuring a base learner to be trained, setting the first and second anomaly score sequences as input features, and training the base learner until model convergence to obtain the base learner; determining the base learner prediction vector through probabilistic prediction analysis based on the base learner; concatenating the first anomaly score sequence, the second anomaly score sequence, the base learner prediction vector, and the meta-feature vector to obtain the meta-learner input vector; acquiring the meta-learner to be trained, using the meta-learner input vector as the training vector, and training the meta-learner until model convergence to determine the high-frequency time-series data anomaly detection model; and inputting the high-frequency time-series data into the high-frequency time-series data anomaly detection model to obtain the high-frequency time-series data anomaly monitoring data.

[0013] Secondly, embodiments of this application also provide a high-frequency time-series data anomaly detection device based on an integrated strategy. The device comprises: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, enable the at least one processor to: acquire high-frequency time-series data and perform data standardization processing on the high-frequency time-series data to obtain structured data; perform sliding window segmentation on the structured data to determine high-frequency time-series segment data; perform data quality checks on the high-frequency time-series segment data to obtain high-frequency time-series segment data to be detected; determine a first anomaly score sequence based on the high-frequency time-series segment data to be detected through unsupervised anomaly identification using anomaly sensitivity; determine a second anomaly score sequence based on the high-frequency time-series segment data to be detected through data volatility analysis; and obtain anomaly monitoring data for the high-frequency time-series data through meta-classification analysis using ensemble learning based on the first and second anomaly score sequences.

[0014] Thirdly, embodiments of this application also provide a non-volatile computer storage medium for high-frequency time-series data anomaly detection based on an integration strategy, storing computer-executable instructions. The computer-executable instructions are characterized by: acquiring high-frequency time-series data and performing data standardization processing on the high-frequency time-series data to obtain structured data; performing sliding window segmentation on the structured data to determine high-frequency time-series segment data; performing data quality checks on the high-frequency time-series segment data to obtain high-frequency time-series segment data to be inspected; determining a first anomaly score sequence based on the high-frequency time-series segment data to be inspected through unsupervised anomaly identification using anomaly sensitivity; determining a second anomaly score sequence based on the high-frequency time-series segment data to be inspected through data volatility analysis; and obtaining anomaly monitoring data for high-frequency time-series data through meta-classification analysis using ensemble learning based on the first and second anomaly score sequences.

[0015] This application provides a method, device, and medium for detecting anomalies in high-frequency time-series data based on an ensemble strategy. By extracting key time-domain features, identifying unsupervised anomalies, detecting rapid data volatility, and constructing a meta-classifier based on an ensemble learning strategy, it solves the technical problem that existing high-frequency time-series data anomaly detection methods cannot adapt to complex industrial environments. Based on the combined prediction of multiple models, it achieves feature interpretation of anomaly characteristics in high-frequency time-series data while retaining awareness of the original data context. This improves the modeling ability and robustness in practical applications, overcomes the data limitation of imbalanced anomaly feature data, and optimizes the precision and recall of high-frequency time-series data anomaly detection. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 A flowchart of a high-frequency time-series data anomaly detection method based on an integration strategy is provided in an embodiment of this application;

[0018] Figure 2 A data segment waveform of a centrifugal pump unit at rated speed is provided as an embodiment of this application.

[0019] Figure 3 This application provides a data segment waveform of a centrifugal pump unit under a loosening fault state at rated speed, as shown in an embodiment of the present application.

[0020] Figure 4 A flowchart of online data anomaly detection and offline update for a high-frequency time series model provided in this application embodiment;

[0021] Figure 5This application provides a diagram illustrating abnormal waveform identification of a bearing outer ring fault state at rated speed, as shown in the embodiment of the present application.

[0022] Figure 6 This application provides a waveform anomaly identification diagram for a data segment of the bearing outer ring health status at rated speed, as provided in an embodiment of the present application.

[0023] Figure 7 This is a schematic diagram of the internal structure of a high-frequency time-series data anomaly detection device based on an integration strategy, provided as an embodiment of this application. Detailed Implementation

[0024] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] This application provides a method, device, and medium for detecting anomalies in high-frequency time-series data based on an ensemble strategy. By extracting key time-domain features, identifying unsupervised anomalies, detecting rapid data volatility, and constructing a meta-classifier based on an ensemble learning strategy, it solves the technical problem that existing high-frequency time-series data anomaly detection methods cannot adapt to complex industrial environments. Based on the combined prediction of multiple models, it achieves feature interpretation of anomaly characteristics in high-frequency time-series data while retaining awareness of the original data context. This improves the modeling ability and robustness in practical applications, overcomes the data limitation of imbalanced anomaly feature data, and optimizes the precision and recall of high-frequency time-series data anomaly detection.

[0026] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0027] Figure 1 A flowchart illustrating a high-frequency time-series data anomaly detection method based on an integration strategy, provided in an embodiment of this application. Figure 1 As shown in the figure, the high-frequency time-series data anomaly detection method based on an integration strategy provided in this application embodiment specifically includes the following steps:

[0028] Step 101: Obtain high-frequency time series data and perform data standardization processing on the high-frequency time series data to obtain structured data.

[0029] Specifically, high-frequency time-series data is standardized to obtain structured data, including: parsing the high-frequency time-series data into a time-series array to obtain a numerical time-series array; and converting the data format of the numerical time-series array to obtain structured data.

[0030] In one embodiment, high-frequency vibration sensors are deployed at key rotating component locations in industrial equipment (e.g., centrifugal pump units). The sensors are precisely connected to corresponding data acquisition devices and seamlessly integrated into an industrial Internet of Things (IoT) acquisition environment to continuously and in real-time acquire and transmit high-frequency monitoring data on the operating status of the key rotating components.

[0031] After data acquisition is complete, the raw data undergoes a critical data format transformation process. The purpose of this step is to convert the raw data stream into a standardized and structured format, explicitly optimized for subsequent anomaly analysis tasks. This transformation may involve multiple sub-steps, such as parsing the raw binary stream into a numerical time-series array, ensuring accurate temporal alignment by adding precise timestamps, processing data encoding, and organizing the data in a way that facilitates efficient ingestion and processing by subsequent machine learning algorithms. This ensures the data's compatibility and readiness before entering the complex analysis phase.

[0032] Step 102: Perform sliding window segmentation on the structured data to determine high-frequency time series data segments.

[0033] For example, this application uses a sliding window to segment structured data to identify high-frequency time-series data segments. The non-overlapping design ensures that each data point is processed only once during the segmentation process, thereby optimizing the use of computing resources.

[0034] Specifically, the structured data is segmented by a sliding window to determine high-frequency time-series data segments, including: obtaining sliding window parameters, and performing non-overlapping window processing on the data stream of high-frequency time-series data based on the sliding window parameters to obtain a non-overlapping sliding window; and segmenting the structured data by a sliding window according to the non-overlapping sliding window to determine high-frequency time-series data segments.

[0035] Figure 2 This is a waveform of a data segment showing the healthy state of a centrifugal pump unit at its rated speed, provided as an embodiment of this application.

[0036] Figure 3 This is a data segment waveform of a centrifugal pump unit in a loose-foot fault state at rated speed, provided as an embodiment of this application.

[0037] In one embodiment, within a set monitoring period (e.g., 3-5 minutes within a single monitoring cycle), a high-frequency data stream (length is...) is... The system will be segmented systematically. This is achieved using a non-overlapping sliding window method, which sets a specific and predefined sliding window size.

[0038] Each generated data segment or fragment is designed to have a specific time length, typically approximately 12 to 24 seconds. The sequence of data segments can be represented as follows: ,in This represents the number of data segments. At a high sampling frequency of 20 kHz, a 12-second data segment will contain up to 240,000 independent data points. This segmentation method is crucial for effectively managing large amounts of data, breaking down continuous data streams into computationally more manageable blocks, thus significantly reducing the computational burden for subsequent analysis steps.

[0039] By striking a balance between data granularity and computational load, inefficiency caused by excessive data volume is avoided, thus improving the processing speed of high-frequency time-series data streams.

[0040] It is important to note that Figure 2 and Figure 3 The vertical axis represents the high-frequency vibration sensor signal. The vibration measurement data of this signal is collected by a Wilconxon 786B-10 100mV / g single-axis accelerometer with a sampling frequency of 20KHz. The output is a normalized signal with the unit "1", which indicates that when the signal magnitude is "1", the acceleration of the vibration is 5g (g is the gravitational acceleration under Earth's environment).

[0041] Step 103: Perform a data quality check on the high-frequency time series segment data to obtain the high-frequency time series segment data to be checked.

[0042] For example, in order to avoid poor data quality leading to a decrease in model training performance and data analysis accuracy, this application implements preventative high-frequency time series data quality control by performing data quality checks on high-frequency time series data.

[0043] Specifically, data quality checks are performed on high-frequency time series data to obtain the high-frequency time series data to be checked, including: detecting the missing data rate of the high-frequency time series data to obtain the missing data rate; and determining the high-frequency time series data to be checked based on the missing data rate and a missing rate threshold.

[0044] For example, if the numerical missing rate of a single data segment exceeds a preset threshold If the specific segment fails to pass the test, it will be discarded and no further anomaly detection will be performed.

[0045] If all data fragments have a high missing rate (i.e., exceeding...), The proportion of discarded segments exceeds a preset threshold. If this occurs, the subsequent anomaly detection process for the entire monitoring period will be suspended, and an anomaly alarm indicating excessively high data missing rate will be issued.

[0046] This application ensures the data input to the anomaly detection system has sufficient integrity and reliability through data quality checks. By actively identifying and excluding low-quality data, this invention avoids the negative impact of unreliable data on anomaly detection results, thereby improving the robustness and reliability of the entire anomaly detection system. The final number obtained is... The high-frequency time-series data (sequences) to be examined ,in This refers to the number of data segments that passed the missing data detection. .

[0047] Step 104: Based on the high-frequency time-series segment data to be inspected, determine the first anomaly score sequence through unsupervised anomaly identification based on anomaly sensitivity.

[0048] For example, different principles (density, distance, statistics, trees, etc.) result in varying sensitivities to different types of anomalies or data structures. This application leverages the inherent diversity of unsupervised anomaly identification based on anomaly sensitivity to provide complementary anomaly signals, achieving multi-dimensional sensitivity coverage of high-frequency time-series data and improving the accuracy of anomaly data inspection.

[0049] Specifically, based on the high-frequency time-series data to be inspected, the first anomaly score sequence is determined through unsupervised anomaly identification based on anomaly sensitivity. This includes: extracting key features in the time domain from the high-frequency time-series data to be inspected to determine the key feature sequence; performing unsupervised anomaly detection on the key feature sequence using multiple parallel algorithms to obtain anomaly feature points; calculating the anomaly score corresponding to the anomaly feature point and normalizing the anomaly score to determine the first anomaly score sequence.

[0050] Figure 4 This is a flowchart illustrating the online data anomaly detection and offline update process for a high-frequency time series model, as provided in an embodiment of this application.

[0051] For each data segment, key temporal features are first calculated. This process sequences the data segments. Abstracted into a set of feature sequences that significantly reduce the amount of data. This feature abstraction not only significantly improves the computational efficiency of subsequent analysis, but also enhances the effectiveness of anomaly detection by highlighting potential patterns related to anomalies in the data. For example, kurtosis is an effective feature for capturing impulsive or sudden changes in data, playing a crucial role in detecting additional outliers or transient changes. Other extractable time-domain features include mean, standard deviation, variance (reflecting the horizontal offset and fluctuation range of the data), skewness (describing the asymmetry of the data distribution), root mean square (RMS, reflecting signal energy), peak-to-peak value (reflecting signal amplitude variation), waveform factor, and impulse factor (other shape descriptors). A diverse set of unsupervised anomaly detection algorithms can be applied in parallel to these feature sequences. In order to identify abnormal feature points.

[0052] Algorithms for unsupervised anomaly detection with high sensitivity include, but are not limited to: Isolation Forest: an efficient tree-based ensemble method that effectively isolates outliers through random partitioning, particularly suitable for high-dimensional datasets. Normalized K-Nearest Neighbors (KNN): a distance-based method that identifies anomalies by calculating the distances of data points to their K nearest neighbors; sensitive to density variations. Data normalization is crucial to ensure the accuracy of distance metrics. Z-score: a statistical method that identifies outliers by calculating the standard deviation multiple of a data point from the mean, typically assuming the data follows a normal distribution. Interquartile Range (IQR): a statistical method that defines outliers as points falling outside 1.5 times the IQR of the first or third quartile. Density-Based Spatial Clustering of Application with Noise (DBSCAN): a density-based clustering algorithm that identifies points not belonging to any dense cluster as noise points; effective for non-spherical clusters and data of varying densities.

[0053] Each algorithm generates anomaly scores for each feature point in the feature sequence. These scores are normalized and then collectively form a pre-score sequence.

[0054] For example, the Anomaly detection algorithm For feature sequences abnormal score sequence .vector It is no longer a final judgment, but a quantitative description of time-series data segments across multiple anomaly dimensions, a vector. These will be used as input features for the learner.

[0055] Step 105: Based on the high-frequency time series data to be inspected, determine the second abnormal score sequence through data volatility analysis.

[0056] For example, existing technologies typically employ static feature analysis, while high-frequency time series require the detection of transient change outliers and horizontal offset outliers, which cannot be easily obtained directly using existing technologies. This application achieves an effective supplement to anomaly feature detection by using data volatility analysis in parallel with unsupervised anomaly identification based on anomaly sensitivity, ensuring that a wider range of anomaly types can be captured, and providing more direct and faster signal support for some types of data deviations, further improving the speed of anomaly identification.

[0057] Specifically, based on the high-frequency time series data to be inspected, a second abnormal score sequence is determined through data volatility analysis. This includes: setting volatility monitoring parameters, which include: detection window length, normal range limit coefficient, and volatility detection evaluation index; comparing the volatility of adjacent time windows based on the detection window length and volatility detection evaluation index, and selecting time points where the volatility of adjacent time windows is greater than the normal range limit coefficient to determine the volatility offset point; and calculating the abnormal score corresponding to the volatility offset point to determine the second abnormal score sequence.

[0058] In one embodiment, a detection window is set. Length, normal range limit coefficient Evaluation indicators for detection window fluctuation (Including but not limited to decimal intervals, standard deviations, interquartile ranges, etc.)

[0059] In data fragments Based on the testing and evaluation indicators Compare the volatility of two adjacent time windows. If the difference between the volatility measurements of the two time windows exceeds [a certain threshold], [the following action is taken]. The intermediate time point is then marked as the volatility offset point, and the anomaly score for each data segment is finally output.

[0060] Furthermore, after determining the second abnormal score sequence based on the high-frequency time series data to be inspected through data volatility analysis, the method also includes: extracting contextual meta-features from the high-frequency time series data to be inspected to obtain meta-feature vectors.

[0061] In one embodiment, for a sequence of data fragments Extract each segment Each descriptive meta-feature (mean, standard deviation, skewness, margin factor, etc.) is combined into a meta-feature vector. The aim is to capture the inherent characteristics of raw data and provide meta-learners with additional decision-making basis beyond outlier scores.

[0062] Step 106: Based on the first and second anomaly score sequences, meta-classification analysis through ensemble learning is used to obtain anomaly monitoring data for high-frequency time series data.

[0063] For example, existing anomaly detection algorithms, such as the K-Nearest Neighbors (KNN) algorithm, directly perform anomaly detection on high-frequency time-series data. These algorithms calculate the distance between each detected point and all other points, resulting in extremely high computational complexity and slow execution time for massive amounts of high-frequency time-series data, leading to inefficiency. When equipment malfunctions, high-frequency time-series data exhibits periodic spikes due to the fault. These spikes are waveform changes caused by equipment malfunction, not data anomalies. However, methods such as interquartile range (IMR) and Z-score tend to treat "sparse" data points as anomalies, misidentifying the aforementioned spikes and resulting in low anomaly detection accuracy. This application integrates learning-based meta-classification analysis, achieving both high efficiency and accuracy in high-frequency time-series data anomaly detection.

[0064] Specifically, based on the first and second anomaly score sequences, meta-classification analysis through ensemble learning is used to obtain anomaly monitoring data for high-frequency time-series data. This includes: configuring a base learner to be trained, using the first and second anomaly score sequences as input features, and training the base learner until model convergence to obtain the base learner; determining the base learner prediction vector through probabilistic prediction analysis based on the base learner; concatenating the first anomaly score sequence, the second anomaly score sequence, the base learner prediction vector, and the meta-feature vector to obtain the meta-learner input vector; acquiring the meta-learner to be trained, using the meta-learner input vector as its training vector, and training the meta-learner until model convergence to determine the high-frequency time-series data anomaly detection model; and inputting the high-frequency time-series data into the high-frequency time-series data anomaly detection model to obtain the high-frequency time-series data anomaly monitoring data.

[0065] Figure 5 This is a waveform anomaly identification diagram of a bearing outer ring fault state at rated speed, provided as an embodiment of this application.

[0066] Figure 6 This is a waveform display diagram illustrating the identification of abnormalities in the data segment of the outer ring health status of a bearing at rated speed, provided in an embodiment of this application.

[0067] In one embodiment, a base learner is first trained and predicted. This step is the first layer of the stacking ensemble, and its task is to learn how to interpret the sequence of anomaly score features generated by the preceding algorithm.

[0068] Next, a base learner is constructed, utilizing unsupervised anomaly recognition based on anomaly sensitivity to a total of Select the abnormal score sequence generated by the algorithm. A diverse range of base learners This includes, but is not limited to, decision trees, support vector machines, and Naive Bayes.

[0069] When performing cross-validation or making predictions on new data, each trained base learner Will the input Generate a prediction probability The prediction probabilities of all base learners together constitute the base learner prediction vector. , This represents the professional opinions given by multiple base learners after analyzing the previous abnormal scores.

[0070] Then, the final input vector of the meta-learner is constructed. The purpose is information fusion, integrating information from different levels and dimensions to build the most comprehensive input view for the meta-learner. The base learner's predicted vector is then used... Contextual metafeature vectors and abnormal scores Perform splicing, for each data segment The final input vector .

[0071] This vector simultaneously provides the meta-learner with the base learner's interpretation and judgment of outlier scores, the inherent physical characteristics of the initial data window, and the raw outlier scores from the preceding algorithm. This allows the meta-learner to discover patterns that the base learner might overlook, or to correct errors when the base learners collectively make mistakes.

[0072] Finally, meta-learner training and final decision-making are performed, selecting a non-linear model (such as XGBoost or LightGBM) as the meta-learner. Using a set of training data with real labels. Train them.

[0073] Meta-learner Learn a complex mapping function from a high-dimensional fusion space to the final label. Due to its input The richness of the rules allows meta-learners to learn extremely complex rules.

[0074] Table 1 shows a comparison of the performance indicators of the technical solution adopted in this application with those of existing technologies for detecting anomalies in high-frequency time-series data.

[0075] Table 1 Performance metrics of each model on the test set

[0076]

[0077] Existing technologies have low anomaly recall rates for high-frequency time-series data anomaly detection, while the Z-score algorithm, which has extremely high anomaly recall rates, cannot meet the anomaly precision requirements of complex industrial environments. The technical solution adopted in this application, through experiments, has achieved the highest relative F1 score among existing technologies, and can meet the requirements of most complex industrial environments.

[0078] This application evolves the standard Stacking model into a deeply integrated system that combines not only the model's predictions but also the interpretation of "features of features," while always retaining awareness of the original data context, thus possessing stronger modeling capabilities and robustness.

[0079] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide a high-frequency time-series data anomaly detection device based on an integration strategy, the structure of which is as follows: Figure 7 As shown.

[0080] Figure 7 This is a schematic diagram of the internal structure of a high-frequency time-series data anomaly detection device based on an integrated strategy, provided as an embodiment of this application. Figure 7 As shown, the device includes:

[0081] At least one processor 701;

[0082] And a memory 702 that is communicatively connected to at least one processor;

[0083] The memory 702 stores instructions executable by at least one processor, which are executed by at least one processor 701 to enable at least one processor 701 to:

[0084] High-frequency time-series data is acquired and standardized to obtain structured data. Sliding window segmentation is then applied to the structured data to determine high-frequency time-series segments. Data quality checks are performed on these segments to obtain the high-frequency time-series segments to be inspected. Based on these segments, an unsupervised anomaly identification based on anomaly sensitivity is used to determine a first anomaly score sequence. A second anomaly score sequence is determined based on data volatility analysis. Finally, based on the first and second anomaly score sequences, meta-classification analysis using ensemble learning is employed to obtain anomaly monitoring data for the high-frequency time-series data.

[0085] Some embodiments of this application provide corresponding to Figure 1 A non-volatile computer storage medium for high-frequency time-series data anomaly detection based on an integrated strategy, storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0086] High-frequency time-series data is acquired and standardized to obtain structured data. Sliding window segmentation is then applied to the structured data to determine high-frequency time-series segments. Data quality checks are performed on these segments to obtain the high-frequency time-series segments to be inspected. Based on these segments, an unsupervised anomaly identification based on anomaly sensitivity is used to determine a first anomaly score sequence. A second anomaly score sequence is determined based on data volatility analysis. Finally, based on the first and second anomaly score sequences, meta-classification analysis using ensemble learning is employed to obtain anomaly monitoring data for the high-frequency time-series data.

Claims

1. A method for detecting anomalies in high-frequency time-series data based on an integrated strategy, characterized in that, The method includes: High-frequency time-series data is acquired, and the high-frequency time-series data is subjected to data standardization processing to obtain structured data; The structured data is segmented using a sliding window to determine high-frequency time-series data segments; The high-frequency time series data is subjected to a data quality check to obtain the high-frequency time series data to be checked; Based on the high-frequency time-series data to be inspected, a first anomaly score sequence is determined through unsupervised anomaly identification based on anomaly sensitivity. Based on the high-frequency time series data to be detected, a second anomaly score sequence is determined through data volatility analysis; Based on the first anomaly score sequence and the second anomaly score sequence, meta-classification analysis through ensemble learning is used to obtain anomaly monitoring data of the high-frequency time series data; Based on the high-frequency time-series data to be inspected, an unsupervised anomaly identification based on anomaly sensitivity is used to determine the first anomaly score sequence, specifically including: The key features of the high-frequency time-series segment data to be tested are extracted in the time domain to determine the key feature sequence; Unsupervised anomaly detection is performed on the key feature sequence using multiple parallel algorithms to obtain anomalous feature points; Calculate the anomaly score corresponding to the anomaly feature point, and normalize the anomaly score to determine the first anomaly score sequence; Based on the high-frequency time series data to be detected, a second anomaly score sequence is determined through data volatility analysis, specifically including: Set volatility monitoring parameters; wherein, the volatility monitoring parameters include: detection window length, normal range limit coefficient, and volatility detection evaluation index; Based on the detection window length and the volatility detection evaluation index, the volatility of adjacent time windows is compared, and the time point when the volatility of the adjacent time window is greater than the normal range limit coefficient is selected to determine the volatility offset point. Calculate the outlier score corresponding to the volatility offset point to determine the second outlier score sequence; After determining the second anomaly score sequence based on the high-frequency time series data to be detected through data volatility analysis, the method further includes: Contextual meta-features are extracted from the high-frequency time-series segment data to be detected to obtain meta-feature vectors; Based on the first anomaly score sequence and the second anomaly score sequence, meta-classification analysis through ensemble learning is used to obtain anomaly monitoring data for the high-frequency time series data, specifically including: Configure the base learner to be trained, and set the first abnormal score sequence and the second abnormal score sequence as input features. Train the base learner to be trained until the model converges to obtain the base learner. Based on the base learner, the base learner prediction vector is determined through probabilistic prediction analysis; The first abnormal score sequence, the second abnormal score sequence, the base learner prediction vector, and the meta-feature vector are concatenated to obtain the meta-learner input vector. Obtain the meta-learner to be trained, and use the input vector of the meta-learner as the training vector of the meta-learner to be trained, and train the meta-learner to be trained until the model converges, so as to determine the high-frequency time series data anomaly detection model. The high-frequency time series data is input into the high-frequency time series data anomaly detection model to obtain the anomaly monitoring data of the high-frequency time series data.

2. The high-frequency time-series data anomaly detection method based on an integrated strategy according to claim 1, characterized in that, The high-frequency time-series data is subjected to data standardization processing to obtain structured data, specifically including: The high-frequency time-series data is parsed to obtain a numerical time-series array; The numerical time series array is converted into a data format to obtain the structured data.

3. The high-frequency time-series data anomaly detection method based on an integrated strategy according to claim 1, characterized in that, The structured data is segmented using a sliding window to determine high-frequency time-series data segments, specifically including: Obtain the sliding window parameters, and based on the sliding window parameters, perform non-overlapping window processing on the data stream of the high-frequency time series data to obtain a non-overlapping sliding window; The structured data is segmented using a sliding window based on the non-overlapping sliding window to determine the high-frequency time-series data segments.

4. The high-frequency time-series data anomaly detection method based on an integrated strategy according to claim 1, characterized in that, Performing data quality checks on the high-frequency time series data segments to obtain the high-frequency time series data segments to be checked specifically includes: Data missing rate detection is performed on the high-frequency time-series data to obtain the data missing rate; Based on the missing data segment rate, the high-frequency time series segment data to be inspected is obtained by determining the missing rate threshold.

5. A high-frequency time-series data anomaly detection device based on an integrated strategy, characterized in that, The device includes: At least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: High-frequency time-series data is acquired, and the high-frequency time-series data is subjected to data standardization processing to obtain structured data; The structured data is segmented using a sliding window to determine high-frequency time-series data segments; The high-frequency time series data is subjected to a data quality check to obtain the high-frequency time series data to be checked; Based on the high-frequency time-series data to be inspected, a first anomaly score sequence is determined through unsupervised anomaly identification based on anomaly sensitivity. Based on the high-frequency time series data to be detected, a second anomaly score sequence is determined through data volatility analysis; Based on the first anomaly score sequence and the second anomaly score sequence, meta-classification analysis through ensemble learning is used to obtain anomaly monitoring data of the high-frequency time series data; Based on the high-frequency time-series data to be inspected, an unsupervised anomaly identification based on anomaly sensitivity is used to determine the first anomaly score sequence, specifically including: The key features of the high-frequency time-series segment data to be tested are extracted in the time domain to determine the key feature sequence; Unsupervised anomaly detection is performed on the key feature sequence using multiple parallel algorithms to obtain anomalous feature points; Calculate the anomaly score corresponding to the anomaly feature point, and normalize the anomaly score to determine the first anomaly score sequence; Based on the high-frequency time series data to be detected, a second anomaly score sequence is determined through data volatility analysis, specifically including: Set volatility monitoring parameters; wherein, the volatility monitoring parameters include: detection window length, normal range limit coefficient, and volatility detection evaluation index; Based on the detection window length and the volatility detection evaluation index, the volatility of adjacent time windows is compared, and the time point when the volatility of the adjacent time window is greater than the normal range limit coefficient is selected to determine the volatility offset point. Calculate the outlier score corresponding to the volatility offset point to determine the second outlier score sequence; After determining the second anomaly score sequence based on the high-frequency time series data to be detected through data volatility analysis, the method further includes: Contextual meta-features are extracted from the high-frequency time-series segment data to be detected to obtain meta-feature vectors; Based on the first anomaly score sequence and the second anomaly score sequence, meta-classification analysis through ensemble learning is used to obtain anomaly monitoring data for the high-frequency time series data, specifically including: Configure the base learner to be trained, and set the first abnormal score sequence and the second abnormal score sequence as input features. Train the base learner to be trained until the model converges to obtain the base learner. Based on the base learner, the base learner prediction vector is determined through probabilistic prediction analysis; The first abnormal score sequence, the second abnormal score sequence, the base learner prediction vector, and the meta-feature vector are concatenated to obtain the meta-learner input vector. Obtain the meta-learner to be trained, and use the input vector of the meta-learner as the training vector of the meta-learner to be trained, and train the meta-learner to be trained until the model converges, so as to determine the high-frequency time series data anomaly detection model. The high-frequency time series data is input into the high-frequency time series data anomaly detection model to obtain the anomaly monitoring data of the high-frequency time series data.

6. A non-volatile computer storage medium for high-frequency time-series data anomaly detection based on an integrated strategy, storing computer-executable instructions, characterized in that, The computer-executable instructions are set as follows: High-frequency time-series data is acquired, and the high-frequency time-series data is subjected to data standardization processing to obtain structured data; The structured data is segmented using a sliding window to determine high-frequency time-series data segments; The high-frequency time series data is subjected to a data quality check to obtain the high-frequency time series data to be checked; Based on the high-frequency time-series data to be inspected, a first anomaly score sequence is determined through unsupervised anomaly identification based on anomaly sensitivity. Based on the high-frequency time series data to be detected, a second anomaly score sequence is determined through data volatility analysis; Based on the first anomaly score sequence and the second anomaly score sequence, meta-classification analysis through ensemble learning is used to obtain anomaly monitoring data of the high-frequency time series data; Based on the high-frequency time-series data to be inspected, an unsupervised anomaly identification based on anomaly sensitivity is used to determine the first anomaly score sequence, specifically including: The key features of the high-frequency time-series segment data to be tested are extracted in the time domain to determine the key feature sequence; Unsupervised anomaly detection is performed on the key feature sequence using multiple parallel algorithms to obtain anomalous feature points; Calculate the anomaly score corresponding to the anomaly feature point, and normalize the anomaly score to determine the first anomaly score sequence; Based on the high-frequency time series data to be detected, a second anomaly score sequence is determined through data volatility analysis, specifically including: Set volatility monitoring parameters; wherein, the volatility monitoring parameters include: detection window length, normal range limit coefficient, and volatility detection evaluation index; Based on the detection window length and the volatility detection evaluation index, the volatility of adjacent time windows is compared, and the time point when the volatility of the adjacent time window is greater than the normal range limit coefficient is selected to determine the volatility offset point. Calculate the outlier score corresponding to the volatility offset point to determine the second outlier score sequence; After determining the second anomaly score sequence based on the high-frequency time series data to be detected through data volatility analysis, the method further includes: Contextual meta-features are extracted from the high-frequency time-series segment data to be detected to obtain meta-feature vectors; Based on the first anomaly score sequence and the second anomaly score sequence, meta-classification analysis through ensemble learning is used to obtain anomaly monitoring data for the high-frequency time series data, specifically including: Configure the base learner to be trained, and set the first abnormal score sequence and the second abnormal score sequence as input features. Train the base learner to be trained until the model converges to obtain the base learner. Based on the base learner, the base learner prediction vector is determined through probabilistic prediction analysis; The first abnormal score sequence, the second abnormal score sequence, the base learner prediction vector, and the meta-feature vector are concatenated to obtain the meta-learner input vector. Obtain the meta-learner to be trained, and use the input vector of the meta-learner as the training vector of the meta-learner to be trained, and train the meta-learner to be trained until the model converges, so as to determine the high-frequency time series data anomaly detection model. The high-frequency time series data is input into the high-frequency time series data anomaly detection model to obtain the anomaly monitoring data of the high-frequency time series data.

Images