Anomaly detection method and device of data, electronic equipment and storage medium
By dividing the time-continuous data into sub-data sequences and calculating similarity, and using dynamic time-normalized distance and correlation coefficient for anomaly detection, the false alarm problem caused by single-point jitter in existing technologies is solved, and the accuracy and efficiency of anomaly detection are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA PETROLEUM & CHEMICAL CORP
- Filing Date
- 2025-01-02
- Publication Date
- 2026-07-03
AI Technical Summary
When processing unevenly distributed data, existing technologies have high computational complexity for anomaly detection methods based on distance and density, lack objectivity for threshold-based detection methods, and are highly sensitive to the selection of the number of clusters, resulting in a large number of false alarms caused by single-point jitter.
By acquiring time-continuous data, the data is divided into sub-data sequences based on a preset time period. The similarity between sub-data sequences is calculated using dynamic time regularization distance and correlation coefficient, and anomaly detection is performed using similarity thresholds and clustering algorithms.
It reduces false alarms caused by single-point jitter and improves the accuracy and efficiency of anomaly detection.
Smart Images
Figure CN122332997A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of data processing technology, and in particular relates to a method, apparatus, electronic device and storage medium for detecting data anomalies. Background Technology
[0002] Data anomalies can be categorized into several types, including outliers, fluctuations, fault anomalies, and time series anomalies. Currently, anomaly detection methods based on distance and density have high computational complexity and are prone to errors when handling unevenly distributed data. Threshold-based detection methods rely on human experience for threshold setting, lacking objectivity. Clustering-based methods are highly sensitive to the number of clusters selected, making it difficult to attribute outlier scores to objects. These techniques typically focus on analyzing outliers (single points) in device data, but single-point fluctuations are common in data, leading to numerous false alarms. Summary of the Invention
[0003] This application provides a data anomaly detection method, apparatus, electronic device, and storage medium. By considering the similarity between sub-data sequences, the anomaly detection of sub-data sequences can be achieved, thereby reducing false alarms caused by single-point jitter.
[0004] According to a first aspect of this application, embodiments of this application provide a method for detecting data anomalies, including:
[0005] Obtain data that is continuous over time;
[0006] The data is divided into sub-data sequences based on a preset time period;
[0007] Determine the similarity between any two sub-data sequences;
[0008] Anomaly detection is performed on each sub-data sequence based on the similarity between any two sub-data sequences.
[0009] In some embodiments, determining the similarity between any two sub-data sequences includes:
[0010] Determine the dynamic time-warped distance between any two target sub-data sequences, and determine the correlation coefficient between the two target sub-data sequences;
[0011] The similarity between two target sub-data sequences is calculated based on the dynamic time warp distance and the correlation coefficient to obtain the similarity between any two sub-data sequences.
[0012] In some embodiments, calculating the similarity between two target sub-data sequences based on the dynamic time-warped distance and the correlation coefficient includes:
[0013] Subtract the correlation coefficient from 1 to obtain the calculated value;
[0014] Multiplying the calculated value by the dynamic time-normalized distance yields the similarity between the two target sub-data sequences.
[0015] In some embodiments, the anomaly detection of each sub-data sequence based on the similarity between any two sub-data sequences includes:
[0016] Each sub-data sequence is grouped based on the similarity between any two sub-data sequences to obtain at least one group;
[0017] Determine the number of sub-data sequences in each group;
[0018] Anomaly detection is performed on the sub-data sequences in each group based on the number of sub-data sequences in each group.
[0019] In some embodiments, grouping the sub-data sequences based on the similarity between any two sub-data sequences to obtain at least one group includes:
[0020] Two sub-data sequences with a similarity greater than a similarity threshold are identified as the first group, and any sub-data sequence in the first group is identified as the seed sequence.
[0021] Two sub-data sequences with a similarity less than the similarity threshold are grouped into the second group;
[0022] Sub-data sequences in each second group whose similarity to the seed sequence is greater than a similarity threshold are merged into the first group to obtain a merged first group and a merged second group, wherein the merged second group includes: unmerged sub-data sequences.
[0023] In some embodiments, the anomaly detection of sub-data sequences in each group based on the number of sub-data sequences in each group includes:
[0024] Sub-data sequences in groups with a sub-data sequence count of 1 are identified as outliers.
[0025] In some embodiments, the method further includes:
[0026] If the sub-data sequence contains abnormal data, an alarm message will be output.
[0027] According to a second aspect of this application, embodiments of this application provide a data anomaly detection device, comprising:
[0028] The acquisition module is used to acquire data that is continuous over time.
[0029] A partitioning module is used to divide the data into sub-data sequences based on a preset time period;
[0030] The determination module is used to determine the similarity between any two sub-data sequences;
[0031] The detection module is used to perform anomaly detection on each sub-data sequence based on the similarity between any two sub-data sequences.
[0032] According to a third aspect of this application, an embodiment of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in any of the preceding claims.
[0033] According to a fourth aspect of this application, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any of the preceding claims.
[0034] According to a fifth aspect of this application, embodiments of this application provide a computer program product that, when run on a terminal device, causes an electronic device to perform any of the methods described above.
[0035] The beneficial effects of the embodiments in this application compared with the prior art are:
[0036] The data anomaly detection method provided in this application involves acquiring time-continuous data; dividing the data into sub-data sequences based on a preset time period; determining the similarity between any two sub-data sequences; and performing anomaly detection on each sub-data sequence based on the similarity between any two sub-data sequences. By considering the similarity between sub-data sequences, the method can detect sub-data sequences, thereby reducing false alarms caused by single-point jitter.
[0037] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0039] Figure 1 A schematic diagram illustrating the implementation process of a data anomaly detection method provided for the purposes of this application;
[0040] Figure 2 A schematic diagram illustrating the implementation process of another data anomaly detection method provided for the implementation of this application;
[0041] Figure 3 This application provides a schematic diagram of a data processing procedure as illustrated in its embodiments.
[0042] Figure 4 This is a schematic diagram of the structure of the data anomaly detection device provided in the embodiments of this application;
[0043] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0044] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0045] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.
[0046] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0047] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to control." Similarly, the phrases "if determined" or "if controlled" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once controlled," or "in response to control."
[0048] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0049] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.
[0050] Based on the technical problems in related technologies, embodiments of this application provide a method for detecting data anomalies, which can be applied to electronic devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). It should be noted that embodiments of this application do not impose any limitations on the specific type of electronic device.
[0051] The data anomaly detection method provided in this application can be implemented by the processor of an electronic device calling program code, wherein the program code can be stored in a computer storage medium.
[0052] Figure 1 A schematic diagram illustrating the implementation flow of a data anomaly detection method provided for the purposes of this application is shown below. Figure 1 As shown, the methods for detecting data anomalies include:
[0053] Step S101: Obtain data that is continuous in time.
[0054] In this embodiment, the data can be sensor data, device operating data, financial market data, network traffic data, etc. This data has timestamps, and the timestamps are continuous.
[0055] In this embodiment, data can be directly collected by a sensor, and the electronic device communicates with the sensor to obtain sensor data. In some embodiments, the sensor can send the collected data to a cloud server or local database via wireless communication (such as Wi-Fi, Bluetooth, LoRa, etc.), and the electronic device can obtain the data from the cloud server or local database.
[0056] In some embodiments, industrial automation systems (such as SCADA, PLC) can be used to monitor equipment status and record data, which may include temperature, pressure, running time, etc. Electronic devices can obtain equipment operating data from the industrial automation system through API interfaces.
[0057] In some embodiments, network monitoring tools can be used to obtain network traffic data, or network data can be obtained from router and switch logs, etc.
[0058] Step S102: Divide the data into sub-data sequences based on a preset time period.
[0059] In this embodiment of the application, a time window can be set as needed to obtain a preset time period. For example, the preset time period can be 1 minute, 5 minutes, etc.
[0060] In this embodiment, a sliding window technique can be used to divide data into sub-data sequences, each with a fixed duration. The duration of each sub-data sequence is consistent. For example, taking device data as an example, with a preset time period of K, the device data can be divided into multiple sub-data sequences.
[0061] Step S103: Determine the similarity between any two sub-data sequences.
[0062] In this embodiment, the similarity between any two sub-data sequences can be determined by the correlation coefficient and dynamic time-warped distance between any two sub-data sequences.
[0063] In this embodiment of the application, step S103 can be implemented through the following steps:
[0064] Step S1031: Determine the dynamic time warping distance between any two target sub-data sequences, and determine the correlation coefficient between the two target sub-data sequences.
[0065] In this embodiment, Dynamic Time Warping (DTW) is a technique for measuring the similarity between two time series. It can be used to compare differences in time, velocity, or length, helping to understand and measure the relationship between time series. Consider two time series and their respective lengths. Here, all elements are assumed to be located in the same d-dimensional space. Any two sub-data series can include X and Y, and the DTW distance from X to Y can be calculated using formula (1):
[0066]
[0067] DTW is calculated as the square root of the sum of squared distances between each point in X and its nearest point in Y. DTW performs n×m comparisons between each point in the X sequence and each point in the Y sequence. For each point x in X... i DTW selects the nearest point y in Y. j Perform distance calculation.
[0068] In this embodiment of the application, the dynamic time warping distance can be used to measure the shape similarity between two sub-data sequences.
[0069] In this embodiment of the application, in addition to directly measuring the distance between sub-data sequences, the data is also modeled based on time series, and the similarity of their model parameters is compared. Pearson's similarity measurement method is shown in formula (2):
[0070]
[0071] Where COR(X) T ,Y T ) represents X T and Y T The correlation coefficient between the two sub-data sequences and X T and Y T The correlation coefficient is the mean of the series. A correlation coefficient of 1 indicates that the series are completely identical; a correlation coefficient of -1 indicates that there is a negative correlation between the series; and a correlation coefficient of 0 indicates that there is no correlation.
[0072] In this embodiment of the application, the similarity of Pearson can reflect their trend similarity.
[0073] Step S1032: Calculate the similarity between two target sub-data sequences based on the dynamic time warp distance and the correlation coefficient to obtain the similarity between any two sub-data sequences.
[0074] In this embodiment, similarity can be determined based on DTW distance and correlation coefficient. Generally, the smaller the DTW distance and the closer the correlation coefficient is to 1, the more similar the two sequences are.
[0075] In this embodiment, a weighted formula can be set to calculate the final similarity. The weights of the weighted formula can be adjusted according to the application scenario. For example, if shape similarity is considered more important, the weight corresponding to the dynamic time-warped distance in the weighted formula can be increased.
[0076] In some embodiments, step S1032 can be implemented by the following steps:
[0077] Step S1: Subtract the correlation coefficient from 1 to obtain the calculated value;
[0078] Step S2: Multiply the calculated value by the dynamic time-normalized distance to obtain the similarity between the two target sub-data sequences.
[0079] In this embodiment of the application, the above steps can be represented by calculation formula (3):
[0080] SIMILARITY = DTW(x,y) · (1 - COR(x,y)) (3).
[0081] Step S104: Perform anomaly detection on each sub-data sequence based on the similarity between any two sub-data sequences.
[0082] In this embodiment of the application, a similarity threshold can be set. Sequences below the threshold are marked as abnormal. Clustering algorithms such as K-means or DBSCAN are used to group sequences with high similarity. Abnormal sequences may form isolated sequences, and isolated sequences are abnormal sequences.
[0083] In some embodiments, supervised or unsupervised learning models (such as Isolation Forest, Support Vector Machine, etc.) can be used for anomaly detection. The similarity between any two sub-data sequences can be input into the unsupervised learning model to obtain the anomaly detection results.
[0084] The data anomaly detection method provided in this application involves acquiring time-continuous data; dividing the data into sub-data sequences based on a preset time period; determining the similarity between any two sub-data sequences; and performing anomaly detection on each sub-data sequence based on the similarity between any two sub-data sequences. By considering the similarity between sub-data sequences, the method can detect sub-data sequences, thereby reducing false alarms caused by single-point jitter.
[0085] In some embodiments, step S104 can be implemented through the following steps:
[0086] Step S1041: Group each sub-data sequence based on the similarity between any two sub-data sequences to obtain at least one group.
[0087] In this embodiment of the application, sub-data sequences can be grouped using clustering algorithms (such as hierarchical clustering, K-means, or DBSCAN).
[0088] In some embodiments, step S1041 can be implemented by the following steps:
[0089] Step S11: Determine the first group of two sub-data sequences whose similarity is greater than the similarity threshold, and determine any sub-data sequence in the first group as a seed sequence.
[0090] In this embodiment, the similarity between all sub-data sequences can be calculated to generate a similarity matrix. For each pair of sub-data sequences, if their similarity is greater than a set threshold, they are grouped into a first group. Any sub-data sequence is selected from the first group as a seed sequence.
[0091] In this embodiment of the application, all sequence pairs with similarity greater than a threshold can be found through the similarity matrix, and their indices are added to the first group, thereby determining the first group.
[0092] In this application embodiment, the similarity threshold can be configured. For example, it can be configured to 0.6, and in some embodiments, it can be configured to 0.7, etc.
[0093] Step S12: The two sub-data sequences with similarity less than the similarity threshold are grouped into the second group.
[0094] In this embodiment of the application, for each pair of sub-data sequences, if their similarity is less than a threshold, they are grouped into a second group.
[0095] Step S13: Merge the sub-data sequences in each second group that have a similarity greater than the seed sequence into the first group to obtain the merged first group and the merged second group, wherein the merged second group includes: the unmerged sub-data sequences.
[0096] In this embodiment, for each sub-data sequence in the second group, their similarity to the seed sequence is checked. If the similarity is greater than a threshold, these sub-data sequences are merged into the first group. The second group is then updated, and the sub-data sequences in the merged second group are the unmerged sub-data sequences.
[0097] Step S1042: Determine the number of sub-data sequences in each group.
[0098] In this embodiment of the application, the number of sub-data sequences in each group can be recorded.
[0099] Step S1043: Perform anomaly detection on the sub-data sequences in each group based on the number of sub-data sequences in each group.
[0100] In this embodiment, sub-data sequences in groups with a sub-data sequence count of 1 are identified as abnormal data. In this embodiment, if a target group has a sub-data sequence count of 1, then that sequence can be considered an isolated sequence.
[0101] In this embodiment of the application, sub-data sequences in groups where the number of sub-data sequences is greater than 1 are determined as normal data.
[0102] In this embodiment, abnormal data may be or include real-time or recorded sensor data associated with an abnormal state, or data associated with a device experiencing or otherwise related to an abnormal situation. Abnormal data may also include other data (e.g., operation setpoints, operation configurations, control data) associated with or related to an abnormal state or a device experiencing or otherwise related to an abnormal state. Abnormal data may also include data indicating an abnormal situation or which device is experiencing or otherwise related to an abnormal situation.
[0103] Based on the foregoing embodiments, this application provides a method for detecting data anomalies. Figure 2 This is a schematic diagram illustrating the implementation process of a data anomaly detection method provided in an embodiment of this application, as shown below. Figure 2 As shown, it includes:
[0104] Step S201: Input continuous data.
[0105] In this embodiment, the continuous data can be the data described in the above embodiments. This data is a time series, which is a set of data points arranged in chronological order, where each data point represents an observation or statistical result at a specific point in time. These data points can be continuous or discrete.
[0106] Step S202: Select the detection cycle duration.
[0107] In this embodiment, the cycle duration is the same as the preset time cycle in the above embodiments.
[0108] Step S203: Set each sequence segment as a cluster.
[0109] Step S204: Calculate the Pearson and DTW distance matrices.
[0110] In this embodiment of the application, the Pearson and DTW distances between any two sequences can be calculated.
[0111] Step S205: After grouping, determine whether the number of clusters is equal to 1.
[0112] In this embodiment of the application, if the number of clusters is 1, then step S206 is executed; if it is not 1, then step S208 is executed.
[0113] Step S206: Identify abnormal discrete points.
[0114] Step S207: Alarm activated.
[0115] In this application embodiment, an alarm can be triggered by sound; in some embodiments, an alarm can be triggered by a signal light.
[0116] Step S208: Merge the nearest cluster.
[0117] Step S209: Update the distance matrix.
[0118] The method provided in this application matches each sequence segment according to correlation, calculates the distance between continuous points using the DTW algorithm, and finds time series with similar trends for grouping. Sequence data that deviates from the group (outliers) are determined to be (continuous) abnormal points.
[0119] In the embodiments of this application, Figure 3 This is a schematic diagram illustrating a data processing procedure provided in an embodiment of this application, such as... Figure 3 As shown, all time series data 1 are divided into clusters 2 according to a custom period. Partition clustering uses similarity metrics to combine the most similar cluster pairs. The similarity between clusters is judged by dynamic time programming (DTW) to measure the distance between series and Pearson similarity. The closest clusters are found and merged. When the similarity threshold is set high, it can be used to detect fluctuation points. When the similarity threshold is set low, it can be used to detect clustering trends. Through detection, abnormal data in the data can be obtained.
[0120] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0121] According to the foregoing embodiments, this application provides a data anomaly detection device. The various modules and units included in the device can be implemented by a processor in a computer device; of course, they can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.
[0122] This application provides a data anomaly detection device. Figure 4 This is a schematic diagram of the structure of a data anomaly detection device provided in an embodiment of this application, as shown below. Figure 4 As shown, the data anomaly detection device 400 includes:
[0123] Module 401 is used to acquire data that is continuous in time.
[0124] The partitioning module 402 is used to divide the data into sub-data sequences based on a preset time period;
[0125] Module 403 is used to determine the similarity between any two sub-data sequences;
[0126] The detection module 404 is used to perform anomaly detection on each sub-data sequence based on the similarity between any two sub-data sequences.
[0127] In some embodiments, determining the similarity between any two sub-data sequences includes:
[0128] Determine the dynamic time-warped distance between any two target sub-data sequences, and determine the correlation coefficient between the two target sub-data sequences;
[0129] The similarity between two target sub-data sequences is calculated based on the dynamic time warp distance and the correlation coefficient to obtain the similarity between any two sub-data sequences.
[0130] In some embodiments, calculating the similarity between two target sub-data sequences based on the dynamic time-warped distance and the correlation coefficient includes:
[0131] Subtract the correlation coefficient from 1 to obtain the calculated value;
[0132] Multiplying the calculated value by the dynamic time-normalized distance yields the similarity between the two target sub-data sequences.
[0133] In some embodiments, the anomaly detection of each sub-data sequence based on the similarity between any two sub-data sequences includes:
[0134] Each sub-data sequence is grouped based on the similarity between any two sub-data sequences to obtain at least one group;
[0135] Determine the number of sub-data sequences in each group;
[0136] Anomaly detection is performed on the sub-data sequences in each group based on the number of sub-data sequences in each group.
[0137] In some embodiments, grouping the sub-data sequences based on the similarity between any two sub-data sequences to obtain at least one group includes:
[0138] Two sub-data sequences with a similarity greater than a similarity threshold are identified as the first group, and any sub-data sequence in the first group is identified as the seed sequence.
[0139] Two sub-data sequences with a similarity less than the similarity threshold are grouped into the second group;
[0140] Sub-data sequences in each second group whose similarity to the seed sequence is greater than a similarity threshold are merged into the first group to obtain a merged first group and a merged second group, wherein the merged second group includes: unmerged sub-data sequences.
[0141] In some embodiments, the anomaly detection of sub-data sequences in each group based on the number of sub-data sequences in each group includes:
[0142] Sub-data sequences in groups with a sub-data sequence count of 1 are identified as outliers.
[0143] In some embodiments, the data anomaly detection device further includes:
[0144] The alarm module is used to output alarm information.
[0145] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.
[0146] in addition, Figure 4 The data anomaly detection device shown can be a software unit, hardware unit, or a combination of software and hardware built into existing electronic devices. It can also be integrated into electronic devices as a separate component or exist as a standalone terminal device.
[0147] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0148] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 3 in this embodiment may include: at least one processor 30 ( Figure 5 Only one processor 30, memory 31, and computer program 32 stored in memory 31 and executable on at least one processor 30 are shown. When the processor 30 executes the computer program 32, it implements the steps in any of the above method embodiments, or the processor 30 executes the computer program 32 to implement the functions of each module / unit in the above device embodiments.
[0149] For example, computer program 32 may be divided into one or more modules / units, one or more of which are stored in memory 31 and executed by processor 30 to complete this application. One or more modules / units may be a series of computer program 32 instruction segments capable of performing a specific function, which describe the execution process of computer program 32 in electronic device 3.
[0150] This application also provides a computer-readable storage medium storing a computer program 32, which, when executed by a processor 30, implements the steps described in the above-described method embodiments.
[0151] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.
[0152] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program 32 instructing related hardware. The computer program 32 can be stored in a computer-readable storage medium, and when executed by the processor 30, it can implement the steps of the various method embodiments described above. The computer program 32 includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a terminal, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.
[0153] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0154] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0155] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0156] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0157] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method of anomaly detection of data, characterized by, include: Obtain data that is continuous over time; The data is divided into sub-data sequences based on a preset time period; Determine the similarity between any two sub-data sequences; Anomaly detection is performed on each sub-data sequence based on the similarity between any two sub-data sequences.
2. The method of claim 1, wherein, Determining the similarity between any two sub-data sequences includes: Determine the dynamic time-warped distance between any two target sub-data sequences, and determine the correlation coefficient between the two target sub-data sequences; The similarity between two target sub-data sequences is calculated based on the dynamic time warp distance and the correlation coefficient to obtain the similarity between any two sub-data sequences.
3. The method of claim 2, wherein, The calculation of the similarity between the two target sub-data sequences based on the dynamic time-warped distance and the correlation coefficient includes: Subtract the correlation coefficient from 1 to obtain the calculated value; Multiplying the calculated value by the dynamic time-normalized distance yields the similarity between the two target sub-data sequences.
4. The method according to claim 1, characterized in that, The anomaly detection of each sub-data sequence based on the similarity between any two sub-data sequences includes: Each sub-data sequence is grouped based on the similarity between any two sub-data sequences to obtain at least one group; Determine the number of sub-data sequences in each group; Anomaly detection is performed on the sub-data sequences in each group based on the number of sub-data sequences in each group.
5. The method of claim 4, wherein, The method of grouping each sub-data sequence based on the similarity between any two sub-data sequences to obtain at least one group includes: Two sub-data sequences with a similarity greater than a similarity threshold are identified as the first group, and any sub-data sequence in the first group is identified as the seed sequence. Two sub-data sequences with a similarity less than the similarity threshold are grouped into the second group; Sub-data sequences in each second group whose similarity to the seed sequence is greater than a similarity threshold are merged into the first group to obtain a merged first group and a merged second group, wherein the merged second group includes: unmerged sub-data sequences.
6. The method of claim 4, wherein, The anomaly detection of sub-data sequences in each group based on the number of sub-data sequences in each group includes: Sub-data sequences in groups with a sub-data sequence count of 1 are identified as outliers.
7. The method of claim 6, wherein, The method further includes: Output alarm information.
8. An abnormality detection device of data, characterized by, include: The acquisition module is used to acquire data that is continuous over time. A partitioning module is used to divide the data into sub-data sequences based on a preset time period; The determination module is used to determine the similarity between any two sub-data sequences; The detection module is used to perform anomaly detection on each sub-data sequence based on the similarity between any two sub-data sequences.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.