Abnormality detection method and device and storage medium

Abstract

The invention relates to the technical field of data processing, and discloses an anomaly detection method and device and a storage medium. The method comprises the steps of obtaining first time sequence detection data when a target detection object is in a normal state and second time sequence detection data to be subjected to anomaly detection; determining a first statistical feature of the first time sequence detection data and a second statistical feature of the second time sequence detection data; based on the difference between the first statistical feature and the second statistical feature, determining whether the second time sequence detection data corresponding to the second statistical feature is abnormal or not; under the condition that the second time series detection data corresponding to the second statistical feature is abnormal, determining an abnormal type of the abnormal time series data based on the first statistical feature and / or a third statistical feature of the abnormal time series data; the problem that missing detection or false detection is prone to occurring in a traditional anomaly detection mode can be solved, and anomaly detection accuracy is improved. And meanwhile, the accuracy of abnormal type judgment can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to an anomaly detection method, apparatus and storage medium. Background Technology

[0002] With the development of the Industrial Internet and intelligent manufacturing, real-time monitoring of the operating status of the environment or equipment for anomaly detection is widely used. Generally, anomaly detection methods include: collecting a time-series data of the target object (such as the water environment to be monitored, water environment monitoring equipment, etc.) through sensors; and performing anomaly detection on the time-series data to identify abnormal segments.

[0003] A typical anomaly detection method includes: comparing individual data values ​​in a time detection sequence with preset thresholds; if the data value is lower or higher than the preset threshold, an anomaly is determined to exist.

[0004] However, relying on a fixed threshold to detect anomalies in a single data value in a time-based detection sequence is prone to misjudgment, resulting in low accuracy in anomaly detection. Summary of the Invention

[0005] In view of this, this disclosure proposes an anomaly detection method, apparatus and storage medium, which can solve the problem of missed detection or false detection when performing anomaly detection by comparing a single data point with a threshold, thereby improving the accuracy of anomaly detection.

[0006] According to one aspect of this disclosure, an anomaly detection method is provided, the method comprising:

[0007] Acquire the first time-series detection data when the target object is in a normal state, and the second time-series detection data to be used for anomaly detection;

[0008] Determine a first statistical feature corresponding to the first time series detection data and a second statistical feature corresponding to the second time series detection data; wherein the feature type of the first statistical feature is the same as the feature type of the second statistical feature;

[0009] Based on the difference between the first statistical feature and the second statistical feature, determine whether there is an anomaly in the second time series detection data corresponding to the second statistical feature;

[0010] If there is an anomaly in the second time series detection data corresponding to the second statistical feature, the second time series detection data is regarded as abnormal time series data. The anomaly type of the abnormal time series data is determined based on the first statistical feature and / or the third statistical feature corresponding to the abnormal time series data. The anomaly type includes baseline drift and / or abnormal data jitter. The first statistical feature includes: mean, standard deviation and slope; and / or, the third statistical feature includes statistical features obtained by statistically analyzing the data deviation between the data value at each time step in the abnormal time series data and the data value at the corresponding time step in the smoothed abnormal time series data.

[0011] In one possible implementation, when the first statistical feature includes: mean, standard deviation, and slope, determining the anomaly type of the abnormal time series data based on the first statistical feature and / or the third statistical feature corresponding to the abnormal time series data includes:

[0012] Determine the feature ratio between the third statistical feature corresponding to the abnormal time series data and the first statistical feature; the feature ratio includes the feature ratio corresponding to the mean, the feature ratio corresponding to the standard deviation, and the feature ratio corresponding to the slope; if the feature ratio exceeds a preset range, determine that the abnormality type of the abnormal time series data includes baseline drift; and / or,

[0013] Based on the data value at each moment in the abnormal time series data, the mean and standard deviation in the first statistical feature, the degree of mean deviation is determined; based on the slope in the third statistical feature and the slope in the first statistical feature, the degree of slope deviation is determined; based on the degree of mean deviation and the degree of slope deviation, the jitter score of the abnormal time series data is determined; if the jitter score exceeds the normal jitter score range, the abnormal type of the abnormal time series data is determined to include abnormal data jitter.

[0014] In one possible implementation, where the third statistical feature includes a statistical feature obtained by statistically analyzing the data deviation between the data value at each time step in the abnormal time series data and the corresponding data value at the smoothed abnormal time series data, determining the anomaly type of the abnormal time series data based on the first statistical feature and / or the third statistical feature corresponding to the abnormal time series data includes:

[0015] The abnormal time series data is smoothed to obtain smoothed abnormal time series data;

[0016] Determine the data deviation between the data value at each moment in the abnormal time series data and the data value at the corresponding moment in the smoothed abnormal time series data;

[0017] The data deviations at each time point are segmented to obtain multiple deviation segments;

[0018] Determine the mean and standard deviation of each of the plurality of deviation segments;

[0019] The threshold for the degree of dispersion of each deviation segment is determined based on the mean and standard deviation of each deviation segment;

[0020] If the data deviation in any deviation segment exceeds the dispersion threshold, the abnormality type of the abnormal time series data is determined to include abnormal data jitter.

[0021] In one possible implementation, the first time-series detection data of the target object in a normal state is obtained, including:

[0022] The original monitoring time series of the target detection object is decomposed and the target component is extracted. The target component includes a trend time series and / or a residual time series.

[0023] The target component is divided into multiple time-series data segments;

[0024] For each of the plurality of time-series data segments, a fourth statistical feature of the time-series data segment is obtained, and an instability score of the time-series data segment is determined based on the fourth statistical feature.

[0025] Based on the instability scores of the multiple time-series data segments, candidate time-series data segments whose instability scores fall within a preset score range are determined from the multiple time-series data segments;

[0026] From all candidate time series data segments, a target time series data segment that meets the preset continuity requirement is determined, and the target time series data segment constitutes the first time series detection data.

[0027] In one possible implementation, the target time-series data segment that meets the preset continuity requirement is determined from all candidate time-series data segments, including:

[0028] Determine the first total duration of all candidate time-series data segments;

[0029] If the first total duration meets the preset first duration condition, the candidate time series data segments among all candidate time series data segments that satisfy the condition that the adjacent interval duration is less than or equal to the preset interval duration are determined as continuous candidate time series data segments.

[0030] If the second total duration of the plurality of consecutive candidate time series data segments meets the preset second duration condition, the plurality of consecutive candidate time series data segments are determined as the target time series data segment.

[0031] In one possible implementation, the fourth statistical feature includes three feature types, and accordingly, determining the instability score of the time-series data segment based on the fourth statistical feature includes:

[0032] The fourth statistical feature of each feature type is normalized to obtain the corresponding normalized feature value;

[0033] The instability score is determined based on the preset weights corresponding to each feature type and the normalized feature values.

[0034] In one possible implementation, obtaining the fourth statistical feature of the time-series data segment includes:

[0035] Determine the standard deviation of the time-series data segment;

[0036] The mean change between the mean of the time series data segment and the mean of an adjacent time series data segment is determined to obtain the mean change corresponding to the time series data segment;

[0037] Determine the mean and standard deviation of the mean change corresponding to the plurality of time series data segments, and based on the mean change corresponding to the time series data segments, the mean and standard deviation of the mean change, determine the standard score of the mean change of the time series data segments;

[0038] The three fourth statistical characteristics of the time series data segment include: the standard deviation of the time series data segment, the change in mean, and the standard score of the change in mean.

[0039] In one possible implementation, the second time-series detection data to be detected for anomaly is obtained, including:

[0040] The original time series data of the target object to be detected is decomposed and the components to be detected are extracted; the components to be detected include the time series of the trend to be detected and / or the time series of the residual to be detected.

[0041] If the duration of the component to be detected is greater than a preset duration threshold, the component to be detected is segmented to obtain multiple second time series detection data.

[0042] In one possible implementation, the number of the second time series detection data is multiple, and each second time series detection data corresponds to a set of the second statistical features;

[0043] Accordingly, based on the difference between the first statistical feature and the second statistical feature, it is determined whether there are anomalies in the second time series detection data corresponding to the second statistical feature, including:

[0044] Cluster the first statistical feature and multiple sets of the second statistical features to obtain at least one feature cluster;

[0045] Among multiple sets of second statistical features, a target second statistical feature belonging to a different feature cluster from the first statistical feature is determined, and the second time series detection data corresponding to the target second statistical feature is determined as abnormal time series data.

[0046] In one possible implementation, the first time series detection data includes multiple components. Accordingly, based on the difference between the first statistical feature and the second statistical feature, determining whether the second time series detection data corresponding to the second statistical feature is abnormal includes:

[0047] Determine the mean and standard deviation of the first statistical feature corresponding to each first time series detection data in multiple first time series detection data;

[0048] Based on the mean and standard deviation of the first statistical feature, the first statistical feature of each first time series detection data is standardized to obtain the standardized first statistical feature;

[0049] Based on the mean and standard deviation of the first statistical feature, the second statistical feature is standardized to obtain the standardized second statistical feature;

[0050] By comparing the differences between the standardized first statistical feature and the standardized second statistical feature, it is determined whether there are any anomalies in the second time series detection data corresponding to the second statistical feature.

[0051] According to another aspect of this disclosure, an anomaly detection apparatus is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described method.

[0052] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method.

[0053] By acquiring first-time-series detection data of the target object in a normal state and second-time-series detection data to be detected for anomalies; determining the first statistical feature of the first-time-series detection data and the second statistical feature of the second-time-series detection data; determining whether there is anomaly in the second-time-series detection data corresponding to the second statistical feature based on the difference between the first and second statistical features; if there is anomaly in the second-time-series detection data corresponding to the second statistical feature, the second-time-series detection data is regarded as anomalous time-series data, and the anomaly type of the anomalous time-series data is determined based on the first statistical feature and / or the third statistical feature of the anomalous time-series data; on the one hand, since anomaly detection can be performed from the perspective of the overall data distribution, the problem of missed or false detections that easily occur when performing anomaly detection by comparing a single data point with a threshold can be solved, thus improving the accuracy of anomaly detection. On the other hand, since the first statistical feature of the first-time-series detection data in a normal state is used as a reference for anomaly judgment, the accuracy of anomaly detection can be further improved. On the other hand, anomaly detection based on the first and second statistical features, compared to directly using the first and second time-series detection data, can reduce the amount of data computation and improve anomaly identification efficiency. Simultaneously, it reduces the impact of noise data in the first and second time-series detection data, further improving the accuracy of anomaly detection. Furthermore, identifying baseline drift and data anomaly jitter based on the statistical distribution characteristics of these two phenomena can solve the problem of traditional anomaly type identification methods failing to accurately distinguish and simultaneously identify baseline drift and data anomaly jitter, thus improving the accuracy of anomaly type identification.

[0054] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0055] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0056] Figure 1 A flowchart of an anomaly detection method according to an embodiment of the present disclosure is shown;

[0057] Figure 2 A schematic diagram illustrating seasonal decomposition of an original monitoring time series according to an embodiment of the present disclosure is shown.

[0058] Figure 3 A schematic diagram illustrating the segmentation of a target component according to an embodiment of the present disclosure is shown;

[0059] Figure 4A schematic diagram illustrating the segmentation of the component to be detected according to an embodiment of the present disclosure is shown;

[0060] Figure 5 A block diagram of an anomaly detection apparatus according to another embodiment of the present disclosure is shown. Detailed Implementation

[0061] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0062] As used herein, the terms “comprising,” “including,” “having,” or variations thereof are open-ended and include one or more of the stated features, integrals, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integrals, elements, steps, components, functions, or groups thereof.

[0063] When an element is referred to as “connected,” “coupled,” “responding,” or a variation thereof relative to another element, it may be directly connected, coupled, or responding to another element, or there may be an intermediate element present.

[0064] Although the terms first, second, third, etc., may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another. Therefore, without departing from the teachings of the inventive concept, a first element / operation in some embodiments may be referred to as a second element / operation in other embodiments.

[0065] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0066] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0067] Figure 1A flowchart of an anomaly detection method according to an embodiment of this disclosure is shown. This embodiment illustrates the method in an electronic device, which can be a user terminal or a server. The user terminal can be a computer, tablet computer, mobile phone, etc. This embodiment does not limit the implementation of the electronic device. Optionally, the electronic device is communicatively connected to a sensor used to monitor the target object to obtain the time-series detection data collected by the sensor for anomaly detection.

[0068] like Figure 1 As shown, the method includes:

[0069] Step 101: Obtain the first time series detection data when the target object is in a normal state, and the second time series detection data to be detected for anomalies.

[0070] In one example, the first time series detection data is the historical time series of the target object when it is in a normal state (i.e., no abnormality exists). Optionally, the first time series detection data can be obtained by manual screening or can be automatically identified by electronic devices. This embodiment does not limit the method of obtaining the first time series detection data.

[0071] The target detection object can be the environment, such as aquatic environment, soil environment, etc.; or it can be operating equipment, such as industrial equipment, etc. This embodiment does not limit the application scenario of the anomaly detection method or the type of target detection object.

[0072] In one example, the target detection object is the aquatic environment, and the first time series detection data can be time series data corresponding to any monitoring indicator such as flow rate, water quality, and liquid level. Among them, the monitoring indicators for water quality types include any one of the following: chemical oxygen demand, conductivity, ammonia nitrogen, total hardness, temperature, pH value, turbidity, total organic carbon, five-day biochemical oxygen demand, total phosphorus, total nitrogen, suspended solids, total dissolved solids, petroleum hydrocarbons, anionic surfactants, cyanide, sulfide, fluoride, organophosphorus compounds, sulfate, mercury, chromium, cadmium, arsenic, lead, nickel, beryllium, silver, selenium, copper, zinc, manganese, iron, volatile phenols, benzene series compounds, aniline compounds, and nitrobenzene.

[0073] The monitoring index types of the second time series detection data and the first time series detection data are the same. For example, if the first time series detection data is the time series data corresponding to the chemical oxygen demand under normal water environment conditions, then the second time series detection data is the time series data corresponding to the chemical oxygen demand under the test conditions.

[0074] Taking the first time-series detection data obtained automatically through electronic device identification as an example, the acquisition of the first time-series detection data when the target object is in a normal state includes the following steps 1011-1015:

[0075] Step 1011: Decompose the original monitoring time series of the target object and extract the target component. The target component includes the trend time series and / or the residual time series.

[0076] The original monitoring time series can be a time series collected by the sensor. The duration of the original monitoring time series can be 1 day, 12 hours, etc., and this embodiment does not limit the duration of the original monitoring time series.

[0077] The original monitoring time series of the target object is decomposed, including seasonal decomposition. Seasonal decomposition separates systematic components (such as trends and periodicity) from unsystematic noise in the original monitoring time series. Specifically, the components obtained from seasonal decomposition mainly include the following:

[0078] Trend time series: used to reflect the long-term direction of change of the original monitoring time series, such as an increasing trend or a decreasing trend.

[0079] Periodic time series: used to reflect recurring fluctuations within a fixed period.

[0080] Residual time series: used to reflect random fluctuations in the original monitoring time series that cannot be explained by trends and periods.

[0081] refer to Figure 2 The original monitoring time series shown is assumed to contain anomalies within time period T. After seasonal decomposition, the original monitoring time series yields a trend time series 201, a residual time series 202, and a periodic time series 203. According to... Figure 2 It can be seen that the periodic time series corresponding to time period T is consistent with the periodic time series corresponding to other time periods outside of time period T. Therefore, it can be concluded that the normal first time series detection data in the original monitoring time series cannot be determined based on the periodic time series. In this embodiment, the periodic time series is removed, and the trend time series and / or residual time series are used to determine the first time series detection data, which can ensure the accuracy of obtaining the first time series detection data.

[0082] Because trend time series better reflect the long-term stationary characteristics of the original monitoring time series compared to residual time series, and remove the effects of seasonality and random fluctuations, the target component is, for example, a trend time series.

[0083] Step 1012: Divide the target component into multiple time-series data segments.

[0084] Because the target component may contain anomalous data segments, repeatedly dividing such segments into at least two adjacent time-series data fragments would render those fragments unusable as primary time-series detection data. Analyzing these fragments for normal data would waste computational resources. Therefore, in one example, adjacent time-series data fragments are designed to not overlap, preventing duplicate data between them. This avoids the problem of at least two adjacent time-series data fragments containing the same anomalous time-series data, and ensures that neither fragment can be used as normal data after analysis, thus preventing wasted computational resources.

[0085] For example, the target component is divided into multiple time-series data segments, including: segmenting the target component starting from the initial time of the target component using a first sliding window and a first step length. The window duration of the first sliding window is less than the duration of the target component, and the first step length is greater than or equal to the window duration of the first sliding window, thereby ensuring that adjacent time-series data segments do not overlap.

[0086] With the target component as Figure 2 Taking the trend time series 201 shown as an example, the window duration of the first sliding window is t, and the first step length is t. When segmenting the target component using the first sliding window and the first step length, refer to... Figure 3 The segmentation process shown begins by initializing the first sliding window at the initial time of the target component, obtaining the time series covered by the first sliding window, i.e., the time series data segment at time [0, t-1]. Then, the first sliding window slides for a length t and is located at time t of the target component, obtaining the time series covered by the first sliding window, i.e., the time series data segment at time [t, 2t-1]. The first sliding window continues to slide, and this process is repeated until the target component segmentation is completed. The length of the time series covered by the first sliding window after its last slide is less than or equal to t.

[0087] Step 1013: For each time series data segment in the multiple time series data segments, obtain the fourth statistical feature of the time series data segment, and determine the instability score of the time series data segment based on the fourth statistical feature.

[0088] In one example, the fourth statistical characteristic of a time series data segment includes the standard deviation, mean change, and standard score of the mean change. The standard deviation indicates the degree of volatility within the time series data segment. The mean change indicates short-term baseline drift between adjacent time series data segments. The standard score of the mean change indicates the degree to which the mean change deviates from a normal distribution.

[0089] Accordingly, the fourth statistical characteristic of the time series data segment is obtained, including: determining the standard deviation of the time series data segment; determining the change in mean between the mean of the time series data segment and the mean of an adjacent time series data segment (which may be the previous time series data segment or the next time series data segment), thereby obtaining the change in mean corresponding to the time series data segment; obtaining the mean and standard deviation of the change in mean corresponding to multiple time series data segments, and determining the standard score of the change in mean of the time series data segment based on the change in mean corresponding to the time series data segment, the mean and standard deviation of the change in mean.

[0090] Determining the standard deviation of a time series data segment includes: determining the mean of the data at each time point in the time series data segment; and taking the mean of the sum of the squares of the differences between the data at each time point and the mean to obtain the standard deviation of the time series data segment.

[0091] Taking the next time series data segment as the preceding time series data segment as an example, the mean change of the i-th time series data segment can be expressed as: |μ i -μ i-1 |. Where μ i μ represents the mean of the i-th time series data segment (i.e., the mean of the data at each time point in the i-th time series data segment); i-1 This represents the mean of the (i-1)th time series data segment (i.e., the time series data segment preceding the i-th time series data segment arranged chronologically), where i is an integer greater than 1.

[0092] The standard score of the change in mean can be obtained by calculating the absolute Z-score of the change in mean. The absolute Z-score is used to determine how much the current observation deviates from the historical / normal distribution (or the norm). For example, the standard score of the change in mean can be expressed as follows:

[0093] Z(x) = |(xx) - ) / σ x |

[0094] Where Z(x) represents the standard score of the change in mean for any time series data segment, and x represents the change in mean corresponding to that time series data segment. - σ represents the mean of the changes in the mean across multiple time series data segments. x It represents the standard deviation of the mean change corresponding to multiple time series data segments.

[0095] In other embodiments, the fourth statistical feature may include more or fewer statistical features, and this embodiment does not limit the types of the fourth statistical feature.

[0096] Instability scores are used to indicate the degree of fluctuation in time-series data segments.

[0097] Optionally, when the fourth statistical feature includes three feature types, the instability score of the time series data segment is determined based on the fourth statistical feature, including:

[0098] The fourth statistical feature of each feature type is normalized to obtain the normalized feature value; based on the preset weights corresponding to each feature type and the normalized feature value, the instability score is determined.

[0099] The normalization process is used to transform the fourth statistical feature of different feature types to the same numerical range so that it can be scored according to a unified numerical range to obtain an instability score. Optionally, the unified numerical range can be [0,1], or it can also be [0,100], etc. This embodiment does not limit the numerical range corresponding to the normalization process.

[0100] Taking the normalized numerical range of [0,1] as an example, the normalization of the fourth statistical feature for any feature type is expressed by the following formula:

[0101] X norm =(XX min ) / (X max -X min );

[0102] Among them, X norm This represents the normalized feature value of any time-series data segment. For example, the feature types of the fourth statistical feature include: the standard deviation, the change in mean, and the standard score of the change in mean for the time-series data segment. Then X... norm Let X represent any one of the normalized characteristic values ​​among the standard deviation, mean change, and standard score of mean change for a time series data segment. Let X represent the fourth statistical feature of the current feature type for this time series data segment; X min X represents the minimum value of the fourth statistical feature of this feature type across all time-series data segments; max This represents the maximum value of the fourth statistical feature of this feature type across all time series data segments.

[0103] The electronic device pre-stores preset weights for each feature type. The preset weights for each feature type are greater than or equal to 0 and less than or equal to 1, and the sum of the preset weights for all feature types is 1. Taking the fourth statistical feature of each time series data segment, which includes the standard deviation, mean change, and standard score of the mean change, as an example, the instability score (Sinstability) of this time series data segment can be expressed by the following formula:

[0104] Sinstability = w1 × F1norm +w2×F2 norm +w3×|Z(F2)| norm .

[0105] Among them, F1 norm The normalized eigenvalues ​​corresponding to the standard deviation of the time series data segment are represented by w1; the preset weights corresponding to the standard deviation of the time series data segment are represented by w2. norm |Z(F2)| represents the normalized feature value corresponding to the mean change of a time series data segment; w2 represents the preset weight corresponding to the mean change of a time series data segment; |Z(F2)| norm w3 represents the normalized feature value corresponding to the standard score of the mean change; w3 represents the preset weight corresponding to the standard score of the mean change. The preset weight can be set as needed; the larger the preset weight, the greater the contribution of the corresponding fourth statistical feature to the instability score.

[0106] Optionally, if the fourth statistical feature includes a feature type, the normalized feature value corresponding to the fourth statistical feature of that feature type can be determined as the instability score.

[0107] Step 1014: Based on the instability scores of multiple time series data segments, determine candidate time series data segments whose instability scores are within a preset score range from the multiple time series data segments.

[0108] Optionally, the instability score is positively correlated with the jitter level of the time-series data segment; that is, the higher the instability score, the higher the jitter level of the time-series data segment, indicating that the time-series data segment is more unstable and may contain anomalies. In this case, if the instability score is less than a preset instability threshold, it is determined that the instability score is within the preset score range, that is, the preset score range is the range of values ​​less than the instability threshold. Within this preset score range, the time-series data segment is relatively stable and can be used as a candidate time-series data segment for further analysis to determine whether it can be used as the first time series detection data.

[0109] The range of the instability threshold is consistent with the range of the instability score. For example, if the range of the instability score is [0,1], then the range of the instability threshold is also [0,1]. Or, if the range of the instability score is [0,100], then the range of the instability threshold is also [0,100].

[0110] In other embodiments, the instability score may also be negatively correlated with the jitter of the time-series data segment. In this case, if the instability score is greater than a preset instability threshold, the instability score is determined to be within the preset score range. This embodiment does not limit the way the correlation between the instability score and the jitter of the time-series data segment is expressed.

[0111] Step 1015: Determine the target time series data segment that meets the preset continuity requirement from all candidate time series data segments. The target time series data segment constitutes the first time series detection data.

[0112] When anomalies are included at both ends of a single time-series data segment, the overall instability score of the segment may not be significantly affected by the anomalies, potentially resulting in an instability score within a preset range. In such cases, directly using this time-series data segment as the first time-series detection data would incorrectly treat the segment containing the anomalies as a normal segment, leading to inaccurate anomaly detection. Therefore, in this embodiment, by using multiple candidate time-series data segments that meet continuity requirements as the first time-series detection data, multiple consistently stable candidate time-series data segments can be used as normal time-series data segments, improving the accuracy of selecting normal time-series data segments.

[0113] For example, the continuity requirement includes: the first total duration of all candidate time series data segments meets a preset first duration condition; the adjacent interval duration of adjacent candidate time series data segments is less than or equal to a preset interval duration; and the second total duration of adjacent candidate time series data segments with adjacent interval durations less than or equal to the preset interval duration meets a second duration condition.

[0114] Based on this, target time series data segments that meet the preset continuity requirements are determined from each candidate time series data segment, including:

[0115] Determine the first total duration of all candidate time series data segments; if the first total duration meets the preset first duration condition, determine the candidate time series data segments among all candidate time series data segments that satisfy the condition that the adjacent interval duration is less than or equal to the preset interval duration as consecutive candidate time series data segments; if the second total duration of multiple consecutive candidate time series data segments meets the preset second duration condition, determine the multiple consecutive candidate time series data segments as target time series data segments.

[0116] If there are multiple target time series data segments that meet the above conditions, multiple first time series detection data can be obtained, with each target time series data segment corresponding to one first time series detection data.

[0117] Optionally, the first duration condition is: the ratio of the first total duration to the total duration of the target component is greater than or equal to a preset first proportion threshold; or, the first duration condition is: the first total duration is less than or equal to a first total duration threshold, and the first total duration threshold is less than the sequence duration of the target component. Optionally, the first proportion threshold can be 0.5, 0.6, 0.4, etc., and the first total duration threshold can be 1 hour, 30 minutes, 3 hours, etc. This embodiment does not limit the values ​​of the first proportion threshold and the first total duration threshold.

[0118] The preset interval duration is less than or equal to n acquisition cycles, for example, n=1. In actual implementation, the preset interval duration and n can also be other values. This embodiment does not limit the values ​​of the preset interval duration and n. The acquisition cycle refers to the interval between two adjacent moments in the original monitoring time series, i.e., the acquisition frequency. For example, in a water environment monitoring scenario, the preset interval duration is less than or equal to 1 acquisition cycle. Here, the sampling cycle can be the interval between two adjacent moments in the water quality time series data acquired by the spectral monitoring device, for example, 1 minute, i.e., the acquisition frequency is 1 minute / time. Of course, the acquisition frequency of the spectral monitoring device can also be 3-60 minutes / time, preferably 5-30 minutes / time, particularly preferably 8-20 minutes / time, and most preferably 1-15 minutes / time.

[0119] Optionally, the second duration condition is: the ratio of the second total duration to the sequence duration of the target component is greater than or equal to a preset second proportion threshold; or, the second duration condition is: the second total duration is less than or equal to the second total duration threshold; Optionally, the second proportion threshold can be 0.6, 0.7, etc., and the second total duration threshold can be 1 hour, 2 hours, etc. This embodiment does not limit the values ​​of the second proportion threshold and the second total duration threshold.

[0120] Optionally, the continuity requirement may also include other content, such as: determining the multiple consecutive candidate time series data segments with the largest second total duration as the target time series data segment, etc. This embodiment does not limit the implementation method of the continuity requirement.

[0121] Optionally, if the target component includes a trend time series and a residual time series, the overlapping parts of the target time series data segments corresponding to the trend time series and the target time series data segments corresponding to the residual time series can be used as the first time series detection data respectively; or, the target time series data segments corresponding to the trend time series and the target time series data segments corresponding to the residual time series can both be used as the first time series detection data.

[0122] Given that the first time series detection data is determined based on the decomposition of the original monitoring time series, the second time series detection data to be used for anomaly detection is obtained, including:

[0123] The original time series data of the target object is decomposed and the components to be detected are extracted. If the duration of the component to be detected is greater than a preset duration threshold, the component to be detected is segmented to obtain multiple second time series detection data. Among them, the components to be detected include the trend time series to be detected and / or the residual time series to be detected.

[0124] Here, the original time series data to be detected refers to the time series directly collected by the sensor, for which the presence of anomalies has not yet been determined. In this embodiment, the decomposition method of the original time series data to be detected is the same as the decomposition method of the original monitoring time series. That is, if the original monitoring time series is decomposed using a seasonal decomposition method, then the original time series data to be detected is also decomposed using a seasonal decomposition method.

[0125] In addition, the type of the component to be detected is the same as the type of the target component. For example, if the target component is a trend time series, then the component to be detected is the trend time series to be detected; or if the target component is a trend time series and a residual time series, then the component to be detected is the trend time series to be detected and the residual time series to be detected.

[0126] Since the statistical features of the second time-series detection data are subsequently used for anomaly detection, to avoid the problem that the statistical features of the abnormal part might be weakened by the normal part due to the long duration of the component to be detected, thus failing to detect the anomaly, this embodiment segments the component to be detected into multiple second time-series detection data when the duration of the component to be detected exceeds a preset duration threshold. This avoids the problem of the statistical features of the abnormal part being weakened, leading to missed anomalies and ensuring the accuracy of anomaly detection.

[0127] Furthermore, if the data between two adjacent second-time series detection data sets does not overlap, there is a possibility that an anomaly might be assigned to exactly two adjacent sets—one part to the first set and the other to the second. In this case, due to the small proportion of the anomaly, it might be missed. Therefore, in one possible implementation, there is partial overlap between two adjacent second-time series detection data sets. This ensures that even if an anomaly is assigned to two adjacent sets in a single partition, it can be reassigned to the same set in subsequent partitions, thus preventing the anomaly from being missed.

[0128] For example, segmenting the component to be detected includes: segmenting the component to be detected starting from the initial time of the component using a second sliding window and a second step size. The window duration of the second sliding window is less than or equal to a preset duration threshold, and the second step size is less than the window duration of the second sliding window but greater than 0, thereby ensuring partial overlap between two adjacent second time series detection data.

[0129] For example: Reference Figure 4 If the components to be detected are segmented without overlap, anomaly 400 will be divided into two parts, belonging to the second time series detection data 401 and 402 respectively. In this case, the statistical characteristics of anomaly 400 will be weakened, potentially leading to missed anomaly detection. However, if the components to be detected are segmented with partial overlap, even if anomaly 400 is divided into two parts in one segmentation, in the next segmentation, anomaly 400 will belong entirely to the second time series detection data 403, thus reducing the possibility of missed anomaly detection.

[0130] Optionally, if the duration of the component to be detected is less than or equal to a preset duration threshold, the component to be detected can be directly used as the second time series detection data. The preset duration threshold can be 1 hour, 2 hours, etc., and this embodiment does not limit the value of the preset duration threshold.

[0131] In other embodiments, if the first time series detection data is not determined based on the decomposition of the original monitoring time series, such as the method of manually selecting the first time series detection data from the original monitoring time series as described above, then the second time series detection data may also not need to be decomposed from the original time series data to be detected. Instead, if the duration of the original time series data to be detected is greater than a preset duration threshold, the original time series data to be detected can be segmented into multiple second time series detection data. Alternatively, the original time series data to be detected can be directly used as the second time series detection data. This embodiment does not limit the method of obtaining the second time series detection data.

[0132] Step 102: Determine the first statistical feature corresponding to the first time series detection data and the second statistical feature corresponding to the second time series detection data; wherein the feature type of the first statistical feature is the same as the feature type of the second statistical feature.

[0133] Among them, the feature types of the first statistical feature and the second statistical feature (hereinafter referred to as the feature type of the first statistical feature or the feature type of the second statistical feature) are statistical features that can reflect the differences in the distribution of normal data and abnormal time series data.

[0134] In one example, the first and second statistical features have three feature types: mean, standard deviation, and slope. The calculation methods for the mean and standard deviation are detailed above and will not be repeated here. The slope can include the slope of the linear trend obtained by linearly fitting each data point in each set of time series detection data (including the first and second time series detection data).

[0135] In other embodiments, the feature type of the first statistical feature can also be other implementation methods, such as: the standardized slope obtained after standardizing the slope, etc. This embodiment does not limit the implementation method of the feature type of the first statistical feature.

[0136] Step 103: Based on the difference between the first statistical feature and the second statistical feature, determine whether there is an anomaly in the second time series detection data corresponding to the second statistical feature.

[0137] In this embodiment, anomaly detection is performed by the difference between a first statistical feature that reflects the overall distribution of normal data and a second statistical feature that reflects the overall distribution of the data to be tested. This allows for anomaly detection from the perspective of the overall distribution of time series data, which can improve the accuracy of anomaly detection compared to comparing a single data point with a threshold.

[0138] In one example, determining whether there is anomaly in the second time series detection data corresponding to the second statistical feature based on the difference between the first statistical feature and the second statistical feature includes: determining the similarity between the first statistical feature and the second statistical feature; if the similarity is less than or equal to a preset similarity threshold, then determining that there is anomaly in the second time series detection data corresponding to the second statistical feature; if the similarity is greater than the similarity threshold, then determining that there is no anomaly in the second time series detection data corresponding to the second statistical feature.

[0139] The greater the similarity, the more similar the first and second statistical features are; that is, the second statistical feature is closer to the data distribution of normal data, and the possibility of anomalies is lower. Optionally, the similarity can be determined based on Euclidean distance, Mahalanobis distance, or cosine similarity, etc. This embodiment does not limit the calculation method of similarity.

[0140] In another example, the second time series detection data includes multiple sets, each corresponding to a set of second statistical features (i.e., including the mean, standard deviation, and slope of the second time series detection data). Accordingly, based on the difference between the first and second statistical features, determining whether the second time series detection data corresponding to the second statistical feature is abnormal includes: clustering the first statistical feature and multiple sets of second statistical features to obtain at least one feature cluster; identifying a target second statistical feature belonging to a different feature cluster from the first statistical feature among the multiple sets of second statistical features; and determining the second time series detection data corresponding to the target second statistical feature as abnormal time series data.

[0141] Optionally, when there are multiple second time series detection data, each second time series detection data corresponds to a set of second statistical features. Each set of second statistical features may include features of various feature types, such as mean, standard deviation, and / or slope. In this case, if the second statistical feature corresponding to at least one of the multiple second time series detection data is the target second statistical feature, then the at least one second time series detection data is determined to be abnormal time series data. Other second time series detection data outside of this at least one set of second time series detection data are not abnormal time series data (i.e., they belong to normal data as described below). That is, among the various second statistical features, the second time series detection data corresponding to the second statistical feature that belongs to the same feature cluster as the first statistical feature is determined to be normal data, not abnormal time series data.

[0142] In other embodiments, there may be one second time series detection data. In this case, there are at least two first time series detection data. When the second statistical feature corresponding to the second time series detection data is the target second statistical feature, the second time series detection data is determined to be abnormal time series data. When the second statistical feature corresponding to the second time series detection data is not the target second statistical feature, the second time series detection data is determined to be normal data, not abnormal time series data.

[0143] Electronic devices use a clustering algorithm to cluster first and second statistical features. The clustering algorithm divides the first and second statistical features into k disjoint subsets (i.e., feature clusters) such that the data similarity within the same subset is maximized and the similarity between different subsets is minimized.

[0144] Optionally, the clustering algorithm can be the k-means algorithm, the hierarchical clustering (Agglomerative Nesting, AGNES) algorithm, etc. This embodiment does not limit the implementation of the clustering algorithm.

[0145] In one example, after standardizing the first and second statistical features, the differences between the standardized first and second statistical features are compared to determine whether there are anomalies in the second time series detection data corresponding to the second statistical feature. Data standardization eliminates the dimensional differences between the first and second statistical features. Furthermore, when new second time series detection data is acquired, the same statistical measures from the first time series detection data can be used to determine data distribution differences, saving equipment resources.

[0146] For example, the first time series detection data includes multiple data sets; correspondingly, data standardization is performed on the first and second statistical features, including: Z-score standardization with a mean of 0 and a variance of 1 for both the first and second statistical features. Specifically, the mean and standard deviation of the first statistical feature corresponding to each first time series detection data set in the multiple data sets are determined; based on the mean and standard deviation of the first statistical feature, the first statistical feature of each first time series detection data set is standardized to obtain multiple standardized first statistical features; based on the mean and standard deviation of the first statistical feature, the second statistical feature is standardized to obtain standardized second statistical features; the difference between the standardized first statistical feature and the standardized second statistical feature is compared to determine whether there are any anomalies in the second time series detection data corresponding to the second statistical feature.

[0147] Based on the mean and standard deviation of the first statistical feature, the first statistical feature of each first time series detection data is standardized, which can be expressed by the following formula:

[0148] X i,scaled =(X i -μ i,train_feature ) / σ i,train_feature ;

[0149] Among them, X i,scaled Let X represent the standardized first statistical feature of the i-th feature type of any first-time-series detection data. i μ represents the first statistical feature of the i-th feature type in any first time series detection data. i,train_feature σ represents the mean of the first statistical feature of the i-th feature type among multiple first-time-series detection data. i,train_feature Let represent the standard deviation of the first statistical feature of the i-th feature type among multiple first-time-series detection data. i is a positive integer.

[0150] The standardization of the second statistical characteristic based on the mean and standard deviation of the first statistical characteristic can be expressed by the following formula:

[0151] test_data_normalized j = (test_data j -μ j，train_feature ) / σ j，train_feature ;

[0152] Among them, test_data_normalized j Let test_data represent the standardized second statistical feature of the j-th feature type for any second time series detection data. j μ represents the second statistical feature of the j-th feature type of any second time series detection data. j，train_feature σ represents the mean of the first statistical feature of the j-th feature type among multiple first-time-series detection data. j，train_feature Let j represent the standard deviation of the first statistical feature of the j-th feature type among multiple first-time-series detection data. j is a positive integer.

[0153] After obtaining the standardized first statistical feature of at least one feature type corresponding to each first time series detection data, and the standardized second statistical feature of at least one feature type corresponding to each second time series detection data, clustering can be performed based on the standardized first statistical feature and the standardized second statistical feature corresponding to each first time series detection data in the manner described above to obtain abnormal time series data.

[0154] Step 104: If there is an anomaly in the second time series detection data corresponding to the second statistical feature, the second time series detection data is regarded as abnormal time series data. The anomaly type of the abnormal time series data is determined based on the first statistical feature and / or the third statistical feature corresponding to the abnormal time series data. The anomaly type includes baseline drift and / or abnormal data jitter. The first statistical feature includes mean, standard deviation and slope; and / or the third statistical feature includes statistical features obtained by statistically analyzing the data deviation between the data value at each time point in the abnormal time series data and the data value at the corresponding time point in the smoothed abnormal time series data.

[0155] The first statistical characteristic, which includes the mean, standard deviation, and slope, refers to the fact that it comprises three types of features: ① mean; ② standard deviation; and ③ slope. The first and third statistical characteristics can be the same or different.

[0156] The first statistical feature includes the mean, standard deviation, and slope; and / or, the third statistical feature includes statistical features obtained by statistically analyzing the data deviation between the data value at each time point in the outlier time series data and the corresponding data value at the time point in the smoothed outlier time series data. Here, "and / or" means that it may include only the content of the first statistical feature, only the content of the third statistical feature, or both the first and third statistical features simultaneously.

[0157] In this embodiment, when an anomaly is detected in the second time series detection data, the second time series detection data can also be used as abnormal time series data to further determine the anomaly type of the abnormal time series data.

[0158] Traditional anomaly classification methods are generally based on preset rules or machine learning, but these methods have low anomaly classification accuracy and are prone to misclassification of anomaly types.

[0159] Taking the water environment as the target of monitoring as an example, water environment monitoring is a core means of ensuring water resource security and ecological health, and its development has evolved from traditional chemical analysis to intelligent monitoring. In water environment monitoring scenarios, anomalies are mainly caused by aging monitoring equipment, environmental interference, or sudden pollution events. Typical anomaly types include baseline drift and data jitter.

[0160] Baseline drift refers to a slow, continuous, and non-periodic trend shift in the original time series data being analyzed. This shift causes the overall baseline level of the data (i.e., the "baseline") to gradually increase or decrease over time.

[0161] Abnormal data jitter refers to short-term, high-frequency, and irregular random fluctuations in the original time series data to be tested. This fluctuation causes data points to oscillate violently within a short period of time, deviating from the normal data trajectory.

[0162] As can be seen from the above, baseline drift and data jitter are types that can be determined based on the overall data distribution. In other embodiments, the anomaly types may also include other types that can be determined based on the overall data distribution, which will not be listed here.

[0163] In one example, when the first statistical feature includes the mean, standard deviation, and slope, determining the anomaly type of the abnormal time series data based on the first statistical feature and / or the third statistical feature of the abnormal time series data includes: determining the feature ratio between the third statistical feature corresponding to the abnormal time series data and the first statistical feature; the feature ratio includes the feature ratio corresponding to the mean, the feature ratio corresponding to the standard deviation, and the feature ratio corresponding to the slope; when the feature ratio exceeds a preset ratio range, determining that the anomaly type of the abnormal time series data includes baseline drift.

[0164] And / or, based on the data value at each time point in the abnormal time series data, the mean and standard deviation in the first statistical feature, determine the degree of mean deviation; based on the slope in the third statistical feature and the slope in the first statistical feature, determine the degree of slope deviation; based on the degree of mean deviation and the degree of slope deviation, determine the jitter score of the abnormal time series data; if the jitter score exceeds the normal jitter score range, determine the abnormal type of the abnormal time series data, including abnormal data jitter.

[0165] The first statistical feature is the same as the third statistical feature corresponding to the abnormal time series data.

[0166] Optionally, if the abnormal time series data is determined by clustering a first statistical feature and multiple sets of second statistical features, the feature ratio between the third statistical feature and the first statistical feature of the abnormal time series data can be: the ratio of the third statistical feature to the cluster center of the feature cluster to which the first statistical feature belongs; or, the ratio of the cluster center of the feature cluster to which the third statistical feature belongs to the cluster center of the feature cluster to which the first statistical feature belongs; or, the feature ratio between the average value of each statistical feature in the feature cluster to which the third statistical feature belongs and the average value of each statistical feature in the feature cluster to which the first statistical feature belongs. This embodiment does not limit the method of determining the feature ratio between the third statistical feature and the first statistical feature.

[0167] Alternatively, if the abnormal time series data is determined based on the similarity between the first and second statistical features, then the ratio of the third statistical feature of the abnormal time series data to the first statistical feature can be determined to obtain the feature ratio. For example, the preset ratio range can be [-6, 6] or [-8, 8], etc., and this embodiment does not limit the implementation method of the preset ratio range.

[0168] Optionally, the preset ratio ranges for different feature types may be the same or different. In this case, if the feature ratios for all feature types exceed the corresponding preset ratio ranges, the anomaly type of the abnormal time series data is determined to include baseline drift; or, if the feature ratios for a preset number of feature types exceed the preset ratio ranges, the anomaly type of the abnormal time series data is determined to include baseline drift. The preset number is greater than or equal to 1 and less than 3.

[0169] For example, if the preset quantity is 2, then when the feature ratio corresponding to at least two feature types among the mean, standard deviation, and slope exceeds the preset ratio range, the abnormal type of the abnormal time series data is determined to include baseline drift.

[0170] In one example, the degree of deviation from the mean is determined based on the data value at each time point in the outlier time series data, the mean, and the standard deviation in the first statistical feature, including:

[0171] Based on the mean and standard deviation in the first statistical feature, determine the normal mean and normal standard deviation for each normal data, which at least includes the first time series detection data;

[0172] The degree of deviation from the mean is obtained by dividing the difference between the data value at each time point in the abnormal time series data and the normal mean by the normal standard deviation.

[0173] For example, the degree of deviation from the mean is determined by the Z-Score value of the data value at each time point in the outlier time series data, specifically, it can be expressed by the following formula:

[0174] z i = (y i – μ 正常 ) / σ 正常 ;

[0175] Among them, z i This represents the data value y at time i in the abnormal time series data. i The degree of deviation from the mean; μ 正常 Indicates the normal mean; σ 正常 This represents the normal standard deviation.

[0176] In one example, the degree of slope deviation is determined based on the slope in the third statistical feature and the slope in the first statistical feature, including:

[0177] Based on the slope in the first statistical feature, determine the normal average slope of each normal data point;

[0178] The difference between the slope in the third statistical characteristic and the normal average slope is determined to obtain the degree of slope deviation.

[0179] Optionally, normal data may include not only the first time series detection data but also the second time series detection data excluding the abnormal time series data. Alternatively, normal data may only include the first time series detection data. Optionally, if the abnormal time series data is determined by clustering the first statistical feature and multiple sets of second statistical features, the normal mean, normal standard deviation, and normal average slope can be the mean, standard deviation, and average slope of the normal data corresponding to the cluster centers of the feature clusters to which the first statistical feature belongs; or, they can be the mean, standard deviation, and average slope of each set of normal data within the feature cluster to which the first statistical feature belongs; or, they can be the mean, standard deviation, and average slope of the first time series detection data corresponding to the first statistical feature. This embodiment does not limit the method for determining the normal mean, normal standard deviation, and normal average slope. Alternatively, regardless of whether the abnormal time series data is determined by clustering the first statistical feature and multiple groups of second statistical features, the normal mean, normal standard deviation, and normal average slope can be the mean, standard deviation, and average slope of each group of normal data. This embodiment does not limit the method of determining the normal mean, normal standard deviation, and normal average slope.

[0180] The normal average slope refers to the average of the changes in data values ​​(i.e., instantaneous slopes) between two adjacent time points in each set of normal data (i.e., including at least each first time series detection data). In this case, the slope in the first statistical feature includes the instantaneous slope and / or the average of the instantaneous slopes. Specifically, the normal average slope can be expressed as:

[0181]

[0182] Among them, y i - y i-1 This represents the data value y at time i in the normal data. i The instantaneous slope, that is, the instantaneous slope is y. i The data value y at time i-1 i-1 The difference; T represents the total amount of data in the normal data, that is, the normal data includes data values ​​at T time points.

[0183] The slope in the third statistical feature includes the instantaneous slope between the data value at each time step and the data value at the next adjacent time step. For example, taking the next adjacent time step as the previous time step, the instantaneous slope k between the data value at each time step and the data value at the previous time step... i It can be represented as: k i = (y i - y i-1 ); where k i This represents the data value y at time i in the abnormal time series data. i The instantaneous slope, y i-1This represents the data value of the previous time step in the i-th time step of the abnormal time series data.

[0184] Correspondingly, the degree of slope deviation (or the degree of slope abrupt change) can be expressed as: k Ci = |k i – k 正常 |;where, k Ci The instantaneous slope k represents the data value at time i. i The corresponding slope deviation; k 正常 | indicates the normal average slope.

[0185] Optionally, a jitter score for the abnormal time series data is determined based on the degree of deviation from the mean and the degree of deviation from the slope, including:

[0186] The slope deviation is standardized to obtain the standardized slope deviation; the mean deviation and the standardized slope deviation are summed to obtain the jitter score.

[0187] For example, the slope deviation is standardized to obtain the standardized slope deviation, including:

[0188] Determine the mean change between the instantaneous slope and the normal average slope at each time point in the normal data; determine the mean and standard deviation of the mean change; use the mean and standard deviation of the mean change to standardize the slope deviation, and obtain the standardized slope deviation.

[0189] The change in mean can be expressed as: |k Ci正常 –k 正常 |,k Ci正常 = y i - y i-1 k Ci正常 y represents the instantaneous slope at time i in the normal data. i y represents the data value at time i in the normal data. i-1 This represents the data value at time i-1; i is an integer greater than 1.

[0190] Mean change μ kC正常 It can be represented as: μ kC正常 = (Sum of changes in the mean) / Number of data points for the change in the mean.

[0191] The standard deviation σ of the change in mean kC正常 It can be represented as:

[0192] ;

[0193] Where T represents the amount of normal data. In this example, the standard deviation is used as the sample standard deviation for illustration. In actual implementation, the standard deviation can also be the population standard deviation. This embodiment does not limit the implementation method of the standard deviation.

[0194] The standardization of slope deviation using the mean and standard deviation of the change in mean can be expressed by the following formula: k czi = (k Ci -μ kC正常 ) / σ kC正常 ; where k Ci k represents the degree of slope deviation at time i in the abnormal time series data. czi This indicates the degree of deviation of the standardized slope at time i.

[0195] The jitter score can be expressed as: S i = | z i | + |k czi |;

[0196] Among them, S i z represents the jitter score corresponding to the data value at time i in the abnormal time series data. i k represents the degree of deviation of the mean of the data value at time i in an abnormal time series data. czi This indicates the degree of deviation of the standardized slope at time i.

[0197] In other embodiments, the jitter score can also be obtained by weighted summation of the deviation from the mean and the deviation from the standardized slope. This embodiment does not limit the calculation method of the jitter score.

[0198] Optionally, the electronic device is based on the mean μ of the mean change. kC正常 and standard deviation σ kC正常 Determine the jitter score threshold, for example: the jitter score threshold is: μ kC正常 +aσ kC正常 Where 'a' is a positive integer, such as 3 or 5. This embodiment does not limit the value of 'a'. If the jitter score is greater than the jitter score threshold, then the jitter score is determined to exceed the normal jitter score range. In other words, the jitter score range can refer to the score threshold.

[0199] Optionally, if the jitter score at a certain moment exceeds the normal jitter score range, other conditions can be combined to further determine whether the anomaly type at that moment includes abnormal data jitter. For example, other conditions include: the absolute value of the mean deviation of the data value at the i-th moment is greater than the mean deviation threshold, and the absolute value of the standardized slope deviation corresponding to the i-th moment is greater than the slope deviation threshold. The mean deviation threshold and the slope deviation threshold can be 2 or other values. This embodiment does not limit the values ​​of the mean deviation threshold and the slope deviation threshold.

[0200] For example, other conditions include: the jitter score at at least two consecutive moments exceeds the normal jitter score range, and the slope deviation is greater than the slope deviation threshold.

[0201] In another example, where the third statistical feature is a statistical feature obtained by statistically analyzing the data deviation between the data value at each time step in the anomalous time series data and the corresponding data value in the smoothed anomalous time series data, the anomalous type of the anomalous time series data is determined based on the third statistical feature of the anomalous time series data, including:

[0202] The abnormal time series data is smoothed to obtain smoothed abnormal time series data; the data deviation between the data value at each time step in the abnormal time series data and the corresponding data value at the smoothed abnormal time series data is determined; the data deviation at each time step is segmented to obtain multiple deviation segments; the mean and standard deviation of each deviation segment are determined; the dispersion threshold corresponding to each deviation segment is determined based on the mean and standard deviation of each deviation segment; if the data deviation in any deviation segment is greater than the dispersion threshold, the abnormal type of the abnormal time series data is determined, including data jitter.

[0203] In this embodiment, abnormal time series data is processed using data smoothing technology. The data deviation between the data value of each abnormal time series data point and the smoothed abnormal time series data is calculated, ensuring that the calculation results primarily reflect high-frequency jitter components and eliminating the influence of baseline drift on the judgment of abnormal data jitter. Furthermore, by analyzing the statistical distribution characteristics (i.e., mean and standard deviation) of multiple deviation segments, the distribution characteristics of the deviations are used as the basis for determining abnormal data jitter, thereby improving the accuracy of detection.

[0204] For example, data smoothing techniques include exponential smoothing techniques. Accordingly, outlier time series data is smoothed to obtain smoothed outlier time series data, which can be expressed by the following formula:

[0205] S t =m×Y t +(1-m)×St-1 ;

[0206] Among them, S t S represents the smoothed outlier time series data corresponding to the data value at time t in the outlier time series data. t-1 Y represents the smoothed outlier time series data corresponding to the data value of the time preceding time t (i.e., time t-1) in the outlier time series data, where t is a positive integer starting from 1; t S0 represents the data value at time t in the abnormal time series data. When t=1, S0 is a preset value, for example: S0 = Y1. In other embodiments, the value of S0 can also be other values. This embodiment does not limit the value of S0. m is a preset smoothing coefficient. The value range of m is (0,1), for example: m is a value in [0.2, 0.3]. In other embodiments, the value of m can also be other values. This embodiment does not limit the value of m.

[0207] For example, the data deviations corresponding to each time point are segmented to obtain multiple deviation segments. This includes: for the time series composed of data deviations corresponding to each time point, segmentation is performed starting from the initial time of the time series using a third sliding window and a third step size. The window duration of the third sliding window is less than the duration of the time series composed of data deviations, and the third step size is less than the window duration of the third sliding window but greater than 0, thereby ensuring partial overlap between adjacent groups of deviation segments; or, the third step size is greater than or equal to the window duration of the third sliding window, thereby ensuring that adjacent groups of deviation segments do not overlap. This embodiment does not limit the segmentation method.

[0208] For example, the dispersion threshold corresponding to each deviation segment is determined based on the mean and standard deviation of each deviation segment, which can be expressed as: μ±kσ;

[0209] Where μ represents the mean of each deviation segment; σ represents the standard deviation of each deviation segment; and k represents a preset coefficient greater than 1, which can be 2 or 3. This embodiment does not limit the value of k.

[0210] Optionally, the electronic device can determine the anomaly type of the abnormal time series data, including data jitter, when x consecutive data deviations in the deviation segment are greater than the dispersion threshold, where x is a preset positive integer.

[0211] Optionally, the electronic device may, if it is determined that the anomaly type of the abnormal time series data includes baseline drift, further determine whether the anomaly type includes data jitter; or, regardless of whether the anomaly type includes baseline drift, determine whether the anomaly type includes data jitter.

[0212] In the embodiments of this disclosure, alarm processing can be further performed based on the determined anomaly type (baseline drift and / or data jitter). For example, when the determined anomaly type is baseline drift, an alarm message related to "baseline drift risk" can be output; when the determined anomaly type is data jitter, an alarm message related to "data jitter anomaly" can be output; and when the determined anomaly types are both baseline drift and data jitter, an alarm message of "baseline drift and data jitter" can be output, so as to carry out targeted analysis and processing in subsequent water environment management stages.

[0213] In summary, the anomaly detection method provided in this embodiment acquires first time-series detection data of the target object in a normal state and second time-series detection data to be detected for anomalies; determines a first statistical feature of the first time-series detection data and a second statistical feature of the second time-series detection data; determines whether there is an anomaly in the second time-series detection data corresponding to the second statistical feature based on the difference between the first and second statistical features; if there is an anomaly in the second time-series detection data corresponding to the second statistical feature, the second time-series detection data is regarded as anomalous time-series data, and the anomaly type of the anomalous time-series data is determined based on the first statistical feature and / or the third statistical feature of the anomalous time-series data. On the one hand, since anomaly detection can be performed from the perspective of the overall data distribution, it can solve the problem of missed or false detections that easily occur when performing anomaly detection by comparing a single data point with a threshold, thus improving the accuracy of anomaly detection. On the other hand, since the first statistical feature of the first time-series detection data in a normal state is used as a reference for anomaly judgment, the accuracy of anomaly detection can be further improved. On the other hand, anomaly detection based on the first and second statistical features reduces computational load and improves anomaly identification efficiency compared to directly using the first and second time-series detection data. Simultaneously, it reduces the impact of noise in the first and second time-series detection data, further improving anomaly detection accuracy. Furthermore, identifying baseline drift and data jitter based on their statistical distribution characteristics solves the problem of traditional anomaly type identification methods failing to accurately distinguish between them, thus improving the accuracy of anomaly type identification.

[0214] Furthermore, by determining target time-series data segments that meet preset continuity requirements based on the original monitoring time series, the first time-series detection data can be obtained, eliminating the need for manual screening and improving anomaly detection efficiency. Simultaneously, it enables the use of high-quality first time-series data for anomaly detection, enhancing accuracy and robustness, thereby improving the ability to identify genuine anomalies.

[0215] In addition, by segmenting the component to be detected when its duration exceeds a preset duration threshold, multiple second time series detection data are obtained; this avoids the problem of weakened statistical features of abnormal parts leading to missed detection of abnormalities, and ensures the accuracy of abnormality detection.

[0216] In addition, by ensuring partial overlap between two adjacent second time series detection data during segmentation, it can be guaranteed that even if an anomaly is assigned to two adjacent second time series detection data in a certain segmentation, it can be assigned to the same second time series detection data in subsequent segments, thereby avoiding the problem of anomaly being missed and ensuring the accuracy of anomaly detection.

[0217] Furthermore, by processing anomalous time-series data using data smoothing techniques, the data deviation between each anomalous time-series data value and the smoothed anomalous time-series data is calculated. This ensures that the calculation results primarily reflect high-frequency jitter components, successfully eliminating the dominant influence of slow baseline drift on volatility measurement and removing the impact of baseline drift on the judgment of anomalous data jitter. Additionally, by analyzing the statistical distribution characteristics (i.e., mean and standard deviation) of multiple deviation segments rather than a single statistic (such as window standard deviation), the distribution characteristics of the deviations are used as the basis for judging anomalous data jitter, which improves the sensitivity and accuracy of detection.

[0218] In addition, determining the anomaly type of abnormal time series data based on the third statistical feature corresponding to the abnormal time series data can solve the problem of difficulty in distinguishing and accurately identifying abnormal jitter in baseline drift superimposed data, providing stronger technical support for applications such as in-depth equipment status assessment, accurate fault location, and risk level judgment.

[0219] In the field of water environment monitoring, water quality time series data are data sequences of the physicochemical properties of water bodies collected continuously in chronological order by sensors. These data exhibit distinct and complex characteristics. First, they show strong periodicity and seasonality, influenced by diurnal cycles, tides, seasons, and human activity cycles. Second, the data may contain long-term trends, such as slow increases or decreases due to environmental changes or continuous pollution. Furthermore, multiple correlations exist between different water quality parameters; changes in one parameter often trigger cascading effects on others. Finally, the data commonly contains noise and missing values ​​due to sensor malfunctions or environmental interference, and often includes abrupt abnormal fluctuations and persistent baseline drift caused by sudden pollution or equipment failure. These characteristics collectively constitute the challenges and key aspects of water quality data analysis. The application of statistical characteristics such as the mean, standard deviation, and slope of normal and abnormal data to detect baseline drift has significant advantages. The method is computationally simple, and by comparing the statistical characteristics of different time windows, it can clearly reveal the persistent shift in the overall data level. It effectively distinguishes changes in "state" and has strong resistance to short-term interference: By establishing a statistical benchmark for the "normal" state and conducting a holistic evaluation of the entire data window, it can robustly capture systematic and continuous changes without being disturbed by transient abnormal fluctuations. Simultaneously, different patterns of change in mean, standard deviation, and slope help identify potential pollution types, providing in-depth insights for environmental management. By employing exponential smoothing to detect abnormal fluctuations in water quality time series data, any sharp fluctuations will show a significant difference between the actual observed values ​​and the smoothed data, immediately identifying anomalies. This method effectively defines a dynamic "normal" range, thus accurately capturing instantaneous abrupt changes deviating from expected patterns. Furthermore, as a smoothing process, it effectively filters out random noise. Moreover, exponential smoothing naturally adapts to slow data changes, ensuring that even with slow background level drift, it can still effectively detect abnormal fluctuations relative to the "current" background level.

[0220] This disclosure also provides an anomaly detection device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method.

[0221] This disclosure also provides a non-volatile computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0222] This disclosure also provides a computer program product, including a computer program or a non-volatile computer-readable storage medium carrying the computer program, wherein the computer program, when executed by a processor, implements the steps of the above method.

[0223] Figure 5 This is a block diagram illustrating an anomaly detection device 1900 according to an exemplary embodiment. For example, device 1900 may be provided as a server or terminal device. (Refer to...) Figure 5 The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0224] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output interface 1958 (I / O interface). Device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM macOS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0225] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of the device 1900 to perform the above-described method.

[0226] Computer-readable storage media can be tangible devices capable of holding and storing programs / instructions used by instruction execution devices. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0227] The computer program (or computer-readable program instructions) described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage medium in the respective computing / processing device.

[0228] The computer program (or computer program instructions) used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions to implement various aspects of this disclosure.

[0229] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0230] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0231] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0232] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0233] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An anomaly detection method, characterized in that, The method includes: Acquire the first time-series detection data when the target object is in a normal state, and the second time-series detection data to be used for anomaly detection; Determine a first statistical feature corresponding to the first time series detection data and a second statistical feature corresponding to the second time series detection data; wherein the feature type of the first statistical feature is the same as the feature type of the second statistical feature; Based on the difference between the first statistical feature and the second statistical feature, determine whether there is an anomaly in the second time series detection data corresponding to the second statistical feature; In the event that the second time series detection data corresponding to the second statistical feature is abnormal, the second time series detection data is regarded as abnormal time series data. The abnormality type of the abnormal time series data is determined based on the first statistical feature and / or the third statistical feature corresponding to the abnormal time series data. The abnormality type includes baseline drift and / or abnormal data jitter. The first statistical feature includes mean, standard deviation and slope. The third statistical feature includes statistical features obtained by statistically analyzing the data deviation between the data value at each time step in the abnormal time series data and the data value at the corresponding time step in the smoothed abnormal time series data.

2. The method according to claim 1, characterized in that, When the first statistical feature includes mean, standard deviation, and slope, determining the anomaly type of the abnormal time series data based on the first statistical feature and / or the third statistical feature corresponding to the abnormal time series data includes: Determine the feature ratio between the third statistical feature corresponding to the abnormal time series data and the first statistical feature; the feature ratio includes the feature ratio corresponding to the mean, the feature ratio corresponding to the standard deviation, and the feature ratio corresponding to the slope; if the feature ratio exceeds a preset range, determine that the abnormality type of the abnormal time series data includes baseline drift; and / or, Based on the data value at each moment in the abnormal time series data, the mean and standard deviation in the first statistical feature, the degree of mean deviation is determined; based on the slope in the third statistical feature and the slope in the first statistical feature, the degree of slope deviation is determined; based on the degree of mean deviation and the degree of slope deviation, the jitter score of the abnormal time series data is determined; if the jitter score exceeds the normal jitter score range, the abnormal type of the abnormal time series data is determined to include abnormal data jitter.

3. The method according to claim 1, characterized in that, When the third statistical feature includes a statistical feature obtained by statistically analyzing the data deviation between the data value at each time step in the abnormal time series data and the data value at the corresponding time step in the smoothed abnormal time series data, determining the anomaly type of the abnormal time series data based on the first statistical feature and / or the third statistical feature corresponding to the abnormal time series data includes: The abnormal time series data is smoothed to obtain smoothed abnormal time series data; Determine the data deviation between the data value at each moment in the abnormal time series data and the data value at the corresponding moment in the smoothed abnormal time series data; The data deviations at each time point are segmented to obtain multiple deviation segments; Determine the mean and standard deviation of each of the plurality of deviation segments; The threshold for the degree of dispersion of each deviation segment is determined based on the mean and standard deviation of each deviation segment; If the data deviation in any deviation segment exceeds the dispersion threshold, the abnormality type of the abnormal time series data is determined to include abnormal data jitter.

4. The method according to claim 1, characterized in that, Obtain the first time-series detection data when the target object is in a normal state, including: The original monitoring time series of the target detection object is decomposed and the target component is extracted. The target component includes a trend time series and / or a residual time series. The target component is divided into multiple time-series data segments; For each of the plurality of time-series data segments, a fourth statistical feature of the time-series data segment is obtained, and an instability score of the time-series data segment is determined based on the fourth statistical feature. Based on the instability scores of the multiple time-series data segments, candidate time-series data segments whose instability scores fall within a preset score range are determined from the multiple time-series data segments; From all candidate time series data segments, a target time series data segment that meets the preset continuity requirement is determined, and the target time series data segment constitutes the first time series detection data.

5. The method according to claim 4, characterized in that, From all candidate time series data segments, target time series data segments that meet the preset continuity requirements are identified, including: Determine the first total duration of all candidate time-series data segments; If the first total duration meets the preset first duration condition, the candidate time series data segments among all candidate time series data segments that satisfy the condition that the adjacent interval duration is less than or equal to the preset interval duration are determined as continuous candidate time series data segments. If the second total duration of multiple consecutive candidate time series data segments meets a preset second duration condition, the multiple consecutive candidate time series data segments are determined as the target time series data segment.

6. The method according to claim 4, characterized in that, The fourth statistical feature includes three feature types. Accordingly, the instability score of the time series data segment is determined based on the fourth statistical feature, including: The fourth statistical feature of each feature type is normalized to obtain the corresponding normalized feature value; The instability score is determined based on the preset weights corresponding to each feature type and the normalized feature values.

7. The method according to claim 6, characterized in that, Obtaining the fourth statistical feature of the time-series data segment includes: Determine the standard deviation of the time-series data segment; The mean change between the mean of the time series data segment and the mean of an adjacent time series data segment is determined to obtain the mean change corresponding to the time series data segment; Determine the mean and standard deviation of the mean change corresponding to the plurality of time series data segments, and based on the mean change corresponding to the time series data segments, the mean and standard deviation of the mean change, determine the standard score of the mean change of the time series data segments; The three fourth statistical characteristics of the time series data segment include: the standard deviation of the time series data segment, the change in mean, and the standard score of the change in mean.

8. The method according to claim 1, characterized in that, Obtain the second time-series detection data to be used for anomaly detection, including: The original time series data of the target object to be detected is decomposed and the components to be detected are extracted; the components to be detected include the time series of the trend to be detected and / or the time series of the residual to be detected. If the duration of the component to be detected is greater than a preset duration threshold, the component to be detected is segmented to obtain multiple second time series detection data.

9. The method according to claim 1, characterized in that, The number of second time series detection data is multiple, and each second time series detection data corresponds to a set of second statistical features; Accordingly, based on the difference between the first statistical feature and the second statistical feature, it is determined whether there are anomalies in the second time series detection data corresponding to the second statistical feature, including: Cluster the first statistical feature and multiple sets of the second statistical features to obtain at least one feature cluster; Among multiple sets of second statistical features, a target second statistical feature belonging to a different feature cluster from the first statistical feature is determined, and the second time series detection data corresponding to the target second statistical feature is determined as abnormal time series data.

10. The method according to claim 1, characterized in that, The first time series detection data includes multiple data sets. Correspondingly, based on the difference between the first statistical feature and the second statistical feature, it is determined whether the second time series detection data corresponding to the second statistical feature is abnormal, including: Determine the mean and standard deviation of the first statistical feature corresponding to each first time series detection data in multiple first time series detection data; Based on the mean and standard deviation of the first statistical feature, the first statistical feature of each first time series detection data is standardized to obtain the standardized first statistical feature; Based on the mean and standard deviation of the first statistical feature, the second statistical feature is standardized to obtain the standardized second statistical feature; By comparing the differences between the standardized first statistical feature and the standardized second statistical feature, it is determined whether there are any anomalies in the second time series detection data corresponding to the second statistical feature.

11. An anomaly detection device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 10.

12. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 10.

Images