Abnormal data identification method, system, computer and readable storage medium

Abstract

The present invention provides a method, system, computer, and readable storage medium for identifying abnormal data. The method includes collecting monitoring data at a preset collection interval, storing the monitoring data into a plurality of monitoring data groups according to a preset grouping time threshold; performing interruption judgment on the monitoring data in each of the monitoring data groups according to a preset interruption time threshold to obtain interrupted monitoring data and first non-interrupted monitoring data; performing window median correction on the interrupted monitoring data to obtain the second non-interrupted monitoring data; and identifying abnormal trends on the first non-interrupted monitoring data and the second non-interrupted monitoring data to complete the identification of abnormal data. The present invention realizes automated diagnosis and identification of data anomalies, has simple configuration, strong applicability, and greatly improved timeliness.

Description

Technical Field

[0001] The present invention belongs to the technical field of data identification, and specifically relates to an abnormal data identification method, system, device and readable storage medium. Background Art

[0002] With the development of the economy, monitoring equipment has been widely used in various fields. Now, in order to facilitate the unified management of monitoring data of monitoring equipment, a monitoring equipment management system is usually used in the background or cloud to achieve unified management of monitoring data. However, due to environmental factors and equipment factors, there may be some abnormal data in the monitoring data. Therefore, it is necessary to identify the abnormal data in order to analyze the monitoring equipment or the monitoring environment. However, in the existing technology, abnormal data is usually identified by manually tracking the monitoring data and identifying the abnormal data. However, the manual identification method is relatively inefficient, not intelligent, and has low timeliness. Summary of the Invention

[0003] In order to solve the above technical problems, the present invention provides an abnormal data identification method, system, device and readable storage medium, which are used to solve the technical problems existing in the prior art.

[0004] In a first aspect, the invention provides the following technical solution: a method for identifying abnormal data, the method comprising:

[0005] Collect monitoring data at preset collection intervals, and store the monitoring data into a number of monitoring data groups according to preset grouping time thresholds;

[0006] Performing interruption judgment on the monitoring data in each of the monitoring data groups according to a preset interruption time threshold to obtain interruption monitoring data and first non-interruption monitoring data;

[0007] Performing window median correction on the interruption monitoring data to obtain second non-interruption monitoring data;

[0008] Abnormal trend identification is performed on the first non-interruption monitoring data and the second non-interruption monitoring data to complete the identification of abnormal data.

[0009] Compared with the prior art, the beneficial effects of the present application are as follows: the present application obtains the monitoring data uploaded by the monitoring device and performs an interruption judgment on it, so that when abnormal trend identification is performed subsequently, the trend identification error is large due to data interruption, and the window median correction is performed after the interruption judgment. The interrupted monitoring data is corrected by the median selected by the window, so as to ensure that the second non-interrupted monitoring data after correction is closer to the original monitoring data of the interruption, and the authenticity of the second non-interrupted monitoring data after correction is higher. Then, abnormal trend identification is performed on the first non-interrupted monitoring data and the second non-interrupted monitoring data. After the abnormal trend is identified, the monitoring data is reversed according to the abnormal trend, and the monitoring data is used as abnormal data. Therefore, the present invention can realize automatic diagnosis and identification of monitoring data, can quickly and accurately identify abnormal data in the monitoring data, has simple configuration, strong applicability, and greatly improved timeliness.

[0010] Preferably, the step of collecting monitoring data at a preset collection interval and storing the monitoring data into a plurality of monitoring data groups according to a preset grouping time threshold comprises:

[0011] Collect monitoring data and current monitoring time according to preset collection intervals;

[0012] Arrange each of the monitoring data according to its corresponding current monitoring time;

[0013] The arranged monitoring data are divided according to a preset grouping time threshold and stored in each monitoring data group in sequence.

[0014] Preferably, the step of performing interruption judgment on the monitoring data in each monitoring data group according to a preset interruption time threshold to obtain interruption monitoring data and first non-interruption monitoring data includes:

[0015] Determining a collection time point of each monitoring data in the corresponding monitoring data group;

[0016] Determining a plurality of search ranges with the collection time point corresponding to each of the monitoring data as a starting point and a preset interruption time threshold as a search length, and searching for the interruption inspection monitoring data within each of the search ranges, wherein the preset interruption time threshold is greater than the preset collection interval;

[0017] If the interruption inspection monitoring data is found in each of the search ranges, the monitoring data corresponding to the search range will be used as the first non-interruption monitoring data; if the interruption inspection monitoring data is not found in each of the search ranges, the monitoring data corresponding to the search range will be used as the interruption monitoring data.

[0018] Preferably, the step of performing window median correction on the interruption monitoring data to obtain the second non-interruption monitoring data includes:

[0019] Selecting a plurality of correction windows before and after the collection time point corresponding to the interruption monitoring data, wherein the window length of each correction window increases in sequence;

[0020] Determining the number of monitoring data within all the correction windows, and selecting the median of all the monitoring data within the correction windows;

[0021] The mean of all the medians is calculated, and the interruption monitoring data is corrected by the mean of all the medians to obtain the second non-interruption monitoring data.

[0022] Preferably, the step of identifying abnormal trends of the first non-interrupted monitoring data and the second non-interrupted monitoring data to complete the identification of abnormal data includes:

[0023] Performing burr filtering on the first non-interruption monitoring data and the second non-interruption monitoring data, and storing the first non-interruption monitoring data and the second non-interruption monitoring data after the burr filtering into a trend recognition array;

[0024] Selecting a plurality of windows in the trend identification array, sliding down one data each time the window is selected, to obtain a plurality of sliding windows;

[0025] Selecting and calculating a sliding median and a sliding mean of the first non-interrupted monitoring data and the second non-interrupted monitoring data within the sliding window, and storing the sliding median and the sliding mean in a median array and a mean array, respectively;

[0026] Abnormal trend identification is performed based on the median array and the mean array to complete the identification of abnormal data.

[0027] Preferably, the step of performing abnormal trend identification based on the median array and the mean array to complete the identification of abnormal data includes:

[0028] Determine whether the difference between two adjacent sliding medians and two adjacent sliding means in the median array and the mean array is greater than a preset value and whether the extreme difference values ​​of the median array and the mean array are greater than a preset extreme difference value;

[0029] If the difference between two adjacent sliding medians and two adjacent sliding means in the median array and the mean array is greater than a preset value, and the extreme difference between the median array and the mean array is greater than a preset extreme difference value, then an abnormal trend exists in the trend identification array;

[0030] Determining the starting point and the ending point of the abnormal trend according to the trend identification array;

[0031] According to the starting point and the ending point, the position of the corresponding monitoring data group is located, and the monitoring data between the starting point and the ending point is identified as abnormal data in the monitoring data.

[0032] Preferably, the step of performing burr filtering on the first non-interruption monitoring data and the second non-interruption monitoring data, and storing the first non-interruption monitoring data and the second non-interruption monitoring data after burr filtering into a trend recognition array includes:

[0033] Determining collection nodes of the first non-interruption monitoring data and the second non-interruption monitoring data, and arranging the first non-interruption monitoring data and the second non-interruption monitoring data according to their collection nodes;

[0034] Determine whether a difference between the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to any one of the collection nodes and the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to a previous collection node adjacent to the collection node is greater than a preset filtering threshold;

[0035] If the difference between the first non-interrupt monitoring data or the second non-interrupt monitoring data corresponding to the collection node and the first non-interrupt monitoring data or the second non-interrupt monitoring data corresponding to the previous collection node adjacent to the collection node is greater than a preset filtering threshold, the first non-interrupt monitoring data or the second non-interrupt monitoring data corresponding to the collection node is filtered, and the filtered first non-interrupt monitoring data and the second non-interrupt monitoring data are stored in a trend identification array.

[0036] In a second aspect, the invention provides the following technical solution: an abnormal data identification system, the system comprising:

[0037] A collection module, configured to collect monitoring data at preset collection intervals and store the monitoring data into a plurality of monitoring data groups according to preset grouping time thresholds;

[0038] an interruption determination module, configured to perform interruption determination on the monitoring data in each of the monitoring data groups according to a preset interruption time threshold, so as to obtain interruption monitoring data and first non-interruption monitoring data;

[0039] a correction module, configured to perform window median correction on the interruption monitoring data to obtain second non-interruption monitoring data;

[0040] The identification module is used to identify abnormal trends of the first non-interruption monitoring data and the second non-interruption monitoring data to complete the identification of abnormal data.

[0041] In a third aspect, the invention provides the following technical solution: a computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned abnormal data identification method when executing the computer program.

[0042] In a fourth aspect, the invention provides the following technical solution: a readable storage medium having a computer program stored thereon, and the computer program implements the above-mentioned abnormal data identification method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A flowchart of the abnormal data identification method provided by the first embodiment of the present invention;

[0045] Figure 2 A detailed flow chart of step S1 in the abnormal data identification method provided in the first embodiment of the present invention;

[0046] Figure 3 A detailed flow chart of step S2 in the abnormal data identification method provided in the first embodiment of the present invention;

[0047] Figure 4 Detailed flow chart of step S3 in the abnormal data identification method provided by the first embodiment of the present invention;

[0048] Figure 5 A detailed flow chart of step S4 in the abnormal data identification method provided by the first embodiment of the present invention;

[0049] Figure 6 Detailed flowchart of step S41 in the abnormal data identification method provided by the first embodiment of the present invention;

[0050] Figure 7 A detailed flow chart of step S44 in the abnormal data identification method provided by the first embodiment of the present invention;

[0051] Figure 8 A structural block diagram of an abnormal data identification system provided by a second embodiment of the present invention;

[0052] Figure 9 A hardware structure block diagram of a computer provided in another embodiment of the present invention.

[0053] The embodiments of the present invention will be further described below with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the embodiments of the present invention, and should not be construed as limiting the present invention.

[0055] In the description of the embodiments of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention.

[0056] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0057] In the embodiments of the present invention, unless otherwise expressly specified or limited, the terms "installed," "connected," "connected," "fixed," etc. should be understood in a broad sense. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the embodiments of the present invention based on specific circumstances.

[0058] Example 1

[0059] like Figure 1 As shown, in a first embodiment of the present invention, the invention provides the following technical solution, a method for identifying abnormal data, the method comprising:

[0060] S1, collecting monitoring data at a preset collection interval, and storing the monitoring data into several monitoring data groups according to a preset grouping time threshold;

[0061] Specifically, in this step, the monitoring data monitored by the monitoring equipment will be sent to the cloud or platform according to the preset collection interval, so as to obtain a series of monitoring data collected at equal time intervals, where the collection interval is the time interval, generally 0.5S, 1S, 10S, 1min, etc. After obtaining the monitoring data, it will be stored in the corresponding monitoring data group to facilitate subsequent steps to analyze and process several monitoring data in the monitoring data group.

[0062] like Figure 2 As shown, step S1 includes:

[0063] S11, collecting monitoring data and current monitoring time according to the preset collection interval;

[0064] Specifically, after obtaining the corresponding monitoring data, since the monitoring data is data monitored by the monitoring device at different times, the current monitoring time corresponding to the monitoring data will be obtained at the same time as the monitoring data is collected;

[0065] S12, arranging the monitoring data according to their corresponding current monitoring time;

[0066] Specifically, if the monitoring data is collected directly, it will be directly presented on the platform. However, if the monitoring data is not arranged, the acquired monitoring data will be messy, and it is difficult to analyze the anomalies of the monitoring data, that is, to identify abnormal data. Therefore, according to the preset collection interval and the monitoring time of the retaining wall, a data set arranged in time order can be obtained;

[0067] S13, dividing the arranged monitoring data according to a preset grouping time threshold and storing them in each monitoring data group in sequence;

[0068] Specifically, in the specific monitoring data group stored, it can be divided according to the preset grouping time threshold. For example, 0.5h or 1h or 2h is the preset grouping time threshold, and the monitoring data corresponding to the preset grouping time threshold is stored in the monitoring data group, and the corresponding monitoring data group is also arranged according to the preset grouping time threshold for subsequent viewing and analysis.

[0069] S2. performing interruption judgment on the monitoring data in each of the monitoring data groups according to a preset interruption time threshold to obtain interruption monitoring data and first non-interruption monitoring data;

[0070] Specifically, due to the influence of various external factors, such as equipment damage, sudden environmental changes, etc., the monitoring equipment may be unable to obtain monitoring data at a certain point in time or within a certain period of time, which means that the monitoring data is interrupted at that point. Therefore, in this step, it is necessary to identify the interruption position of the monitoring data, and correspondingly identify the data where the interruption occurs as the interrupted monitoring data and the data where the interruption occurs as the first non-interrupted data.

[0071] like Figure 3 As shown, step S2 includes:

[0072] S21, determining the collection time point of each monitoring data in the corresponding monitoring data group;

[0073] Specifically, the purpose of this step is to obtain the time when the monitoring data of the monitoring device is collected, so as to facilitate the subsequent identification of the interruption location;

[0074] S22, determining a plurality of search ranges with the collection time point corresponding to each monitoring data as a starting point and a preset interruption time threshold as a search length, and searching for interruption inspection monitoring data within each search range, wherein the preset interruption time threshold is greater than the preset collection interval;

[0075] Specifically, in this step, in order to facilitate the identification of the interruption location, a search range is set, with the collection time point of each detection data as the starting point and the preset interruption time threshold as the search length, and a one-dimensional search range is determined based on this, so that the interruption location of the monitoring data can be determined;

[0076] It is worth mentioning that when determining the search range, it is necessary to ensure that the preset interruption time threshold is greater than the preset collection interval. If the monitoring data is interrupted, the interruption duration must be greater than the preset collection interval, so that the interruption location and interrupted data can be determined;

[0077] S23. If the interruption inspection monitoring data is found within each of the search ranges, the monitoring data corresponding to the search range is used as the first non-interruption monitoring data; if the interruption inspection monitoring data is not found within each of the search ranges, the monitoring data corresponding to the search range is used as the interruption monitoring data;

[0078] Among them, if the interruption inspection monitoring data is found within the search range, it means that the data before the monitoring data is normal data and there is no data interruption. At this time, the monitoring data corresponding to the search range established based on the monitoring data is used as the non-interruption monitoring data. If the interruption inspection monitoring data is not found within the search range, it means that the collection interval between the interruption inspection monitoring data and the monitoring data is greater than the preset collection interval, that is, there is a data interruption. At this time, the monitoring data corresponding to the search range established based on the monitoring data is used as the interruption monitoring data;

[0079] It is worth mentioning that the interruption inspection monitoring data is the previous monitoring data of the monitoring data. Therefore, in the process of searching for data within the search range, the search range can be determined by taking the collection time point of each monitoring data as the starting point and the preset interruption time threshold as the search length. After the collection time point of each monitoring data, the interruption inspection monitoring data is searched according to the search range, wherein the interruption inspection monitoring data is the next monitoring data of the monitoring data. Similarly, the search range can be determined by taking the monitoring data as the starting point and taking the preset interruption time threshold as the search length before and after it, and the remaining monitoring data can be searched within the search range.

[0080] S3, performing window median correction on the interruption monitoring data to obtain second non-interruption monitoring data;

[0081] Specifically, due to data interruption in the monitoring data, there is no monitoring data at the time point of the middle period. As a result, in the subsequent outlier identification process, the monitoring data before and after the data interruption may be regarded as abnormal data. Therefore, it is necessary to perform window median correction on the interrupted monitoring data and replace the original interrupted monitoring data to ensure the authenticity of the monitoring data.

[0082] like Figure 4 As shown, step S3 includes:

[0083] Step S31, selecting a plurality of correction windows before and after the collection time point corresponding to the interrupt monitoring data, wherein the window length of each correction window increases in sequence;

[0084] Specifically, the correction window is selected several times before and after the monitoring data is interrupted, and the window lengths of the correction windows selected several times are increased in sequence, so as to ensure the authenticity of the data. At the same time, when selecting the correction window, it is necessary to ensure that the correction window selected later should include the correction window selected previously, and the window length of the correction window with the smallest window length should be greater than the preset window length, and the preset window length should be at least 10 monitoring data. In this way, the authenticity of the data is reflected by increasing the number of samples;

[0085] Step S32: determining the number of monitoring data within all the correction windows, and selecting the median of all the monitoring data within the correction windows;

[0086] The median of the monitoring data of the selected correction window can represent the average data value within the correction window to a certain extent in a mathematical sense. Therefore, selecting the median can reduce the difficulty and complexity of data calculation to a certain extent.

[0087] Step S33: calculating the mean of all the medians, and correcting the interruption monitoring data by the mean of all the medians to obtain the second non-interruption monitoring data;

[0088] Specifically, by calculating the mean of all medians, we can obtain the monitoring data that should have been obtained at the point where the interruption occurred and the mean of the interrupted monitoring data. Therefore, the mean of all medians can represent the monitoring data at the point where the data is interrupted, and the mean is corrected to replace the corresponding interrupted monitoring data to obtain a new monitoring data group, which is basically the same as the monitoring data in the monitoring data group where no interruption occurred.

[0089] S4. Identifying abnormal trends of the first non-interrupted monitoring data and the second non-interrupted monitoring data to complete identification of abnormal data;

[0090] Specifically, by identifying abnormal trends in data, it is possible to determine that the monitoring data corresponding to the abnormal trends are abnormal.

[0091] like Figure 5 As shown, step S4 includes:

[0092] S41, performing burr filtering on the first non-interruption monitoring data and the second non-interruption monitoring data, and storing the first non-interruption monitoring data and the second non-interruption monitoring data after the burr filtering into a trend recognition array;

[0093] Specifically, the first non-interrupted monitoring data and the second non-interrupted monitoring data are subjected to burr filtering, and the obviously abnormal data therein can be preliminarily filtered out to avoid interference in the subsequent abnormal trend identification process;

[0094] like Figure 6 As shown, the step S41 includes:

[0095] S411: Determine collection nodes for the first non-interruption monitoring data and the second non-interruption monitoring data, and arrange the first non-interruption monitoring data and the second non-interruption monitoring data according to their collection nodes;

[0096] Specifically, the first non-interruption monitoring data and the second non-interruption monitoring data are arranged according to their collection nodes to facilitate subsequent deburring processing;

[0097] S412: Determine whether a difference between the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to any one of the collection nodes and the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to a previous collection node adjacent to the collection node is greater than a preset filtering threshold;

[0098] Specifically, this step is used to determine whether the difference between the first non-interruption monitoring data or the second non-interruption monitoring data collected by two adjacent collection nodes exceeds a preset filtering threshold;

[0099] S413: If a difference between the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to the collection node and the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to a previous collection node adjacent to the collection node is greater than a preset filtering threshold, filtering the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to the collection node, and storing the filtered first non-interrupted monitoring data and the filtered second non-interrupted monitoring data in a trend identification array;

[0100] Among them, only when the difference between the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to the collection node and the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to the previous collection node adjacent to the collection node is greater than the preset filtering threshold, it is necessary to filter the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to the collection node to filter out the data with the larger difference; and when the difference between the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to the collection node and the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to the previous collection node adjacent to the collection node is not greater than the preset filtering threshold, it means that the data between the first non-interrupted monitoring data or the second non-interrupted monitoring data is relatively accurate, and there is no need to perform data filtering;

[0101] Performing burr filtering on the first non-interruption monitoring data and the second non-interruption monitoring data, and storing the first non-interruption monitoring data and the second non-interruption monitoring data after the burr filtering into a trend recognition array;

[0102] S42, selecting a plurality of windows from the trend identification array, sliding down one data each time the window is selected to obtain a plurality of sliding windows;

[0103] Specifically, in this step, in the trend identification data, taking one of the first non-interrupted monitoring data or the second non-interrupted monitoring data as the starting point and the preset length as the window length, a sliding window is determined backward or forward, and taking the sliding window as the starting point, one first non-interrupted monitoring data or the second non-interrupted monitoring data is sequentially slid forward or backward to obtain another sliding window. This step is repeated to obtain a plurality of sliding windows, and the number of sliding windows is determined according to the preset length and the number of data in the trend identification data;

[0104] S43: Select and calculate the sliding median and sliding mean of the first non-interrupted monitoring data and the second non-interrupted monitoring data within the sliding window, and store the sliding median and the sliding mean in a median array and a mean array, respectively;

[0105] By calculating the sliding median and sliding mean of all sliding windows and representing the corresponding sliding windows with the sliding median and sliding mean, the amount of calculation is reduced while ensuring the accuracy of the data;

[0106] S44: Identify abnormal trends based on the median array and the mean array to complete the identification of abnormal data.

[0107] like Figure 7 As shown, further, the step S44 also includes:

[0108] S441, determining whether the difference between two adjacent sliding medians and two adjacent sliding means in the median array and the mean array is greater than a preset value, and whether the range values ​​of the median array and the mean array are greater than a preset range value;

[0109] S442: If the difference between two adjacent sliding medians and two adjacent sliding means in the median array and the mean array is greater than a preset value, and the range value of the median array and the mean array is greater than a preset range value, then an abnormal trend exists in the trend identification array;

[0110] Specifically, the difference between two adjacent sliding medians and two adjacent sliding means in the median array and the mean array is compared with a preset value, so as to reflect the slope of the curve drawn by the monitoring data. When the slope is greater than the preset value, it means that the corresponding first non-interrupted monitoring data and the second non-interrupted monitoring data have undergone a sudden change, and the mutated data can be regarded as abnormal data.

[0111] S443, determining the starting point and the ending point of the abnormal trend according to the trend identification array;

[0112] S444: Locate the position of the corresponding monitoring data group according to the starting point and the ending point, and identify the monitoring data between the starting point and the ending point as abnormal data in the monitoring data;

[0113] Specifically, the abnormal trend is also reflected as the slope of the curve drawn by the monitoring data. When the slope acts on the curve drawn by the monitoring data, there will be a trend starting point and a trend ending point. Only all monitoring data between the trend starting point and the trend ending point can be regarded as abnormal data. By substituting the trend starting point and the trend ending point into the monitoring data group and locating the position of the corresponding monitoring data group, the corresponding monitoring data group and the corresponding monitoring data can be determined as abnormal data.

[0114] The advantage of the first embodiment is that: the present application obtains the monitoring data uploaded by the monitoring device and makes an interruption judgment on the monitoring data, so that when abnormal trend identification is performed subsequently, the trend identification error is large due to data interruption, and the window median correction is performed after the interruption judgment. The interrupted monitoring data is corrected by the median selected by the window, so as to ensure that the second non-interrupted monitoring data after correction is closer to the original monitoring data of the interruption, and the authenticity of the second non-interrupted monitoring data after correction is higher. Then, abnormal trend identification is performed on the first non-interrupted monitoring data and the second non-interrupted monitoring data. After the abnormal trend is identified, the monitoring data is inferred based on the abnormal trend, and the monitoring data is used as abnormal data. Therefore, the present invention can realize automatic diagnosis and identification of monitoring data, can quickly and accurately identify abnormal data in the monitoring data, has simple configuration, strong applicability, and greatly improved timeliness.

[0115] Example 2

[0116] like Figure 8 As shown, a second embodiment of the present invention provides an abnormal data identification system, the system comprising:

[0117] Collection module 1, used to collect monitoring data at preset collection intervals and store the monitoring data into a number of monitoring data groups according to preset grouping time thresholds;

[0118] An interruption determination module 2 is configured to perform interruption determination on the monitoring data in each of the monitoring data groups according to a preset interruption time threshold, so as to obtain interruption monitoring data and first non-interruption monitoring data;

[0119] Correction module 3, used for performing window median correction on the interruption monitoring data to obtain second non-interruption monitoring data;

[0120] The identification module 4 is configured to identify abnormal trends of the first non-interruption monitoring data and the second non-interruption monitoring data to complete the identification of abnormal data.

[0121] Wherein, the acquisition module 1 includes:

[0122] The collection submodule is used to collect monitoring data and current monitoring time according to the preset collection interval;

[0123] An arrangement submodule, for arranging each monitoring data according to its corresponding current monitoring time;

[0124] The data storage module is used to divide the arranged monitoring data according to the preset grouping time threshold and store them in each monitoring data group in sequence.

[0125] The interruption judgment module 2 includes:

[0126] A determination submodule, configured to determine a collection time point of each monitoring data in the corresponding monitoring data group;

[0127] a search submodule, configured to determine a plurality of search ranges with the collection time point corresponding to each of the monitoring data as a starting point and a preset interruption time threshold as a search length, and search for the interruption inspection monitoring data within each of the search ranges, wherein the preset interruption time threshold is greater than the preset collection interval;

[0128] The data division submodule is used to use the monitoring data corresponding to the search range as the first non-interruption monitoring data if the interruption inspection monitoring data is found within each search range; and to use the monitoring data corresponding to the search range as the interruption monitoring data if the interruption inspection monitoring data is not found within each search range.

[0129] The correction module 3 includes:

[0130] A correction window establishment submodule is used to select a number of correction windows before and after the collection time point corresponding to the interrupt monitoring data, and the window length of each correction window increases in sequence;

[0131] a median determination module, configured to determine the number of monitoring data within all the correction windows and select the median of the monitoring data within all the correction windows;

[0132] The mean calculation module is used to calculate the mean of all the medians and correct the interruption monitoring data by the mean of all the medians to obtain the second non-interruption monitoring data.

[0133] The identification module 4 includes:

[0134] a burr filtering submodule, configured to perform burr filtering on the first non-interruption monitoring data and the second non-interruption monitoring data, and store the first non-interruption monitoring data and the second non-interruption monitoring data after burr filtering into a trend recognition array;

[0135] A sliding window establishment submodule is used to select a plurality of windows in the trend identification array, and slide down one data each time the window is selected to obtain a plurality of sliding windows;

[0136] a sliding median and mean determination submodule, configured to select and calculate the sliding median and sliding mean of the first non-interrupted monitoring data and the second non-interrupted monitoring data within the sliding window, and store the sliding median and the sliding mean in a median array and a mean array, respectively;

[0137] The identification submodule is used to identify abnormal trends based on the median array and the mean array to complete the identification of abnormal data.

[0138] Wherein, the identification submodule further includes:

[0139] A difference judgment unit is used to judge whether the difference between two adjacent sliding medians and two adjacent sliding means in the median array and the mean array is greater than a preset value and whether the extreme difference value of the median array and the mean array is greater than a preset extreme difference value;

[0140] an abnormal trend determining unit, configured to determine that an abnormal trend exists in the trend identification array if a difference between two adjacent sliding medians or two adjacent sliding means in the median array or the mean array is greater than a preset value, and an extreme difference between the median array and the mean array is greater than a preset extreme difference value;

[0141] a starting and ending point determination unit, configured to determine the starting point and the ending point of the abnormal trend according to the trend identification array;

[0142] The abnormal data identification unit is used to locate the position of the corresponding monitoring data group according to the starting point and the ending point, and identify the monitoring data between the starting point and the ending point as abnormal data in the monitoring data.

[0143] Wherein, the burr filtering submodule includes:

[0144] a node determining unit, configured to determine collection nodes of the first non-interruption monitoring data and the second non-interruption monitoring data, and arrange the first non-interruption monitoring data and the second non-interruption monitoring data according to their collection nodes;

[0145] a judging unit, configured to judge whether a difference between the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to any one of the collection nodes and the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to a previous collection node adjacent to the collection node is greater than a preset filtering threshold;

[0146] A data filtering unit is used to filter the first non-interrupt monitoring data or the second non-interrupt monitoring data corresponding to the collection node if the difference between the first non-interrupt monitoring data or the second non-interrupt monitoring data corresponding to the collection node and the first non-interrupt monitoring data or the second non-interrupt monitoring data corresponding to the previous collection node adjacent to the collection node is greater than a preset filtering threshold, and store the filtered first non-interrupt monitoring data and the filtered second non-interrupt monitoring data in a trend recognition array.

[0147] In other embodiments of the present invention, the embodiments of the present invention provide the following technical solution: a computer, comprising a memory 102, a processor 101, and a computer program stored on the memory 102 and executable on the processor 101; the processor 101 implements the above-described abnormal data identification method when executing the computer program.

[0148] Specifically, the processor 101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0149] Memory 102 may include a large-capacity memory for data or instructions. By way of example, and not limitation, memory 102 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 102 may include removable or non-removable (or fixed) media. Where appropriate, memory 102 may be internal or external to the data processing device. In certain embodiments, memory 102 is non-volatile memory. In certain embodiments, memory 102 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Where appropriate, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory (FPMDRAM), an extended data out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0150] The memory 102 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 101 .

[0151] The processor 101 implements the above abnormal data identification method by reading and executing computer program instructions stored in the memory 102.

[0152] In some embodiments, the computer may further include a communication interface 103 and a bus 100. Figure 9 As shown, the processor 101 , the memory 102 , and the communication interface 103 are connected via a bus 100 and communicate with each other.

[0153] The communication interface 103 is used to implement communication between the various modules, devices, units, and / or devices in the embodiments of the present application. The communication interface 103 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0154] Bus 100 includes hardware, software, or both, and couples components of a computer device to each other. Bus 100 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 100 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Bus 100 may include one or more buses, where appropriate. Although embodiments herein describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.

[0155] The computer can execute the abnormal data identification method of the present application based on the acquired abnormal data identification system, thereby realizing the identification of abnormal data.

[0156] In some further embodiments of the present invention, in combination with the above-mentioned abnormal data identification method, the embodiments of the present invention provide the following technical solutions: a readable storage medium having a computer program stored thereon, and the computer program implements the above-mentioned abnormal data identification method when executed by a processor.

[0157] Those skilled in the art will appreciate that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in conjunction with such instruction execution system, apparatus, or device. For purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device.

[0158] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0159] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the aforementioned embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following technologies known in the art may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0160] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0161] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for identifying abnormal data, characterized in that: The method comprises: Collect monitoring data at preset collection intervals, and store the monitoring data into a number of monitoring data groups according to preset grouping time thresholds; Performing interruption judgment on the monitoring data in each of the monitoring data groups according to a preset interruption time threshold to obtain interruption monitoring data and first non-interruption monitoring data; Performing window median correction on the interruption monitoring data to obtain second non-interruption monitoring data; Abnormal trend identification is performed on the first non-interruption monitoring data and the second non-interruption monitoring data to complete the identification of abnormal data.

2. The abnormal data identification method according to claim 1, characterized in that: The step of collecting monitoring data at a preset collection interval and storing the monitoring data into a plurality of monitoring data groups according to a preset grouping time threshold comprises: Collect monitoring data and current monitoring time according to the preset collection interval; Arrange each of the monitoring data according to its corresponding current monitoring time; The arranged monitoring data are divided according to a preset grouping time threshold and stored in each monitoring data group in sequence.

3. The abnormal data identification method according to claim 1, characterized in that: The step of performing interruption judgment on the monitoring data in each monitoring data group according to a preset interruption time threshold to obtain interruption monitoring data and first non-interruption monitoring data includes: Determining a collection time point of each monitoring data in the corresponding monitoring data group; Determining a plurality of search ranges with the collection time point corresponding to each of the monitoring data as a starting point and a preset interruption time threshold as a search length, and searching for the interruption inspection monitoring data within each of the search ranges, wherein the preset interruption time threshold is greater than the preset collection interval; If the interruption inspection monitoring data is found in each of the search ranges, the monitoring data corresponding to the search range will be used as the first non-interruption monitoring data; if the interruption inspection monitoring data is not found in each of the search ranges, the monitoring data corresponding to the search range will be used as the interruption monitoring data.

4. The abnormal data identification method according to claim 3, characterized in that: The step of performing window median correction on the interruption monitoring data to obtain the second non-interruption monitoring data includes: Selecting a plurality of correction windows before and after the collection time point corresponding to the interrupt monitoring data, wherein the window length of each correction window increases in sequence; Determining the number of monitoring data within all the correction windows, and selecting the median of all the monitoring data within the correction windows; The mean of all the medians is calculated, and the interruption monitoring data is corrected by the mean of all the medians to obtain the second non-interruption monitoring data.

5. The abnormal data identification method according to claim 1, characterized in that: The step of identifying abnormal trends of the first non-interrupted monitoring data and the second non-interrupted monitoring data to complete the identification of abnormal data includes: Performing burr filtering on the first non-interruption monitoring data and the second non-interruption monitoring data, and storing the first non-interruption monitoring data and the second non-interruption monitoring data after the burr filtering into a trend recognition array; Selecting a plurality of windows in the trend identification array, sliding down one data each time the window is selected, to obtain a plurality of sliding windows; Selecting and calculating a sliding median and a sliding mean of the first non-interrupted monitoring data and the second non-interrupted monitoring data within the sliding window, and storing the sliding median and the sliding mean in a median array and a mean array, respectively; Abnormal trend identification is performed based on the median array and the mean array to complete the identification of abnormal data.

6. The abnormal data identification method according to claim 5, characterized in that: The step of performing abnormal trend identification based on the median array and the mean array to complete the identification of abnormal data includes: Determine whether the difference between two adjacent sliding medians and two adjacent sliding means in the median array and the mean array is greater than a preset value and whether the extreme difference values ​​of the median array and the mean array are greater than a preset extreme difference value; If the difference between two adjacent sliding medians and two adjacent sliding means in the median array and the mean array is greater than a preset value, and the extreme difference between the median array and the mean array is greater than a preset extreme difference value, then an abnormal trend exists in the trend identification array; Determining the starting point and the ending point of the abnormal trend according to the trend identification array; According to the starting point and the ending point, the position of the corresponding monitoring data group is located, and the monitoring data between the starting point and the ending point is identified as abnormal data in the monitoring data.

7. The abnormal data identification method according to claim 5, characterized in that: The step of filtering the first non-interruption monitoring data and the second non-interruption monitoring data for burrs, and storing the first non-interruption monitoring data and the second non-interruption monitoring data after burr filtering in a trend recognition array comprises: Determining collection nodes of the first non-interruption monitoring data and the second non-interruption monitoring data, and arranging the first non-interruption monitoring data and the second non-interruption monitoring data according to their collection nodes; Determine whether a difference between the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to any one of the collection nodes and the first non-interrupted monitoring data or the second non-interrupted monitoring data corresponding to a previous collection node adjacent to the collection node is greater than a preset filtering threshold; If the difference between the first non-interrupt monitoring data or the second non-interrupt monitoring data corresponding to the collection node and the first non-interrupt monitoring data or the second non-interrupt monitoring data corresponding to the previous collection node adjacent to the collection node is greater than a preset filtering threshold, the first non-interrupt monitoring data or the second non-interrupt monitoring data corresponding to the collection node is filtered, and the filtered first non-interrupt monitoring data and the second non-interrupt monitoring data are stored in a trend identification array.

8. An abnormal data identification system, characterized in that: The system comprises: A collection module, configured to collect monitoring data at preset collection intervals and store the monitoring data into a plurality of monitoring data groups according to preset grouping time thresholds; an interruption determination module, configured to perform interruption determination on the monitoring data in each of the monitoring data groups according to a preset interruption time threshold, so as to obtain interruption monitoring data and first non-interruption monitoring data; a correction module, configured to perform window median correction on the interruption monitoring data to obtain second non-interruption monitoring data; The identification module is used to identify abnormal trends of the first non-interruption monitoring data and the second non-interruption monitoring data to complete the identification of abnormal data.

9. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the abnormal data identification method according to any one of claims 1 to 7 is implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the abnormal data identification method according to any one of claims 1 to 7 is implemented.

Images