Internet of Things data anomaly detection method, device, equipment and medium

Abstract

The present invention discloses a method, device, equipment and medium for detecting anomalies in IoT data, the method comprising: obtaining IoT data reported by IoT devices, wherein the IoT data comprises time series data; obtaining a preset anomaly detection model, and inputting the time series data into the preset anomaly detection model so that the preset anomaly detection model performs anomaly detection on the time series data to output an anomaly detection result, wherein the preset anomaly detection model comprises multiple anomaly detection algorithms. The present invention can improve the accuracy of anomaly detection on time series data.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to a method, device, equipment and medium for detecting anomaly in Internet of Things data. Background Art

[0002] IoT devices generate a large amount of data during operation, so data reporting is one of the core links. Currently, there are mainly time data reporting and time series data reporting. Time data reporting only contains reporting timestamps, but does not contain specific numerical information. It is commonly seen in scenarios such as status checks and heartbeat packets. Time series data reporting contains not only timestamps, but also specific numerical values ​​at corresponding time points. It is commonly seen in scenarios such as sensor data and equipment operating parameters.

[0003] The data generated by IoT devices may contain abnormal data due to various reasons, so anomaly detection is required for the data reported by IoT. However, the accuracy of anomaly detection for reported time series data is currently low. Summary of the invention

[0004] The embodiments of the present invention provide a method, device, equipment and medium for detecting anomaly in data of the Internet of Things, aiming to solve the problem of low accuracy of current data anomaly detection methods.

[0005] In a first aspect, an embodiment of the present invention provides a method for detecting anomaly in IoT data, the method comprising:

[0006] Acquire IoT data reported by IoT devices, wherein the IoT data includes time series data;

[0007] A preset anomaly detection model is obtained, and the time series data is input into the preset anomaly detection model so that the preset anomaly detection model performs anomaly detection on the time series data to output an anomaly detection result, wherein the preset anomaly detection model includes multiple anomaly detection algorithms.

[0008] In a second aspect, an embodiment of the present invention further provides an IoT data anomaly detection device, the device comprising:

[0009] A first acquisition unit, configured to acquire IoT data reported by an IoT device, wherein the IoT data includes time series data;

[0010] The second acquisition unit is used to acquire a preset anomaly detection model, and input the time series data into the preset anomaly detection model so that the preset anomaly detection model performs anomaly detection on the time series data to output an anomaly detection result, wherein the preset anomaly detection model includes multiple anomaly detection algorithms.

[0011] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory and a processor connected to the memory, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0012] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program can implement the above method when executed by a processor.

[0013] The embodiment of the present invention provides a method, device, equipment and medium for detecting anomalies in Internet of Things data. The method includes: obtaining Internet of Things data reported by Internet of Things devices, wherein the Internet of Things data includes time series data; obtaining a preset anomaly detection model, and inputting the time series data into the preset anomaly detection model so that the preset anomaly detection model performs anomaly detection on the time series data to output an anomaly detection result, wherein the preset anomaly detection model includes multiple anomaly detection algorithms. The embodiment of the present invention can extract time series data from Internet of Things data when the Internet of Things device reports Internet of Things data, and obtain a preset anomaly detection model, the preset anomaly detection model includes multiple anomaly detection algorithms, and then input the time series data into the preset anomaly detection model, and detect the time series data through multiple anomaly detection algorithms, thereby obtaining anomaly detection results, the anomaly detection results include anomalies detected by different anomaly detection algorithms, avoid missing anomalies, so that the abnormal data in the time series data can be detected as much as possible, and finally improve the accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0015] Figure 1 It is a flow chart of the method for detecting anomaly in IoT data provided by an embodiment of the present invention;

[0016] Figure 2 is a schematic block diagram of an IoT data anomaly detection device provided by an embodiment of the present invention;

[0017] Figure 3 It is a schematic block diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, operations, elements, components and / or groups thereof.

[0020] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in the specification of the present invention and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0021] See also Figure 1 , Figure 1 1 is a flow chart of an IoT data anomaly detection method provided by an embodiment of the present invention. The IoT data anomaly detection method according to the embodiment of the present invention can be applied to a computer device to obtain time series data in IoT data and perform anomaly detection on the time series data. Figure 1 As shown, the method includes steps S100-S110.

[0022] S100, obtaining Internet of Things data reported by an Internet of Things device, wherein the Internet of Things data includes time series data.

[0023] In the embodiment of the present invention, the IoT device may be any device connected to the IoT platform, which may periodically report the generated IoT data to the IoT platform. For example, the physical network device may be an IoT sensor, which periodically reports the detected data to the IoT platform, or the IoT device may be a certain operating device, which periodically reports its own operating parameters to the IoT platform.

[0024] Depending on the different types of IoT devices, the data types included in IoT data are also different, for example, time data and time series data. Time series data is extracted from IoT data. Time series data refers to data containing a timestamp and a specific value at the corresponding time point. Taking IoT sensors as an example, IoT sensors can be temperature sensors, which are used to detect the temperature of the environment. The temperature sensor can detect the temperature once at a certain interval. For example, the temperature is detected every five minutes, and time series data related to time and temperature is generated every five minutes. The specific form can be xxxx year xx month xx day xx hour xx minute xx second, temperature xx℃. The temperature sensor reports the detected temperature to the IoT platform at intervals.

[0025] After obtaining the time series data, the time series data can be preprocessed, that is, the time series data can be cleaned to remove duplicate data and invalid data. In addition to simple data cleaning, the time series data can also be denoised and interpolated to improve the quality of the time series data.

[0026] S110, obtaining a preset anomaly detection model, and inputting the time series data into the preset anomaly detection model so that the preset anomaly detection model performs anomaly detection on the time series data to output an anomaly detection result, wherein the preset anomaly detection model includes multiple anomaly detection algorithms.

[0027] In an embodiment of the present invention, a preset anomaly detection model is used to perform anomaly detection on time series data, which may include multiple anomaly detection algorithms. For example, the preset anomaly detection model may include an isolation forest algorithm (IsolationForest), a local outlier factor (LOF) algorithm, a K-means clustering algorithm, and a long short-term memory network (LSTM) algorithm. The isolation forest algorithm separates data points by randomly selecting features and cutting points, and calculates the degree to which data points are isolated to identify anomalies. The local outlier factor algorithm detects anomalies based on local density differences, calculates the local density of each data point and its density difference with neighboring points, and the K-means clustering algorithm identifies anomalies through cluster analysis, calculates the distance from the data point to the center of the cluster to which it belongs, and points exceeding a preset threshold are considered abnormal. The long short-term memory network algorithm uses the LSTM model to predict time series data, calculates the difference between the predicted value and the actual value, and points exceeding the preset threshold are considered abnormal.

[0028] The time series data is input into the preset anomaly detection model, and the preset anomaly detection model performs anomaly detection on the time series data according to the configured anomaly detection algorithm, so as to identify anomalies in the time series data. For example, if the preset anomaly detection model is configured with the above four anomaly detection algorithms, then for the same time series data, the four anomaly detection algorithms will each give a detection result, that is, one time series data has four detection results. When the four detection results are all normal, it indicates that the time series data can pass the anomaly detection. When any one of the four detection results is abnormal, it indicates that the time series data has an anomaly, thereby improving the accuracy of the detection.

[0029] The following is an explanation using a specific embodiment. Assuming that a plurality of IoT sensors are provided in a smart factory to monitor the operating status of the equipment in the smart factory and each link in the production process, the IoT sensors provided will generate a large amount of time series data, including key indicators such as temperature, pressure, and flow, and these IoT sensors will regularly report the generated time series data. When the IoT sensor reports the IoT data, the time series data in the IoT data is obtained, for example, one of the time series data about temperature is obtained, and then the time series data is input into a preset anomaly detection model. The preset anomaly detection model detects the time series data, and then outputs an anomaly detection result with a timestamp, original data, and the results detected by different anomaly detection algorithms. When the results of all anomaly detection algorithms in the anomaly detection result are passed, it indicates that the time series data is normal data.

[0030] In some embodiments, such as this embodiment, the IoT data anomaly detection may further include the following steps:

[0031] A preset detection result is obtained, and the preset detection result and the abnormal detection result are combined into a first structured data set, wherein a format of the preset detection result is the same as a format of the abnormal detection result.

[0032] In an embodiment of the present invention, the preset detection result is a detection result obtained by detecting time series data by other methods, for example, by manual detection. The preset detection result may include a timestamp, original data, and a result, while the abnormal detection result includes a timestamp, original data, and the detection results of each abnormal detection algorithm. The preset detection result and the abnormal detection result are merged to obtain a first structured data set. The first structured data set includes information in the preset detection result and information in the abnormal detection result. It should be noted that the format of the preset detection result is the same as the format of the abnormal detection result, which is convenient for data merging.

[0033] Taking JSON data as an example, if the timestamp of an anomaly detection result is 2024-07-08T12:34:56Z, the data of the IoT sensor is [0.5, 0.3, 0.6], and the detection result of the isolation forest algorithm is true, it means that the data point is identified as an anomaly in the isolation forest algorithm, which may be because the data point is relatively isolated in the feature space; the detection result of the local anomaly factor algorithm is false, indicating that the data point is similar to the surrounding points in local density and is not identified as an anomaly; the detection result of the K-means clustering algorithm is true, and the distance from the data point to the center of its cluster exceeds the preset threshold, so it is identified as an anomaly; the detection result of the long short-term memory network algorithm is false, and the difference between the predicted value and the actual value of the LSTM model does not exceed the preset threshold, so the data point is not identified as an anomaly; the timestamp of the preset detection result is 2024-07-08T12:34:56Z, and the detection result is true, then the merged first structured data set is represented by JSON data as follows:

[0034] "timestamp": "2024-07-08T12:34:56Z",

[0035] "sensor_data": [0.5, 0.3, 0.6],

[0036] "isolation_forest_anomaly": true,

[0037] "lof_anomaly": false,

[0038] "kmeans_anomaly": true,

[0039] "lstm_anomaly": false

[0040] "human_labeled_anomaly": true

[0041] Among them, timestamp represents the timestamp, sensor_data represents the sensor data, isolation_forest_anomaly represents the detection result of the isolation forest algorithm, lof_anomaly represents the detection result of the local anomaly factor algorithm, kmeans_anomaly represents the detection result of the K-means clustering algorithm, lstm_anomaly represents the detection result of the long short-term memory network algorithm, and human_labeled_anomaly represents the result of manual detection, that is, the preset detection result.

[0042] In some embodiments, such as this embodiment, the IoT data anomaly detection may further include the following steps:

[0043] Converting the format of the first structured data set into a format that meets the requirements of the preset anomaly detection model to obtain a training set;

[0044] The training set is used as input data of the preset anomaly detection model and the preset detection result is used as output data of the preset anomaly detection model, and the preset anomaly detection model is trained to fine-tune the preset anomaly detection model.

[0045] In an embodiment of the present invention, the first structured data set can be used to train a preset anomaly detection model to improve the detection accuracy of the preset anomaly detection model. Before the first structured data set is used to train the preset anomaly detection model, the format of the first structured data set needs to be converted into a format that meets the requirements of the preset anomaly detection model, thereby obtaining a training set. The specific conversion method can be to convert text data into a numerical representation or a tag sequence to meet the requirements of the input layer of the preset detection model.

[0046] After obtaining the training set, the training set is input into the preset anomaly detection model, and the preset anomaly detection model is trained so that the anomaly detection result output by the preset anomaly detection model is close to the preset detection result. The training process may include the following steps:

[0047] Model initialization: load the preset anomaly detection model and set the initial parameters suitable for the current task;

[0048] Data loading: input the training set into the preset anomaly detection model;

[0049] Model training: The preset anomaly detection model is trained through the training set so that the output results of the preset anomaly detection model are close to the preset detection results.

[0050] In some embodiments, such as this embodiment, the IoT data anomaly detection may further include the following steps:

[0051] Evaluating the fine-tuned preset anomaly detection model to obtain an evaluation result;

[0052] The fine-tuned preset anomaly detection model is adjusted again according to the evaluation result.

[0053] In an embodiment of the present invention, after completing fine-tuning of the preset anomaly detection model, the preset anomaly detection model may be evaluated, and the preset anomaly detection model may be adjusted again according to the evaluation result to ensure the accuracy of the preset anomaly detection model.

[0054] Evaluation indicators may include: Accuracy, the proportion of correctly identified anomalies to the total data points; Recall, the proportion of correctly identified anomalies to actual anomalies; Precision, the proportion of correctly identified anomalies to those predicted by the model; F1 Score, the harmonic mean of accuracy and recall; Robustness: the consistent performance of the model on different data sets.

[0055] In some embodiments, such as this embodiment, the IoT data anomaly detection may further include the following steps:

[0056] Decomposing the time series data by a time series decomposition method to obtain decomposed data;

[0057] The decomposed data and the time series data are combined into a second structured data set.

[0058] In an embodiment of the present invention, the second structured data set is used to optimize the preset anomaly detection model, and the time series data can be the historical IoT data of the IoT device, that is, the time series data is extracted from the historical physical network data. Then the time series data is decomposed by the time series decomposition method to obtain the decomposed data. Specifically, the time series data can be decomposed into four parts: trend, seasonality, cyclical, and residual by the time series decomposition method, which is convenient for understanding the internal structure and change law of the time series data. Among them, the trend represents the long-term change direction of the data over time, and the trend component can be calculated by moving average method, exponential smoothing method or regression analysis method; seasonality represents the pattern of data recurring in a specific time period (such as one year, one month, etc.), which can be extracted by STL (Seasonal and Trend Decomposition using Loess) decomposition method; periodicity means that if there is periodic change in the data, it can be identified by frequency domain analysis methods such as fast Fourier transform (FFT); residual is the remaining part after subtracting the trend, seasonality and cyclical components from the original data, which is used to represent random fluctuations or measurement errors.

[0059] After obtaining the decomposed data, the decomposed data is merged with the original data to obtain a second structured data set. Taking JSON data as an example, the second structured data set can be expressed as:

[0060] "timestamp": "2024-07-08T12:34:56Z",

[0061] "original_data": 100.5,

[0062] "trend": 98.2,

[0063] "seasonal": 1.8,

[0064] "cyclical": 0.3,

[0065] "residual": 0.2

[0066] Among them, timestamp represents the timestamp, original_data represents the original data, trend represents the trend, seasonal represents the seasonality, cyclical represents the cyclicality, and residual represents the residual.

[0067] In some embodiments, such as this embodiment, the IoT data anomaly detection may further include the following steps:

[0068] Inputting the second structured data set into the preset anomaly detection model and fine-tuning the preset anomaly detection model through a time series cross-validation method;

[0069] The fine-tuned preset anomaly detection model is verified by the RMSE verification model and the MAE verification model to evaluate the prediction accuracy of the fine-tuned preset anomaly detection model.

[0070] In an embodiment of the present invention, the format of the second structured data set can be converted into a format that meets the requirements of the preset anomaly detection model, thereby obtaining a new training set, inputting the new training set into the preset anomaly detection model, and fine-tuning the preset anomaly detection model through a time series cross-validation method. The specific steps are as follows:

[0071] Dataset partitioning: Divide the new training set into multiple consecutive parts in chronological order, each part contains a continuous time period;

[0072] Initial training window selection: Determine an initial training window, which contains the earliest part of the time series data for preliminary training of the model;

[0073] Model training and evaluation: Use the data in the current training window to train the model, and make predictions and evaluations at one or more time points immediately following the training window.

[0074] Iterative training: gradually move the training window, add new data points, and repeat the training and evaluation process until the training window covers all historical data.

[0075] After completing the optimization of the preset anomaly detection model, the preset detection model is verified through the root mean square error (RMSE verification model) and the mean absolute error (MAE verification model) to evaluate its prediction accuracy and ensure that the preset anomaly detection model can accurately detect anomalies.

[0076] Figure 2 is a schematic block diagram of an IoT data anomaly detection device 200 provided by an embodiment of the present invention. Figure 2 As shown, corresponding to the above IoT data anomaly detection method, the present invention also provides an IoT data anomaly detection device 200. The IoT data anomaly detection device 200 includes a unit for executing the above IoT data anomaly detection method. Figure 2 The IoT data anomaly detection device 200 includes a first acquisition unit 201 and a second acquisition unit 202.

[0077] The first acquisition unit 201 is used to acquire the IoT data reported by the IoT device, wherein the IoT data includes time series data;

[0078] The second acquisition unit 202 is used to acquire a preset anomaly detection model, and input the time series data into the preset anomaly detection model so that the preset anomaly detection model performs anomaly detection on the time series data to output an anomaly detection result, wherein the preset anomaly detection model includes multiple anomaly detection algorithms.

[0079] In some embodiments, such as the present embodiment, the IoT data anomaly detection device 200 further includes a third acquisition unit.

[0080] The third acquisition unit is used to acquire a preset detection result, and merge the preset detection result and the abnormal detection result into a first structured data set, wherein the format of the preset detection result is the same as the format of the abnormal detection result.

[0081] In some embodiments, such as the present embodiment, the IoT data anomaly detection device 200 further includes a first conversion unit and a first training unit.

[0082] The first conversion unit is used to convert the format of the first structured data set into a format that meets the requirements of the preset anomaly detection model to obtain a training set;

[0083] The first training unit is used to use the training set as input data of the preset anomaly detection model and the preset detection result as output data of the preset anomaly detection model, to train the preset anomaly detection model so as to fine-tune the preset anomaly detection model.

[0084] In some embodiments, such as the present embodiment, the IoT data anomaly detection device 200 further includes a first evaluation unit and a first adjustment unit.

[0085] The first evaluation unit is used to evaluate the preset anomaly detection model after fine-tuning to obtain an evaluation result;

[0086] The first adjustment unit is used to adjust the preset anomaly detection model after fine-tuning again according to the evaluation result.

[0087] In some embodiments, such as the present embodiment, the IoT data anomaly detection device 200 further includes a first decomposition unit and a first merging unit.

[0088] The first decomposition unit is used to decompose the time series data by a time series decomposition method to obtain decomposed data;

[0089] The first merging unit is used to merge the decomposed data and the time series data into a second structured data set.

[0090] In some embodiments, such as the present embodiment, the IoT data anomaly detection device 200 further includes a second adjustment unit and a first verification unit.

[0091] The second adjustment unit is used to input the second structured data set into the preset anomaly detection model and fine-tune the preset anomaly detection model through a time series cross-validation method;

[0092] The first verification unit is used to verify the fine-tuned preset anomaly detection model through the RMSE verification model and the MAE verification model to evaluate the prediction accuracy of the fine-tuned preset anomaly detection model.

[0093] It should be noted that technicians in the relevant field can clearly understand that the specific implementation process of the above-mentioned Internet of Things data anomaly detection device and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and conciseness of the description, it will not be repeated here.

[0094] The above-mentioned IoT data anomaly detection device can be implemented in the form of a computer program. The computer program can be used in Figure 3 Runs on the computer device shown.

[0095] See also Figure 3 The computer device 300 includes a processor 302 , a memory and a network interface 305 connected via a system bus 301 , wherein the memory may include a non-volatile storage medium 303 and an internal memory 304 .

[0096] The non-volatile storage medium 303 can store an operating system 3031 and a computer program 3032. When the computer program 3032 is executed, the processor 302 can execute a method for detecting anomalies in IoT data.

[0097] The processor 302 is used to provide computing and control capabilities to support the operation of the entire computer device 300 .

[0098] The internal memory 304 provides an environment for the operation of the computer program 3032 in the non-volatile storage medium 303. When the computer program 3032 is executed by the processor 302, the processor 302 can execute a method for detecting anomalies in Internet of Things data.

[0099] The network interface 305 is used to communicate with other devices. Those skilled in the art will appreciate that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 300 to which the solution of the present application is applied. The specific computer device 300 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0100] It should be understood that in the embodiment of the present application, the processor 302 may be a central processing unit (CPU), and the processor 302 may also be other general-purpose processors, digital signal processors (FSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0101] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0102] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program. When the computer program is executed by a processor, any embodiment of the above-mentioned method for detecting anomalies in IoT data is implemented.

[0103] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, or any other computer-readable storage medium that can store program codes.

[0104] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0105] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0106] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0107] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device to execute all or part of the steps of the method described in each embodiment of the present invention.

[0108] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0109] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0110] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting anomaly in IoT data, characterized in that: The method comprises: Acquire IoT data reported by IoT devices, wherein the IoT data includes time series data; Obtaining a preset anomaly detection model, and inputting the time series data into the preset anomaly detection model so that the preset anomaly detection model performs anomaly detection on the time series data to output an anomaly detection result, wherein the preset anomaly detection model includes an isolation forest algorithm, a local anomaly factor algorithm, a K-means clustering algorithm, and a long short-term memory network algorithm, the anomaly detection result includes a plurality of detection results, and one anomaly detection algorithm corresponds to one detection result, and when all the detection results in the anomaly detection results are that there is no anomaly, the time series data passes the anomaly detection; Obtaining a preset detection result, and merging the preset detection result and the abnormal detection result into a first structured data set, wherein the format of the preset detection result is the same as the format of the abnormal detection result, and the preset detection result includes a timestamp, original data, and a manual detection result, and the abnormal detection result includes a timestamp, original data, and detection results detected by each abnormal detection algorithm; Converting the format of the first structured data set into a format that meets the requirements of the preset anomaly detection model to obtain a training set; Using the training set as input data of the preset anomaly detection model and the preset detection result as output data of the preset anomaly detection model, training the preset anomaly detection model to fine-tune the preset anomaly detection model; Decomposing the time series data by a time series decomposition method to obtain decomposed data; Merging the decomposed data and the time series data into a second structured data set, wherein the decomposed data includes trend data, seasonal data, periodic data, and residual data, and the time series data is historical IoT data of the IoT device; The second structured data set is input into the preset anomaly detection model and the preset anomaly detection model is fine-tuned by a time series cross-validation method.

2. The method according to claim 1, characterized in that The method further comprises: Evaluating the fine-tuned preset anomaly detection model to obtain an evaluation result; The fine-tuned preset anomaly detection model is adjusted again according to the evaluation result.

3. The method according to claim 1, characterized in that The method further comprises: The fine-tuned preset anomaly detection model is verified by the RMSE verification model and the MAE verification model to evaluate the prediction accuracy of the fine-tuned preset anomaly detection model.

4. An IoT data anomaly detection device, characterized in that: The device comprises: A first acquisition unit, configured to acquire IoT data reported by an IoT device, wherein the IoT data includes time series data; A second acquisition unit is used to acquire a preset anomaly detection model, and input the time series data into the preset anomaly detection model so that the preset anomaly detection model performs anomaly detection on the time series data to output an anomaly detection result, wherein the preset anomaly detection model includes an isolation forest algorithm, a local anomaly factor algorithm, a K-means clustering algorithm, and a long short-term memory network algorithm, the anomaly detection result includes a plurality of detection results, and one anomaly detection algorithm corresponds to one detection result, and when all the detection results in the anomaly detection results are that there is no anomaly, the time series data passes the anomaly detection; The third acquisition unit is used to acquire a preset detection result, and merge the preset detection result and the abnormal detection result into a first structured data set, wherein the format of the preset detection result is the same as the format of the abnormal detection result, and the preset detection result includes a timestamp, original data and a manual detection result, and the abnormal detection result includes a timestamp, original data and a detection result detected by each abnormal detection algorithm; A first conversion unit, configured to convert the format of the first structured data set into a format that meets the requirements of the preset anomaly detection model to obtain a training set; A first training unit, configured to use the training set as input data of the preset anomaly detection model and the preset detection result as output data of the preset anomaly detection model, to train the preset anomaly detection model so as to fine-tune the preset anomaly detection model; A first decomposition unit, configured to decompose the time series data by a time series decomposition method to obtain decomposed data; a first merging unit, configured to merge the decomposed data and the time series data into a second structured data set, wherein the decomposed data includes trend data, seasonal data, periodic data, and residual data, and the time series data is historical IoT data of the IoT device; The second adjustment unit is used to input the second structured data set into the preset anomaly detection model and fine-tune the preset anomaly detection model through a time series cross-validation method.

5. A computer device, characterized in that: The computer device comprises a memory and a processor connected to the memory; the memory is used to store a computer program; the processor is used to run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 3 can be implemented.

Images