A method for intelligent processing of anomalies in environmental sensing data for IoT edge services

Abstract

This invention proposes an intelligent method for handling anomalies in environmental sensing data for IoT edge services. First, a dynamic threshold detection method is used to monitor all spatiotemporal sequences collected by sensors, filtering out anomalous data. Anomaly scores are then used to eliminate data anomalies caused by environmental anomalies, and inherently anomalous data undergoes anomaly processing. The anomalous data is then passed through a CNN encoder and input into an improved Transformer model, where the sequences are learned along their spatiotemporal dimensions. Adaptive feedback is introduced, with the Transformer model providing correction values ​​for the anomalous data. The data transition rate is used to determine whether the Transformer model's handling of the current moment's anomalous data is appropriate. If anomalies still exist, those caused by inherent data anomalies have been corrected, while data caused by environmental anomalies undergoes enhancement processing. By combining current scene parameters, precise handling of anomalies in data collected by target sensor nodes is achieved.

Description

Technical Field

[0001] This invention relates to the field of data monitoring, specifically to a method for intelligently processing data anomalies in IoT edge service environments. Background Technology

[0002] With the continuous advancement of modern information technology, applications such as the Internet, the Internet of Things, and 5G have become widespread, resulting in massive amounts of data that are playing an increasingly important role. Spatiotemporal series data is an important structured form of massive data, and how to efficiently detect anomalies in various types of spatiotemporal series data is a crucial research topic.

[0003] In existing spatiotemporal sequence data anomaly detection algorithms, most of the difficulties encountered are related to the characteristics of spatiotemporal sequence data, such as data class imbalance, correlation between features, and time dependence between indicator values. In addition, building such a detection model is challenging because the time series data anomaly model not only needs to capture the spatiotemporal dependence of each spatiotemporal sequence, but also needs to consider the interrelationship between different features.

[0004] Summary of the Invention

[0005] The purpose of this invention is to accurately identify abnormal data from massive amounts of data, and to provide operators with information on the normal status of the monitored data.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for intelligent processing of anomalies in sensing data in IoT edge service environments, comprising the following steps:

[0007] S1. Collect IoT edge service environment perception and monitoring data. Using a trained LSTM model, take the spatiotemporal sequence data as input to obtain the corresponding predicted sequence data. Then, use the dynamic threshold detection method and based on the real sequence data to monitor the predicted sequence data and obtain abnormal sequence data.

[0008] S2. Using the outlier score calculation method, outlier sequence data caused by environmental factors are removed from the outlier sequence data, while outlier sequence data caused by non-environmental factors are retained.

[0009] S3. Input the abnormal sequence data into the CNN encoder to obtain the feature mapping corresponding to the abnormal data, and embed the mapped features into the label group to obtain the abnormal sequence data after embedding the label.

[0010] S4. Input the embedded labeled anomalous sequence data into the improved spatiotemporal Transformer encoder, and use the improved multi-head attention module to extract features and adjust parameters of the anomalous data spatiotemporal features; and introduce an adaptive feedback mechanism to correct the anomalous data to obtain the corrected sequence.

[0011] S5. Calculate the data transition rate after correction by the Transformer model based on the prior probability, and determine whether the data transition rate is greater than the preset data transition rate threshold. If it is, use the corrected sequence in S4 as the output result; otherwise, perform enhancement processing on the corrected sequence in S4 to obtain the enhanced sequence, and use the enhanced sequence as the output result.

[0012] Furthermore, in step S1 above, the dynamic threshold ε is calculated according to the following formula:

[0013]

[0014]

[0015]

[0016] in, It is the mean. It is the standard deviation. Represents the true sequence. This represents the predicted sequence, and e represents the general error. This indicates an exponentially weighted average. This represents the prediction error after smoothing over a certain period of time.

[0017] Furthermore, the aforementioned step S2 includes the following sub-steps:

[0018] S2.1 Calculate the error threshold for each time step using the following formula. :

[0019]

[0020]

[0021] in, This represents the normal sequence within this time period. This indicates the prediction error for the time periods before and after this time period.

[0022] S2.2 Calculate the anomaly score of the abnormal sequence according to the following formula. :

[0023] ,

[0024]

[0025] in, Indicates an abnormal sequence;

[0026] S2.3 Determine the degree of deviation of the preset abnormal sequence. Is it greater than If the result is yes, then the abnormal sequence data is considered to contain abnormal sequence data caused by environmental factors, and the abnormal sequence data caused by environmental factors is removed; if the result is no, then the original abnormal data sequence is retained.

[0027] Furthermore, in step S3 above, the sequence expression after the temporal Transformer encoder receives the embedding tags from the CNN encoder is:

[0028]

[0029] in, The output feature map is represented by i, which represents the time dimension, and j, which represents the spatial location dimension. This represents the additional prediction label for the j-th spatial location sequence; Embedding representing spatiotemporal location, This represents the first layer input of the time-transformer encoder.

[0030] Furthermore, the aforementioned step S4 includes the following sub-steps:

[0031] S4.1, Let the attention score matrix ,in, To adjust the parameters, , The matrices are the results of multiplying the embedding vector obtained after the input sequence passes through the encoding layer with a randomly initialized matrix. Let the square root of the first dimension of the matrix be represented, and the retention function be defined as follows:

[0032] ,

[0033] in, The adaptive parameters are used to determine whether the attention score is greater than the adaptive parameters through a retention function. If yes, then retain the information in the multi-head attention module score matrix; otherwise, do not retain the information in the multi-head attention module.

[0034] S4.2 Calculate the scores for each attention module according to the following formula, and provide the parameters. By comparing the scores of each self-attention module with the parameters Adaptive adjustments are made to determine the contribution value of each self-attention module, and this value is fed back to the scoring matrix to adjust the parameters. Make dynamic adjustments:

[0035]

[0036]

[0037] in, , , These are the initialized vector matrices, , , The matrices are the results of multiplying the embedding vector obtained after the input sequence passes through the encoding layer with a randomly initialized matrix.

[0038] After calculating the attention of each head, the results are fed into a fully connected layer, ultimately yielding the learning results of the multi-head self-attention module:

[0039]

[0040] in, The weights of each head before and after entering the fully connected layer are used to evaluate the contribution of each head to the final result, where M is the total number of heads.

[0041] S4.3. The output of the spatiotemporal Transformer encoder is established as follows:

[0042]

[0043]

[0044] in This represents the output of the l-th layer of the time-transformer encoder. Presentation layer normalization;

[0045] S4.4. The output of the temporal Transformer encoder is input into the spatial Transformer encoder to encode global information. The prediction information is reconstructed into feature maps after passing through the spatial Transformer encoder. The spatial decoder receives the feature maps and outputs a sequence of prediction vectors. As an initial correction value for abnormal data;

[0046] S4.5. Combine the corrected values ​​of the spatiotemporal Transformer model data anomalies with the normal data and the data anomalies caused by the environment into a complete sequence L:

[0047]

[0048] in This represents the data of sequence L at time n at spatial location j.

[0049] Furthermore, the aforementioned step S5 includes the following sub-steps:

[0050] S5.1 Define the data transfer rate using the following formula:

[0051]

[0052] ,

[0053] in, This represents the transfer rate from data at time t-1 to data at time t. Represents arbitrary data Transfer to data The probability, Represents any data at time t-1 The probability of occurrence Represents any data in the total sequence S at time t-1. Number of times it appears Represents any data in the total sequence S at time t. Number of times it appears;

[0054] S5.2 Compare the data transfer rate D with the threshold The size, when At that time, it was assumed that sequence L conformed to normal time series rules, therefore no correction was made. The correction result as the final output:

[0055] ,

[0056] ,

[0057] in, This represents the corrected data output by sequence L at time n at spatial position j.

[0058] Otherwise, augment the data using the following formula to create a sequence. For the final output:

[0059] ,

[0060] ,

[0061] .

[0062] The present invention adopts the above technical solution, and its beneficial effects compared with the prior art are as follows:

[0063] 1. Compared with the traditional Transformer model, the introduction of a retention function and adaptive parameters for adaptive feedback adjustment improves the efficiency and accuracy of Transformer model data processing;

[0064] 2. Improve the multi-head self-attention mechanism of the Transformer model, which has a fast learning speed for data and high detection performance, and can meet the requirements of both accuracy and speed.

[0065] 3. Compared with traditional data anomaly detection methods, this invention adopts multiple anomaly detection methods and specific processing methods for abnormal data, which effectively addresses problems such as scene data anomalies.

[0066] 4. Compared with existing methods that judge data accuracy through prior probability, this invention combines prior knowledge of the types of data collected and the relationships between data, and further examines and corrects abnormal data based on the data transition rate.

[0067] 5. This invention uses a recognition model based on spatiotemporal convolution and attention-based LSTM to acquire spatiotemporal sequence information and predict future spatiotemporal sequences. This reduces the redundancy of data processing while meeting the accuracy requirements of data processing. At the same time, it can predict future sequences, reducing the complexity of the entire process. Attached Figure Description

[0068] Figure 1 This is a flowchart of the present invention;

[0069] Figure 2 This is a schematic diagram of an improved Transformer;

[0070] Figure 3 This is a schematic diagram of a time-transformer encoder;

[0071] Figure 4 This is a schematic diagram of a spatial Transformer encoder. Detailed Implementation

[0072] To better understand the technical content of the present invention, specific embodiments are described below in conjunction with the accompanying drawings.

[0073] In this invention, various aspects of the invention are described with reference to the accompanying drawings, in which numerous illustrative embodiments are shown. Embodiments of the invention are not limited to those depicted in the drawings. It should be understood that the invention is implemented through any of the various concepts and embodiments described above, as well as the concepts and embodiments described in detail below, because the concepts and embodiments disclosed herein are not limited to any particular implementation. Furthermore, some aspects of the invention disclosed may be used alone or in any suitable combination with other aspects of the invention disclosed.

[0074] like Figure 1 As shown, the specific operation steps of this invention are as follows:

[0075] Step 1: Collect environmental perception and monitoring data for IoT edge services. Using a trained LSTM model, with spatiotemporal sequence data as input, obtain the corresponding predicted sequence data. Then, using a dynamic threshold detection method and based on real sequence data, monitor the predicted sequence data to obtain abnormal sequence data. Calculate the dynamic threshold ε using the following formula:

[0076] ,

[0077] ,

[0078] ,

[0079] in, It is the mean. It is the standard deviation. Represents the true sequence. This represents the predicted sequence, and e represents the general error. This indicates an exponentially weighted average. This represents the prediction error after smoothing over a certain period of time.

[0080] Step 2: Using the outlier score calculation method, remove outlier sequence data caused by environmental factors from the outlier sequence data, and retain outlier sequence data not caused by environmental factors;

[0081] Calculate the error threshold for each time step using the following formula. :

[0082]

[0083]

[0084] in, This represents the normal sequence within this time period. This indicates the prediction error for the time periods before and after this time period.

[0085] The anomaly score of an abnormal sequence is calculated using the following formula:

[0086] ,

[0087]

[0088] in, Indicates abnormal scores. Indicates an abnormal sequence;

[0089] Determine the degree of deviation of the preset abnormal sequence Is it greater than If the abnormal sequence data is considered to contain abnormal sequence data caused by environmental factors, then the abnormal sequence data caused by environmental factors will be removed; otherwise, the original abnormal data sequence will be retained.

[0090] Step 3: Input the anomalous sequence data into the CNN encoder to obtain the feature maps corresponding to the anomalous data, and embed the mapped features into the labeled group to obtain the number of anomalous sequences after embedding the labels. The temporal Transformer encoder accepts the sequence expression after embedding the labels from the CNN encoder as follows:

[0091]

[0092] in, The output feature map is represented by i, which represents the time dimension, and j, which represents the spatial location dimension. This represents the additional prediction label for the j-th spatial location sequence; Embedding representing spatiotemporal location, This represents the first layer input of the time-transformer encoder.

[0093] Step 4: Input the embedded labeled anomalous sequence data into the improved spatiotemporal Transformer encoder, and use the improved multi-head attention module to extract features and adjust parameters of the spatiotemporal features of the anomalous data; and introduce an adaptive feedback mechanism to correct the anomalous data to obtain the corrected sequence.

[0094] Step 5: Calculate the data transition rate after correction by the Transformer model based on the prior probability, and determine whether the data transition rate is greater than the preset data transition rate threshold. If it is, the corrected sequence in Step 4 is used as the output result; otherwise, the correction value of the abnormal data in Step 3 is enhanced to obtain the enhanced sequence, and the enhanced sequence is used as the output result.

[0095] like Figure 2 As shown, an improved multi-head attention module is used to extract features and adjust parameters for the spatiotemporal features of abnormal data. The process is as follows:

[0096] First, let the attention score matrix... ,in, , The matrices are the results of multiplying the embedding vector obtained after the input sequence passes through the encoding layer with a randomly initialized matrix. Generally, it represents the square root of the first dimension of a matrix, and the retention function is defined as follows:

[0097] ,

[0098] in, The adaptive parameters are used to determine whether the attention score is greater than the adaptive parameters through a retention function. If the information in the multi-head attention module score matrix is ​​positive, then the information in the multi-head attention module score matrix is ​​retained; otherwise, the information in the multi-head attention module score matrix is ​​not retained. The retention function compares the attention score with the adaptive parameters. The magnitude of the value determines whether information in the attention score matrix is ​​retained.

[0099] Then, the scores for each attention module are calculated according to the following formula, with parameters specified. By comparing the scores of each self-attention module with the parameters Adaptive adjustments are made to determine the contribution value of each self-attention module, and this value is fed back to the scoring matrix to adjust the parameters. Make dynamic adjustments:

[0100]

[0101]

[0102] in, , , This represents the initialized vector matrix. , , The matrices are obtained by multiplying the embedding vector (obtained after the input sequence passes through the encoding layer) with a randomly initialized matrix. After calculating the attention of each head, the matrices are input into the fully connected layer to finally obtain the learning results of the multi-head self-attention module.

[0103] ,

[0104] in, The weights of each head before and after entering the fully connected layer are used to evaluate the contribution of each head to the final result, where M is the total number of heads.

[0105] After such Figure 3The output of the time-transformer encoder shown can be described as follows:

[0106]

[0107]

[0108] The output of the time-transformer encoder contains prediction markers. It is a predictive representation of spatial information. Figure 3 The output of the intermediate-time Transformer encoder consists of N sets of sequence blocks.

[0109] like Figure 4 As shown, the output of the temporal Transformer encoder is first input into the spatial Transformer encoder to encode the global information. The predicted information is then reconstructed into a feature map after passing through the spatial Transformer encoder. The spatial decoder receives the feature map and outputs a sequence of predicted vectors. As an initial correction value for abnormal data;

[0110] Then, the corrected values ​​of the spatiotemporal Transformer model data anomalies, normal data, and data with anomalies caused by the environment are combined into a complete sequence L:

[0111]

[0112] in This represents the data of sequence L at time n at spatial location j.

[0113] Finally, the data transition rate after correction by the Transformer model is calculated based on the prior probability. Combined with a preset data transition rate threshold, it is determined whether the data transition rate is greater than the threshold. If it is, the corrected sequence is used as the output result; otherwise, the corrected values ​​of the abnormal data are augmented to obtain an augmented sequence, which is used as the output result. Specifically, the data transition rate is defined by the following formula:

[0114]

[0115] ,

[0116] in, This represents the transfer rate from data at time t-1 to data at time t. Represents arbitrary data Transfer to data The probability, Represents any data at time t-1 The probability of occurrence Represents any data in the total sequence S at time t-1. Number of times it appears Represents any data in the total sequence S at time t. Number of times it appears.

[0117] Compare the data transfer rate D with the threshold The size, when At that time, it was assumed that sequence L conformed to normal time series rules, therefore no correction was made. As the final output:

[0118] ,

[0119] ,

[0120] in, This represents the corrected data output by sequence L at time m at spatial location j.

[0121] Otherwise, augment the data using the following formula to create a sequence. For the final output:

[0122]

[0123]

[0124] .

[0125] While the present invention has been described above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A method for intelligent processing of anomalies in environmental sensing data for IoT edge services, characterized in that, Includes the following steps: S1. Collect IoT edge service environment perception and monitoring data. Using a trained LSTM model, take the spatiotemporal sequence data as input to obtain the corresponding predicted sequence data. Then, use the dynamic threshold detection method and based on the real sequence data to monitor the predicted sequence data and obtain abnormal sequence data. S2. Using the outlier score calculation method, remove outlier sequence data caused by environmental factors from the outlier sequence data, and retain outlier sequence data not caused by environmental factors; including the following sub-steps: S2.1 Calculate the error threshold for each time step using the following formula. : ， ， in, This represents the normal sequence within this time period. This indicates the prediction error for the time periods before and after this time period; S2.2 Calculate the anomaly score of the abnormal sequence according to the following formula. : ， ， in, Indicates an abnormal sequence; S2.3 Determine the degree of deviation of the preset abnormal sequence. Is it greater than If the result is yes, then the abnormal sequence data is considered to contain abnormal sequence data caused by environmental factors, and the abnormal sequence data caused by environmental factors is removed; if the result is no, then the original abnormal data sequence is retained. S3. Input the abnormal sequence data into the CNN encoder to obtain the feature mapping corresponding to the abnormal data, and embed the mapped features into the label group to obtain the abnormal sequence data after embedding the label. S4. Input the embedded labeled anomalous sequence data into the improved spatiotemporal Transformer encoder, and use the improved multi-head attention module to extract features and adjust parameters of the anomalous data spatiotemporal features; and introduce an adaptive feedback mechanism to correct the anomalous data to obtain the corrected sequence. S5. Calculate the data transition rate after correction by the Transformer model based on the prior probability, and determine whether the data transition rate is greater than the preset data transition rate threshold. If it is, use the corrected sequence in S4 as the output result; otherwise, perform enhancement processing on the corrected sequence in S4 to obtain the enhanced sequence, and use the enhanced sequence as the output result.

2. The intelligent processing method for abnormal data in IoT edge service environment perception according to claim 1, characterized in that, In step S1, the dynamic threshold ε is calculated using the following formula: ， ， ， in, It is the mean. It is the standard deviation. Represents the true sequence. This represents the predicted sequence, and e represents the general error. This indicates an exponentially weighted average. This represents the prediction error after smoothing over a certain period of time.

3. The intelligent processing method for abnormal data in IoT edge service environment perception according to claim 2, characterized in that, In step S3, the temporal Transformer encoder, after receiving the embedding tags from the CNN encoder, gives the following sequence expression: ， in, The output feature map is represented by i, which represents the time dimension, and j, which represents the spatial location dimension. This represents the additional prediction label for the j-th spatial location sequence; Embedding representing spatiotemporal location, This represents the first layer input of the time-transformer encoder.

4. The method for intelligent processing of spatiotemporal data anomalies for IoT edge service environment perception according to claim 3, characterized in that, Step S4 includes the following sub-steps: S4.1, Let the attention score matrix ,in, To adjust the parameters, , The matrices are the results of multiplying the embedding vector obtained after the input sequence passes through the encoding layer with a randomly initialized matrix. Let the square root of the first dimension of the matrix be represented, and the retention function be defined as follows: ， in, The adaptive parameters are used to determine whether the attention score is greater than the adaptive parameters through a retention function. If yes, then retain the information in the multi-head attention module score matrix; otherwise, do not retain the information in the multi-head attention module. S4.2 Calculate the scores for each attention module according to the following formula, and provide the parameters. By comparing the scores of each self-attention module with the parameters Adaptive adjustments are made to determine the contribution value of each self-attention module, and this value is fed back to the scoring matrix to adjust the parameters. Make dynamic adjustments: ， ， in, , , These are the initialized vector matrices, , , The matrices are the results of multiplying the embedding vector obtained after the input sequence passes through the encoding layer with a randomly initialized matrix. After calculating the attention of each head, the results are fed into a fully connected layer, ultimately yielding the learning results of the multi-head self-attention module: ， in, The weights of each head before and after entering the fully connected layer are used to evaluate the contribution of each head to the final result, where M is the total number of heads. S4.

3. The output of the spatiotemporal Transformer encoder is established as follows: ， ， in This represents the output of the l-th layer of the time-transformer encoder. Presentation layer normalization; S4.

4. The output of the temporal Transformer encoder is input into the spatial Transformer encoder to encode global information. The prediction information is reconstructed into feature maps after passing through the spatial Transformer encoder. The spatial decoder receives the feature maps and outputs a sequence of prediction vectors. As an initial correction value for abnormal data; S4.

5. Combine the corrected values ​​of the spatiotemporal Transformer model data anomalies with the normal data and the data anomalies caused by the environment into a complete sequence L: ， in This represents the data of sequence L at time n at spatial location j.

5. The intelligent processing method for abnormal data in IoT edge service environment perception according to claim 4, characterized in that, Step S5 includes the following sub-steps: S5.1 Define the data transfer rate using the following formula: ， ， in, This represents the transfer rate from data at time t-1 to data at time t. Represents arbitrary data Transfer to data The probability, Represents any data at time t-1 The probability of occurrence Represents any data in the total sequence S at time t-1. Number of times it appears Represents any data in the total sequence S at time t. Number of times it appears; S5.2 Compare the data transfer rate D with the threshold The size, when At that time, it was assumed that sequence L conformed to normal time series rules, therefore no correction was made. The correction result as the final output: ， ， in, This represents the corrected data output by sequence L at time n at spatial position j. Otherwise, augment the data using the following formula to create a sequence. For the final output: ， , 。

Images