Sensor data anomaly detection method and device, equipment and storage medium

Abstract

The invention relates to the technical field of sensor monitoring and early warning, in particular to a sensor data anomaly detection method and device, equipment and a storage medium, and the method comprises the steps: judging whether sensor data has periodicity or not; if yes, processing the sensor data based on a trained anomaly detection target model to obtain a reconstruction error; estimating a distribution parameter of preset probability distribution based on the sensor data, and calculating a data period of the sensor data; and judging whether the sensor data is abnormal or not based on the reconstruction error, the distribution parameter and the data period. According to the invention, the anomaly detection effect of the periodic sensor data can be improved.

Description

Technical Field

[0001] This application relates to the technical field of sensor monitoring and early warning, and particularly relates to a method, device, equipment and storage medium for detecting abnormal sensor data. Background Art

[0002] When the sensor has insufficient power supply, no GPS signal, insufficient flow card flow, etc., or when the sensor truly detects an abnormal situation, it will cause the sensor to detect abnormal sensor data. In order to facilitate the timely discovery of abnormal sensor data detected by the sensor, it is necessary to perform abnormal detection on the sensor data.

[0003] Currently, the method for detecting abnormal sensor data is as follows: reconstruct the sensor data to obtain reconstructed data, calculate the anomaly score between the sensor data and the reconstructed data, and judge whether the sensor data is abnormal according to the anomaly score and a preset threshold.

[0004] However, when the sensor data is periodic data, usually a time window is first used to segment the sensor data, and then the abnormal detection is performed on the segmented sensor data. When performing abnormal detection, only local features of the sensor data can be extracted, and the global information of the sensor data cannot be extracted. And when performing abnormal detection on periodic data, global information needs to be considered. Therefore, the existing technology has poor abnormal detection effect on periodic sensor data. Summary of the Invention

[0005] In order to facilitate improving the abnormal detection effect on periodic sensor data, this application provides a method, device, equipment and storage medium for detecting abnormal sensor data.

[0006] In a first aspect, this application provides a method for detecting abnormal sensor data, including:

[0007] Judge whether the sensor data is periodic;

[0008] If so, process the sensor data based on the trained anomaly detection target model to obtain a reconstruction error;

[0009] Estimate the distribution parameters of a preset probability distribution based on the sensor data, and calculate the data period of the sensor data;

[0010] Judge whether the sensor data is abnormal based on the reconstruction error, the distribution parameters, and the data period.

[0011] In a second aspect, this application provides a device for detecting abnormal sensor data, including:

[0012] A period judgment module, configured to judge whether the sensor data is periodic;

[0013] A reconstruction calculation module, configured to, if so, process the sensor data based on the anomaly detection target model that has completed training to obtain a reconstruction error;

[0014] A data processing module, configured to estimate distribution parameters of a preset probability distribution based on the sensor data, and calculate a data period of the sensor data;

[0015] An anomaly determination module, configured to determine whether the sensor data is abnormal based on the reconstruction error, the distribution parameters, and the data period.

[0016] In a third aspect, the present application provides a computer device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps in the above method are implemented.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method are implemented.

[0018] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0019] For the above sensor data anomaly detection method, device, equipment and storage medium, it is determined whether the sensor data has periodicity; if so, the sensor data is processed based on the anomaly detection target model that has completed training to obtain a reconstruction error; the distribution parameters of a preset probability distribution are estimated based on the sensor data, and the data period of the sensor data is calculated; it is determined whether the sensor data is abnormal based on the reconstruction error, the distribution parameters, and the data period. Through the above implementation, when performing anomaly detection on periodic sensor data, global information of the sensor can be extracted, thereby improving the anomaly detection effect on periodic sensor data.

[0020] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0022] Figure 1 It is an application environment diagram of a sensor data anomaly detection method in an embodiment of the present application;

[0023] Figure 2 It is a flowchart of a sensor data anomaly detection method provided in an embodiment of the present application;

[0024] Figure 3 It is a schematic structural diagram of a sensor data anomaly detection device provided in an embodiment of the present application;

[0025] Figure 4 It is a schematic structural diagram of a computer device provided in an embodiment of the present application;

[0026] Figure 5 It is an internal structural diagram of a computer-readable storage medium provided in an embodiment of the present application. Detailed implementation manners

[0027] In order to make the objectives, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure, and are not used to limit the present disclosure.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data used can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products or equipment.

[0029] In this article, the term "and / or" is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after.

[0030] To solve the above problems, an embodiment of the present disclosure provides a sensor data anomaly detection method, which can be applied to, for example Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed in the cloud or on other network servers. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0031] Embodiment 1

[0032] Figure 2 It is a flowchart of a method for detecting abnormal sensor data provided by Embodiment 1 of this application. Refer to Figure 2 , this method can be executed by the device that executes this method. The device can be implemented in software and / or hardware. The method includes:

[0033] S110. Determine whether the sensor data has periodicity.

[0034] Among them, sensors are widely used in various industrial scenarios to detect corresponding data in industrial scenarios. Sensor data is the corresponding data in the industrial scenarios detected by sensors. Exemplarily, in the industrial scenario of power supply, corresponding sensors are set to detect voltage data, flow data, and monitoring data in the power supply scenario, etc. It should be noted that sensor data may or may not have periodicity. Since the existing technology has poor detection effects on periodic sensor data, after receiving sensor data, it is necessary to first determine whether the sensor data has periodicity, so as to perform abnormal detection on periodic sensor data and non-periodic sensor data respectively later. In this embodiment, it is determined whether the sensor data has periodicity by performing a Fourier transform on the sensor data; by performing a Fourier transform on a set of sensor data, the frequency-domain waveform of the set of sensor data can be obtained. If a set of sensor data has periodicity, the frequency-domain waveform corresponding to the set of sensor data is a significant sine waveform.

[0035] Specifically, after receiving a set of sensor data sent by a group of sensors, perform a Fourier transform on the set of sensor data to obtain the frequency-domain waveform corresponding to the set of sensor data, and then determine whether there is a significant sine waveform in the frequency-domain waveform, so as to determine whether a corresponding set of sensor data has periodicity.

[0036] S120. If so, process the sensor data based on the anomaly detection target model that has completed training to obtain a reconstruction error.

[0037] Among them, the anomaly detection target model is an anomaly detection model that has completed training. The anomaly detection target model is used to process periodic sensor data to obtain the corresponding reconstructed data of the periodic sensor data, and further calculate the difference between the sensor data and the corresponding reconstructed data, so as to obtain the reconstruction error of the sensor data. Exemplarily, input the periodic sensor data into the anomaly detection target model that has completed training. The anomaly detection target model reconstructs the sensor data through the built-in encoder to obtain the corresponding reconstructed data of the sensor data. Further, the anomaly detection target model calculates the difference between the sensor data and the corresponding reconstructed data, and uses this difference as the reconstruction error of the corresponding sensing data.

[0038] Specifically, if it is determined that the sensor data has periodicity, input the sensor data into the anomaly detection target model that has completed training. The anomaly detection target model reconstructs the sensor data to obtain the corresponding reconstructed data of the sensor data. Further, the anomaly detection target model calculates the difference between the sensor data and the corresponding reconstructed data, and uses this difference as the reconstruction error of the corresponding sensing data.

[0039] It should be noted that if it is determined that the sensor data does not have periodicity, a 3-sigma model, an isolation forest model, etc. that have been pre-trained based on the historical sensor data can be used to process the sensor data to determine whether the corresponding sensor data is abnormal.

[0040] S130. Estimate the distribution parameters of the preset probability distribution based on the sensor data, and calculate the data period of the sensor data.

[0041] Among them, it should be noted that the reconstruction error obtained through the above steps can be used to determine whether the corresponding sensor data is abnormal. However, if a sensor data is a short-term accidental abnormal data, it will be determined that a group of sensor data adjacent to the sensing data is abnormal subsequently, which will still lead to poor anomaly detection effect. Therefore, it is necessary to further process the sensor data and the reconstruction error to improve the anomaly detection effect.

[0042] In this embodiment, the maximum likelihood estimation is specifically used to process a group of sensor data to estimate the distribution parameters of the preset probability distribution. The goal of the maximum likelihood estimation is to find a set of parameters that maximize the likelihood of observing the group of sensor data when these parameters are applied to a probability distribution, and record this set of parameters as the distribution parameters. In this embodiment, the preset probability distribution is specifically a normal distribution.

[0043] In addition, if a set of sensor data is periodic, then this set of sensor data has a corresponding data period; in this embodiment, the data period is obtained according to the significant sine wave corresponding to this set of periodic sensors. First, determine the frequency F corresponding to the peak of the sine wave, and then calculate the data period T according to this frequency, where T = 1 / F.

[0044] Specifically, a set of sensor data obtained by maximum likelihood estimation processing is used to estimate the parameters of the corresponding normal distribution to obtain distribution parameters; in addition, the frequency F corresponding to the peak of the significant sine wave of this set of sensor data is also calculated, and then the data period T is calculated according to this frequency, where T = 1 / F.

[0045] S140. Determine whether the sensor data is abnormal based on the reconstruction error, the distribution parameters, and the data period.

[0046] Among them, the reconstruction error, distribution parameters, and data period calculated according to the above steps can be used to calculate the abnormal score of the corresponding sensor data, and further, whether the corresponding sensor data is abnormal can be determined through this abnormal score.

[0047] Specifically, calculate the abnormal score of the corresponding sensor data based on the reconstruction error, distribution parameters, and the data period, and then determine whether the sensor data is abnormal according to this abnormal score.

[0048] It should be noted that in this embodiment, it is determined whether the sensor data is periodic; if so, the sensor data is processed by the trained anomaly detection target model to obtain a reconstruction error; the distribution parameters of the preset probability distribution are estimated based on the sensor data, and the data period of the sensor data is calculated; it is determined whether the sensor data is abnormal based on the reconstruction error, the distribution parameters, and the data period. Through the above implementation, when performing anomaly detection on periodic sensor data, the global information of the sensor can be extracted, thereby improving the anomaly detection effect on periodic sensor data.

[0049] Embodiment 2

[0050] Embodiment 2 of the present application provides a method for detecting abnormal sensor data, which optimizes the step of "training the anomaly detection target model" in Embodiment 1; it should be noted that for parts not detailed in this embodiment, the descriptions of other embodiments can be referred to. The method includes:

[0051] S210. Determine whether the sensor data is periodic.

[0052] S220. If so, process the sensor data by the trained anomaly detection target model to obtain a reconstruction error.

[0053] Among them, the steps of training the anomaly detection target model include:

[0054] A110. Window the historical sensor data to obtain a training set, and determine whether the training set has periodicity.

[0055] Among them, the historical sensor data is the sensor data recorded in history. Exemplarily, in the industrial scenario of power supply, the historical sensor data includes historical voltage data, historical flow data, historical monitoring data, etc.; the historical sensor data is time series data. In implementation, a time window is preset for the historical sensor data. The window length of the time window is W, and the corresponding step size of the time window is L. The numerical values of W and L are not limited in this embodiment; the historical sensor data can be windowed through the time window to obtain multiple groups of data. The time length of each group of data corresponds to the window length W, and each group of data is a training set. The training set is used to train the initial anomaly detection model to obtain the trained anomaly detection target model; it should be noted that the historical sensor data in the training set may or may not have periodicity, but the anomaly detection target model is for periodic sensor data. Therefore, it is necessary to screen out the training set with periodic historical sensor data inside. For this purpose, it is necessary to determine whether the historical sensor data in the training set has periodicity. The method for determining whether the historical sensor data in the training set has periodicity in this embodiment is the same as the method for determining whether the sensor data has periodicity described above, which will not be elaborated here.

[0056] Specifically, window the historical sensor data according to the preset time window to obtain multiple training sets, and then determine whether the historical sensor data in each training set has periodicity.

[0057] A120. If so, obtain the overall model loss based on the training set and the initial anomaly detection model.

[0058] Among them, the training set is used to train the initial anomaly detection model. The initial anomaly detection model is also the anomaly detection initial model. After the initial anomaly detection model is trained, it is the anomaly detection target model; in this embodiment, the anomaly detection target model adopts the Informer model. The Informer model is an improved model based on the Transformer model, mainly aiming at the problem of slow dot product calculation in the attention extraction process of the Transformer model. It incorporates the structure of ProbSparse-Attention to reduce the complexity of its dot product operation, thereby improving the calculation efficiency; at the same time, a CNN module is added inside its structure to enhance its feature expression ability, so as to facilitate improving the anomaly detection effect based on reconstruction; the overall model loss is also the model loss calculated after the initial anomaly detection model processes the training set.

[0059] Specifically, the training set is processed by the initial anomaly detection model to calculate the corresponding model loss, and this model loss is recorded as the overall model loss.

[0060] A130. Train the initial anomaly detection model based on the overall model loss to obtain the target anomaly detection model.

[0061] Among them, the overall model loss is used to iteratively optimize the model parameters of the anomaly detection model, so as to realize the training of the initial anomaly detection model.

[0062] Specifically, the model parameters of the anomaly detection model are iteratively optimized through the calculated overall model loss, so as to obtain the target anomaly detection model that has completed training.

[0063] S230. Estimate the distribution parameters of the preset probability distribution based on the sensor data, and calculate the data period of the sensor data.

[0064] S240. Judge whether the sensor data is abnormal based on the reconstruction error, the distribution parameters, and the data period.

[0065] Embodiment III

[0066] Embodiment III of the present application provides a method for detecting abnormal sensor data, which optimizes "obtaining the overall model loss based on the training set and the initial anomaly detection model" in Embodiment II; it should be noted that for parts not described in detail in this embodiment, reference can be made to the descriptions of other embodiments. This method includes:

[0067] S310. Judge whether the sensor data has periodicity.

[0068] S320. If so, process the sensor data based on the trained target anomaly detection model to obtain a reconstruction error.

[0069] Among them, the steps of training the target anomaly detection model include:

[0070] A110. Window the historical sensor data to obtain a training set, and judge whether the training set has periodicity.

[0071] A121. If so, obtain an unsupervised loss based on the unsupervised data in the training set and the initial anomaly detection model.

[0072] Among them, the sensor historical data in the training set includes unsupervised data and supervised data. Unsupervised data is sensor historical data without pre-set data labels, and supervised data is sensor historical data with pre-set data labels. Generally, the Informer model is used for anomaly detection of unsupervised data. In order to enable the Informer model to also perform anomaly detection on supervised data, it is necessary to optimize and adjust the model loss of the Informer model with the supervised data and unsupervised data in the training set, so as to obtain a new model that can perform anomaly detection on both supervised data and unsupervised data, and this new model is denoted as the AD_tranformer model. Since the AD_tranformer model can perform anomaly detection on both supervised data and unsupervised data, the AD_tranformer model can be applied to the anomaly detection problem for semi-supervised data. By processing the unsupervised data in the training set through the initial anomaly detection model, the corresponding model loss can be obtained, and this loss is denoted as the unsupervised loss Loss. informer , in this embodiment, the unsupervised loss Loss informer Specifically, the least squares loss is adopted.

[0073] Specifically, if it is determined that the sensor historical data in the periodicity has periodicity, the unsupervised data in the training set is processed through the initial anomaly detection model, so as to obtain the unsupervised loss Loss. informer .

[0074] A122. Obtain the supervised loss based on the supervised data in the training set and the initial anomaly detection model.

[0075] Among them, the initial anomaly detection model can also process the supervised data in the training set, so as to calculate the corresponding model loss, and this model loss is denoted as the supervised loss Loss. sup , in this embodiment, the supervised loss Loss sup Specifically, the cross-entropy loss is adopted. Exemplarily, the expression of the supervised loss Loss sup is as follows:

[0076]

[0077] Among them, y represents the data label of the supervised data, and y is usually encoded as 0 (negative class) or 1 (positive class); represents the probability that the supervised data is predicted as the positive class by the initial anomaly detection model.

[0078] Specifically, the supervised loss Loss is obtained by processing the supervised data in the training set through the initial anomaly detection model. sup .

[0079] A123. Obtain the overall model loss based on the unsupervised loss, the supervised loss, and the supervised preset coefficient.

[0080] Among them, the supervised preset coefficient is a coefficient preset for the supervised loss, and is used to jointly calculate the overall model loss Loss with the unsupervised loss and the supervised loss ad_transfomer , the overall model loss Loss ad_transfomer is used to optimize the model parameters of the initial anomaly detection model.

[0081] Specifically, perform corresponding calculations according to the unsupervised loss, the supervised loss, and the supervised preset coefficient, so as to obtain the overall model loss Loss ad_transfomer .

[0082] A130. Train the initial anomaly detection model based on the overall model loss to obtain the target anomaly detection model.

[0083] S330. Estimate the distribution parameters of the preset probability distribution based on the sensor data, and calculate the data period of the sensor data.

[0084] S340. Determine whether the sensor data is abnormal based on the reconstruction error, the distribution parameters, and the data period.

[0085] Embodiment 4

[0086] Embodiment 4 of the present application provides a method for detecting abnormal sensor data. This method optimizes "obtaining the overall model loss based on the unsupervised loss, the supervised loss, and the supervised preset coefficient" in Embodiment 3; it should be noted that for parts not detailed in this embodiment, reference can be made to the descriptions of other embodiments. This method includes:

[0087] S410. Determine whether the sensor data has periodicity.

[0088] S420. If so, process the sensor data based on the trained target anomaly detection model to obtain a reconstruction error.

[0089] Among them, the steps of training the target anomaly detection model include:

[0090] A110. Window the sensor historical data to obtain a training set, and determine whether the training set has periodicity.

[0091] A121. If so, obtain the unsupervised loss based on the unsupervised data in the training set and the initial anomaly detection model.

[0092] A122. Obtain the supervised loss based on the supervised data in the training set and the initial anomaly detection model.

[0093] A123. Calculate the sum of the product of the supervised preset coefficient and the supervised loss and the unsupervised loss to obtain the overall model loss.

[0094] Wherein, the overall model loss Loss ad_transfomer is calculated by the formula: Loss ad_transfomer = Loss informer + α * Loss sup ;

[0095] Wherein, the Loss ad_transfomer is the overall model loss, the Loss informer is the unsupervised loss, the α is the supervised preset coefficient, and the Loss sup is the supervised loss.

[0096] Specifically, calculate the product α * Loss sup of the supervised preset coefficient α and the supervised loss Loss sup , then calculate the sum Loss sup of this product α * Loss informer and the unsupervised loss Loss informer + α * Loss sup to obtain the overall model loss Loss ad_transfomer .

[0097] A130. Train the initial anomaly detection model based on the overall model loss to obtain the target anomaly detection model.

[0098] S430. Estimate the distribution parameters of the preset probability distribution based on the sensor data and calculate the data period of the sensor data.

[0099] S440. Determine whether the sensor data is abnormal based on the reconstruction error, the distribution parameters, and the data period.

[0100] Embodiment Five

[0101] Embodiment Five of the present application provides a method for detecting abnormal sensor data, which optimizes "determining whether the sensor data is abnormal based on the reconstruction error, the distribution parameters, and the data period" in Embodiment Four; it should be noted that for parts not detailed in this embodiment, reference can be made to the descriptions of other embodiments. The method includes:

[0102] S510. Determine whether the sensor data has periodicity.

[0103] S520. If so, process the sensor data based on the trained target anomaly detection model to obtain a reconstruction error.

[0104] S530. Estimate the distribution parameters of the preset probability distribution based on the sensor data, and calculate the data period of the sensor data.

[0105] S541. Process the reconstruction error, the distribution parameters, and the data period based on a preset abnormal score calculation formula to obtain an abnormal score.

[0106] In one embodiment, the abnormal score calculation formula is: a (i) =(e (i) -μ) T ∑ -1 (e (i) -μ);

[0107] Wherein, the α (i) is the abnormal score of the i-th group of sensor data, the e (i) is the reconstruction error of the i-th group of sensor data, the μ and Σ are distribution parameters, and the T is the data period of the i-th group of sensor data; it should be noted that the sensor data can be divided into multiple groups of sensor data through a time window. In this embodiment, the distribution parameters correspond to a preset probability distribution of a normal distribution N(μ, Σ); in this embodiment, the calculation formula of the reconstruction error e (i) is: e (i) =|x (i) -x' (i) |;

[0108] Wherein, x (i) is the i-th group of sensor data, and x' (i) is the reconstructed data corresponding to the i-th group of sensor data.

[0109] Specifically, taking a group of sensor data as an example, calculate the reconstruction error e (i) , the distribution parameters μ, Σ, and the data period T of this group of sensor data. Further, substitute the reconstruction error e (i) , the distribution parameters μ, Σ, and the data period T into the preset abnormal score calculation formula for calculation, so as to obtain the abnormal score of this group of sensor data.

[0110] S542. Judge whether the sensor data is abnormal based on the abnormal score and a preset score threshold.

[0111] Wherein, the score threshold is a preset threshold, which is used to compare with the abnormal score to judge whether a corresponding group of sensor data is abnormal; for example, if the abnormal score exceeds the preset score threshold, it is judged that the corresponding group of sensor data is abnormal; otherwise, it is judged that the corresponding group of sensor data is normal.

[0112] Specifically, it is determined whether the abnormal score exceeds a preset score threshold. If so, it is determined that a corresponding set of sensor data is abnormal; otherwise, it is determined that a corresponding set of sensor data is normal.

[0113] It should be noted that on the basis of calculating the heterogeneous error, the abnormal score a is further calculated in combination with the maximum likelihood estimation (i) , so as to determine whether a corresponding set of sensor data is abnormal; adopting this method can effectively reduce the false alarm rate of abnormal detection of sensor data, because the method of using the maximum likelihood estimation can calculate the abnormal score based on the probability distribution. For short-term occasional abnormal points, the abnormal score will not be particularly high. Only when the sensor data within a period of time is detected as abnormal, the abnormal score will increase significantly, thereby improving the effect of abnormal detection of sensor data.

[0114] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps in other steps.

[0115] Embodiment Six

[0116] Based on the same inventive concept, an embodiment of the present disclosure also provides a sensor data abnormal detection device for implementing the above-mentioned sensor data abnormal detection method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the sensor data abnormal detection device provided below can refer to the limitations on the sensor data abnormal detection method in the above text, and will not be repeated here.

[0117] In this embodiment, as Figure 3 shown, a sensor data abnormal detection device is provided, including:

[0118] A period judgment module, configured to judge whether the sensor data has periodicity;

[0119] A reconstruction calculation module, configured to, if so, process the sensor data based on the trained abnormal detection target model to obtain a reconstruction error;

[0120] A data processing module, configured to estimate distribution parameters of a preset probability distribution based on the sensor data, and calculate a data period of the sensor data;

[0121] An anomaly judgment module, configured to judge whether the sensor data is abnormal based on the reconstruction error, the distribution parameters, and the data period.

[0122] Each module in the above sensor data anomaly detection device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0123] It should be noted that in this embodiment, it is determined whether the sensor data is periodic; if so, the sensor data is processed by the anomaly detection target model that has completed training to obtain a reconstruction error; the distribution parameters of the preset probability distribution are estimated based on the sensor data, and the data period of the sensor data is calculated; it is judged whether the sensor data is abnormal based on the reconstruction error, the distribution parameters, and the data period. Through the above implementation, when performing anomaly detection on periodic sensor data, the global information of the sensor can be extracted, thereby improving the anomaly detection effect on periodic sensor data.

[0124] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 4 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a sensor data anomaly detection method.

[0125] Those skilled in the art can understand that Figure 4 the structure shown in

[0126] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0127] In one embodiment, a computer-readable storage medium is provided, as Figure 5 shown, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0128] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0129] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0130] Those of ordinary skill in the art can understand that all or part of the processes in the above-described embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided by the present disclosure can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided by the present disclosure can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0131] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0132] The above-described embodiments only represent several implementation manners of the present disclosure. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present disclosure. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present disclosure, several modifications and improvements can still be made, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the appended claims.

Claims

1. A sensor data anomaly detection method, characterized in that: include: Determine whether the sensor data is periodic; If yes, the sensor data is processed based on the trained anomaly detection target model to obtain a reconstruction error; estimating distribution parameters of a preset probability distribution based on the sensor data, and calculating a data period of the sensor data; Whether the sensor data is abnormal is determined based on the reconstruction error, the distribution parameter, and the data period.

2. The method according to claim 1, characterized in that The steps of training the anomaly detection target model include: The sensor historical data is divided into windows to obtain a training set, and whether the training set has periodicity is determined: If yes, obtain the overall model loss based on the training set and the initial anomaly detection model; The anomaly detection target model is obtained by training the anomaly detection initial model based on the overall loss of the model.

3. The method according to claim 2, characterized in that The overall model loss is obtained based on the training set and the initial anomaly detection model, including: Obtaining an unsupervised loss based on the unsupervised data in the training set and the initial anomaly detection model; Obtaining a supervised loss based on the supervised data in the training set and the initial anomaly detection model; The overall model loss is obtained based on the unsupervised loss, the supervised loss and the supervised preset coefficient.

4. The method according to claim 3, characterized in that The overall model loss is obtained based on the unsupervised loss, the supervised loss and the supervised preset coefficient, including: Calculate the sum of the product of the preset supervision coefficient and the supervised loss and the unsupervised loss to obtain the overall loss of the model; Among them, the calculation formula of the overall loss of the model is: Loss ad_transfomer =Loss informer +α*Loss sup ; Among them, the Loss ad_transfomer is the overall loss of the model, the Loss informer is the unsupervised loss, α is the supervised preset coefficient, and Loss sup is the supervised loss.

5. The method according to claim 1, characterized in that The determining whether the sensor data is abnormal based on the reconstruction error, the distribution parameter, and the data period includes: Processing the reconstruction error, the distribution parameter, and the data period based on a preset anomaly score calculation formula to obtain an anomaly score; Whether the sensor data is abnormal is determined based on the abnormality score and a preset score threshold.

6. The method according to claim 5, characterized in that The abnormal score calculation formula is: (i) =(e (i) -μ) T ∑ -1 (e (i) -μ); Among them, the a (i) is the abnormal score of the i-th group of sensor data, the e (i) is the reconstruction error of the i-th group of sensor data, μ and Σ are distribution parameters, and T is the data period of the i-th group of sensor data.

7. A sensor data anomaly detection device, characterized in that: The device comprises: A period determination module is used to determine whether the sensor data is periodic; A reconstruction calculation module, for processing the sensor data based on the trained anomaly detection target model to obtain a reconstruction error; A data processing module, used for estimating distribution parameters of a preset probability distribution based on the sensor data, and calculating a data period of the sensor data; An abnormality judgment module is used to judge whether the sensor data is abnormal based on the reconstruction error, the distribution parameter and the data period.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images