Multivariable data anomaly detection method

Abstract

The invention discloses a multivariable data anomaly detection method, and relates to the technical field of data detection. Comprising the following steps: data collection: collecting, classifying and storing data; data preprocessing: performing data cleaning on the collected data; the anomaly detection algorithm library is used for carrying out data anomaly detection, anomaly diagnosis and anomaly cause interpretation on the cleaned data; abnormal standard library construction: constructing abnormal judgment standards and reason instructions, establishing an abnormal judgment standard library, and providing processing suggestions and reason instructions for each abnormal type; and the visual platform is used for displaying the abnormal causes. According to the method, a key technical means is provided for multivariable data anomaly detection, anomaly diagnosis and anomaly cause explanation in combination with a visualization technology, the service requirement and the user interaction requirement of multivariable data anomaly detection are fully considered, and a development technical route is formed.

Description

Technical Field

[0001] The present invention relates to the technical field of data detection, and more particularly to a multivariate data anomaly detection method. Background Art

[0002] Information systems usually use physical devices or virtual devices (for example, temperature sensors, barcodes, buried points, etc.) to collect data from different components or functional modules of the information system, which are usually presented in multidimensional data (or multivariate data, multivariate data) to reflect the operating status of the information system. Efficient multivariate data anomaly detection is of great significance for early warning of anomalies in information systems and ensuring stable operation. In recent years, multivariate data anomaly detection has been widely used in industrial manufacturing, financial transactions, network security and other fields. By identifying and marking abnormal information in various application scenarios, it plays an important role in promoting the development of intelligent operation and maintenance in various fields. However, with the continuous improvement of Internet hardware facilities and the deepening of system applications, the scale of multivariate data continues to expand, making multivariate data anomaly detection face increasingly severe challenges. In addition, the lack of sensitivity of data sampling technology in application scenarios, the poor timeliness of system state change response speed, and the difficulty in estimating the strength of sampling data time series dependence lead to a high missed detection rate. In the context of large-scale multivariate data and complex coupling relationships, it is a crucial basic work to carry out the description of the coupling relationship between multivariate data, the research on multivariate data anomaly detection under the time series dependency misalignment and time-lag causal effect, and to improve the level of intelligent operation and maintenance.

[0003] At present, the research on multivariate data anomaly detection is mainly carried out in the field of multivariate data spatial feature enhancement. However, in practical applications, the early manifestations of multivariate data anomalies are extremely subtle, and when the boundary between abnormal state and normal state is unclear, the existing multivariate data anomaly detection has a high missed detection rate.

[0004] Therefore, it is an urgent problem for those skilled in the art to propose a multivariate data anomaly detection method to solve the difficulties existing in the prior art. Summary of the invention

[0005] In view of this, the present invention provides a multivariate data anomaly detection method, which closely revolves around the core business of anomaly detection, anomaly diagnosis and anomaly cause explanation, realizes comprehensive analysis and processing of data in a progressive manner, and uses a visualization module to enable users to understand the abnormal phenomena in the data more intuitively.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A multivariate data anomaly detection method comprises the following steps:

[0008] Data collection: used for data collection, classification and storage of information systems in the field of industrial production;

[0009] Data preprocessing: used to clean the collected data;

[0010] Anomaly detection algorithm library: performs data anomaly detection, anomaly diagnosis, and anomaly cause explanation on cleaned data;

[0011] Construction of abnormal standard library: construct abnormal judgment standards and cause descriptions, establish abnormal judgment standard library, and provide processing suggestions and cause descriptions for each abnormal type;

[0012] Visualization platform: displays the causes of anomalies.

[0013] In the above method, optionally, data collection includes: real-time data and historical data;

[0014] Data classification classifies or merges real-time data and historical data according to different business scenarios;

[0015] Data storage stores real-time data and historical data in the cloud according to sensor types.

[0016] The above method optionally uses a database for data storage and data table design for business scenarios.

[0017] In the above method, optional data cleaning includes missing value filling, outlier processing, and data type conversion.

[0018] The above method optionally includes performing data anomaly detection on the cleaned data, including: online detection and offline detection. The online detection is for anomaly detection of real-time data collection business, does not participate in the training process of the algorithm library model, and directly performs business application; the offline detection is for offline training of anomaly detection algorithms and parameter recording for historical data collection business.

[0019] In the above method, optionally, the entity types of the anomaly detection algorithm library are: operator data entity relationship, operator abnormal data entity relationship, operator offline training entity relationship, operator online training entity relationship, operator abnormality diagnosis entity relationship and operator abnormality cause explanation entity relationship.

[0020] In the above method, the specific content of displaying the abnormal cause is optional:

[0021] Anomaly detection is done by marking abnormal points on the curve of the actual operating sensor data;

[0022] Abnormal diagnosis: diagnose abnormal sensors through HR or NDCG indicators;

[0023] The explanation of the abnormal causes is presented through the topological structure of the multivariable coupling network, and the influence relationship of the variables is analyzed.

[0024] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a multivariate data anomaly detection method, and its beneficial method is:

[0025] 1) It covers pre-operations such as data collection, preprocessing, model preloading, and task orchestration to provide a highly adaptable and robust multivariate anomaly detection model;

[0026] 2) Combining visualization technology provides key technical means for multivariate data anomaly detection, anomaly diagnosis and anomaly cause explanation, and fully considers the business needs and user interaction needs of multivariate data anomaly detection to form a development technology route;

[0027] 3) It can not only complete multivariate data anomaly detection, diagnosis and cause explanation, but also support the import of multivariate data anomaly detection models, the import of data sets and the annotation of abnormal points. Through the free combination of different multivariate data anomaly detection models and data sets, it provides higher anomaly detection flexibility and provides an important reference for the practical application of multivariate data anomaly detection theory. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0029] Figure 1 A flowchart of a multivariate data anomaly detection method provided by the present invention;

[0030] Figure 2 The operator data ER diagram provided by the present invention;

[0031] Figure 3 The ER diagram of operator abnormal data provided by the present invention;

[0032] Figure 4 Offline detection ER diagram for operators provided by the present invention;

[0033] Figure 5 An operator online detection ER diagram provided by the present invention;

[0034] Figure 6 The operator abnormality diagnosis ER diagram provided by the present invention;

[0035] Figure 7The ER diagram of the causes of operator abnormalities provided by the present invention;

[0036] Figure 8 A schematic diagram of the associated logic of the operator data table provided by the present invention;

[0037] Fig. 9 A schematic diagram of the associated logic of the operator abnormal data table provided by the present invention;

[0038] Fig.10 The operator algorithm library, online detection and offline detection logic schematic diagram provided by the present invention;

[0039] Fig.11 A schematic diagram of the associated logic of the operator, anomaly detection model and anomaly explanation provided by the present invention;

[0040] Fig.12 A schematic diagram of the multivariate anomaly detection technology provided by the present invention;

[0041] Fig.13 The anomaly detection result diagram provided by the present invention;

[0042] Fig.14 The abnormal detection record diagram provided by the present invention;

[0043] Fig.15 The abnormal diagnosis result diagram provided by the present invention;

[0044] Fig.16 The abnormal diagnosis record diagram provided by the present invention;

[0045] Fig.17 The abnormal cause result diagram provided by the present invention;

[0046] Fig.18 This is a record diagram of the abnormality cause provided by the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] See also Figure 1 As shown, the present invention discloses a multivariate data anomaly detection method, comprising the following steps:

[0049] Data collection: used for data collection, classification and storage in information systems in the field of industrial production;

[0050] Data preprocessing: used to clean the collected data;

[0051] Anomaly detection algorithm library: performs data anomaly detection, anomaly diagnosis, and anomaly cause explanation on cleaned data;

[0052] Construction of abnormal standard library: construct abnormal judgment standards and cause descriptions, establish abnormal judgment standard library, and provide processing suggestions and cause descriptions for each abnormal type;

[0053] Visualization platform: displays the causes of anomalies.

[0054] Furthermore, data collection includes: real-time data and historical data;

[0055] Data classification classifies or merges real-time data and historical data according to different business scenarios;

[0056] Data storage stores real-time data and historical data in the cloud according to sensor types.

[0057] Furthermore, a database is used for data storage and data table design for business scenarios.

[0058] Furthermore, data cleaning includes missing value filling, outlier processing, and data type conversion.

[0059] Specifically, for anomaly detection that deviates from the normal mode, online detection normalizes or standardizes the data, and finally performs data sliding window and sends it to the algorithm library for anomaly detection.

[0060] Furthermore, data anomaly detection on the cleaned data includes: online detection and offline detection. Online detection is for anomaly detection of real-time data collection business, does not participate in the training process of the algorithm library model, and directly performs business application; offline detection is for offline training of anomaly detection algorithms and parameter recording for historical data collection business.

[0061] Specifically, offline detection provides model updates for online detection and optimizes the accuracy of online detection.

[0062] Furthermore, the entity types of the anomaly detection algorithm library are: operator data entity relationship, operator abnormal data entity relationship, operator offline training entity relationship, operator online training entity relationship, operator abnormality diagnosis entity relationship and operator abnormality cause explanation entity relationship.

[0063] Specifically, the operator data entity relationship is as follows Figure 2As shown, operators are responsible for managing (adding, deleting, modifying, and checking) data sets in different business scenarios. In order to quickly view the data sets managed by operators, a data management table is used to associate the relationship between data sets and operators. Finally, the multivariate data set contains multiple sensor points, so each sensor needs to be defined and described.

[0064] The operator data entity relationship includes four entities: operator, data, dataset, and sensor, and the corresponding relationships between different entities are: operator > data → 1 > n, data > dataset → 1 > n, dataset > sensor → 1 > n, operator > sensor → 1 > n, operator > dataset → 1 > n. 1 > n represents a one-to-many relationship, where "operator > data → 1 > n" means that the corresponding relationship between operator and data is one-to-many.

[0065] Operator abnormal data entity relationship is as follows Figure 3 As shown in the figure, operators are responsible for managing (adding, deleting, modifying, and checking) the abnormal records of data sets in different business scenarios. In order to quickly view the abnormal records of data sets managed by operators, the abnormal library is used to associate the relationship between data set abnormalities and operators. Finally, multivariate data set abnormalities contain multiple different abnormal standards. Therefore, each abnormal pattern needs to be defined and described. The user abnormal data entity relationship includes four entities: operator, abnormal library, data set abnormality, and abnormal standard. The corresponding relationships between different entities are: operator>abnormal library→1>n, abnormal library>data set abnormality→1>n, data set abnormality>abnormal standard→1>n, operator>abnormal standard→1>n, operator>data set abnormality→1>n. 1>n represents a one-to-many relationship.

[0066] Operators train entity relations offline such as Figure 4 As shown, the operator is responsible for managing (adding, deleting, modifying and checking) the data sets and model training records in different business scenarios. In order to quickly view the anomaly detection model training records managed by the operator, the algorithm library is used to associate the relationship between the anomaly detection model and the operator. Finally, the results of different parameter configurations and equipment parameters of the multivariate data set and the anomaly detection model are not the same. Therefore, it is necessary to record the training process of the anomaly detection model for data sets in different business scenarios.

[0067] The operator offline training entity relationship includes four entities: operator, algorithm library, model, and training record. The corresponding relationships between different entities are: operator > algorithm library → 1 > n, algorithm library > model → 1 > n, model > training record → 1 > n, operator > model → 1 > n, operator > training record → 1 > n. 1 > n represents a one-to-many relationship.

[0068] Operators train entity relationships online such as Figure 5As shown, the operator is responsible for managing (adding, deleting, modifying and checking) the data sets and model detection records in different business scenarios. In order to quickly view the detection records of the anomaly detection model managed by the operator, the algorithm library is used to associate the relationship between the anomaly detection model and the operator. Finally, the detection results of multivariate data sets and different parameter configurations and equipment parameters of the anomaly detection model are not the same. Therefore, it is necessary to record the detection results of the anomaly detection model on the data sets in different business scenarios.

[0069] The operator offline training entity relationship includes four entities: operator, algorithm library, model, and prediction result. The corresponding relationships between different entities are: operator > algorithm library → 1 > n, algorithm library > model → 1 > n, model > prediction result → 1 > n, operator > model → 1 > n, operator > prediction result → 1 > n. 1 > n represents a one-to-many relationship.

[0070] Operator abnormality diagnosis entity relationship such as Figure 6 As shown, the operator is responsible for managing (adding, deleting, modifying and checking) the data sets and model detection records in different business scenarios. In order to quickly view the diagnosis records of the anomaly detection model managed by the operator, the algorithm library is used to associate the relationship between the anomaly detection model and the operator. Finally, the diagnosis results of multivariate data sets and different parameter configurations and equipment parameters of the anomaly detection model are not the same. Therefore, it is necessary to record the diagnosis results of the anomaly detection model for data sets in different business scenarios.

[0071] The operator abnormal diagnosis entity relationship includes four entities: operator, algorithm library, model, and abnormal diagnosis. The corresponding relationships between different entities are: operator > algorithm library → 1 > n, algorithm library > model → 1 > n, model > abnormal diagnosis → 1 > n, operator > model → 1 > n, operator > abnormal diagnosis → 1 > n. 1 > n represents a one-to-many relationship.

[0072] Operator abnormality causes explain entity relationships such as Figure 7 As shown, the operator is responsible for managing (adding, deleting, modifying and checking) the data sets and the records of the causes of anomalies in different business scenarios. In order to quickly view the records of the causes of anomalies managed by the operator, the algorithm library is used to associate the relationship between the anomaly detection model and the operator. Finally, the causes of different parameter configurations and equipment parameters of the multivariate data set and the anomaly detection model are not the same. Therefore, it is necessary to record the anomaly detection model's anomaly cause results for the data sets in different business scenarios.

[0073] The operator abnormality cause relationship includes four entities: operator, algorithm library, model, and abnormality cause. The corresponding relationships between different entities are: operator > algorithm library → 1 > n, algorithm library > model → 1 > n, model > abnormality cause → 1 > n, operator > model → 1 > n, operator > abnormality cause → 1 > n. 1 > n represents a one-to-many relationship.

[0074] The anomaly detection operator table and other business tables are associated and managed through the operator's ID. The tables corresponding to the operator data entity relationship are adOperator, adDatas, adDataset and adSensor. The tables corresponding to the operator abnormal data entity relationship are adOperator, adStore, adDataSetStore and adStand. The tables corresponding to the operator offline training entity relationship are adOperator, adAlgorithm, adModel and adTrainRecord. The tables corresponding to the operator online training entity relationship are adOperator, adAlgorithm, adModel and adOnline. The tables corresponding to the operator abnormal diagnosis entity relationship are adOperator, adAlgorithm, adModel and adDiagnose. The tables corresponding to the operator abnormal cause explanation entity relationship are adOperator, adAlgorithm, adModel and adExplain. The following are the logical relationships and data table structure details of different business data tables:

[0075] Table 1 List of data tables for multivariate data anomaly detection

[0076] # Data Sheet name 1 adOperator Anomaly Detection Operator 2 adAlgorithm Exception Algorithm Library 3 adModel Anomaly Detection Model 4 adDatas database 5 adDataset Anomaly Detection Dataset 6 adSensor sensor 7 adStore Exception Library 8 adDataSetStore Dataset anomaly library 9 adStand Exception Standard Library 10 adTrainRecord Training records 11 adOnline Abnormal online monitoring 12 adDiagnose Abnormal diagnosis 13 adExplain Abnormal Explanation Table

[0077] The operator data table is associated with the logical diagram as follows Figure 8 As shown, the anomaly detection operator table and database table, the sensor table and anomaly detection dataset are associated through the operator ID, the database table and anomaly detection dataset are associated through the data ID, and the sensor is associated with the anomaly detection dataset table and database table through the dataset ID. The following is the table structure details of anomaly detection operator table 2, database table table 3, anomaly detection dataset table 4, and sensor table 5.

[0078] Table 2 adOperator table structure

[0079] # Fields name Data Types Primary Key Not empty default value 1 authority Operator permissions INT √ 2 id Operator id INT √ √ 3 number Operator Number VARCHAR(255) √

[0080] Table 2 shows the basic details of the anomaly detection operator table. Since the core business of multivariate anomaly detection is multivariate anomaly detection, the basic information about the operator is not recorded, only the operator's authority and operator employee number. The operator authority is mainly set to provide different levels of management for the management database, anomaly database, and algorithm library to improve the security of the anomaly detection system.

[0081] Table 3 adDatas table structure

[0082] # Fields name Data Types Primary Key Not empty default value 1 id Data id INT √ √ 2 describe Dataset Description VARCHAR(255) √ 3 name Dataset name VARCHAR(255) √ 4 adOperator Add person id INT √

[0083] Table 3 is a management table of anomaly detection personnel and data sets in different business scenarios. By associating the operator ID and the data set ID, the efficiency of anomaly detection personnel in managing data sets is improved.

[0084] Table 4 adDataset table structure

[0085] # Fields name Data Types Primary Key Not empty default value 1 id Dataset id INT √ √ 2 sensorld Sensor ID INT √ 3 timeStamp Timestamp DATETIME √ 4 dataId Data id VARCHAR(255) √ 5 batchId Batch ID INT √ 6 value Collected data values DECIMAL(24,6) √ 7 adLevel Abnormal Level INT √ 8 adOperator Operator INT √

[0086] Table 4 is a table of data sets under different business scenarios. Different sensor data under the same business scenario is stored. The batch ID is used to ensure that there is a delay in different sensor data during real-time data transmission. Therefore, the batch number is used to unify the delayed data to ensure data consistency. The sensor ID is the device to which the data is uploaded.

[0087] Table 5 adSensor table structure

[0088]

[0089]

[0090] Table 5 is a detailed table of data collection sensor devices, which is associated with the adDataset table through the dataset ID and sensor ID.

[0091] The associated logic diagram of the operator abnormal data table is as follows Fig. 9 As shown, the anomaly detection operator table and the anomaly association table, the anomaly standard table and the data set anomaly table are associated through the operator ID, the data table and the data set anomaly table are associated through the anomaly data set ID, and the anomaly standard table associates the anomaly detection data table and the data set anomaly table through the data set ID. Anomaly detection operator table 2 has been introduced. The following are the table structure details of anomaly association table 6, data set anomaly table 7 and anomaly standard table 8, as follows:

[0092] Table 6 adStore table structure

[0093] # Fields name Data Types Primary Key Not empty default value 1 id Exception id INT √ √ 2 name Abnormal dataset name VARCHAR(255) √ 3 describe Abnormal Dataset Description VARCHAR(255) √ 4 adOperator Operator INT √

[0094] Table 6 is an operator and data set anomaly association table. Anomaly detection operators manage abnormal data in different business scenarios through operator IDs.

[0095] Table 7 adDataSetStore table structure

[0096] # Fields name Data Types Primary Key Not empty default value 1 adOperator Operator INT √ 2 id Abnormal dataset id INT √ √ 3 sensorId Sensor ID INT √ 4 StoreId Exception library id INT √ 5 adLevel Exception id INT √ 6 value Outliers DECIMAL(24,6) √ 7 batchId Batch Id INT √

[0097] Table 8 adStand table structure

[0098]

[0099]

[0100] Table 7 is the abnormal data set table, which is responsible for recording the abnormal information of the data set. The abnormal library association table is associated with the abnormal association table through the abnormal association table ID, and the sensor ID is the device information associated with the abnormal data. The batch ID is to ensure that there is a delay in the data of different sensors during the real-time data transmission process, so the batch number is used to unify the delayed data to ensure data consistency. At the same time, it reflects which sensor in which batch has an abnormality.

[0101] Table 8 is an abnormal standard table, which is responsible for recording the corresponding device information in the abnormal information of the data set, including the sensor ID, and the abnormal upper limit value and abnormal lower limit value of the sensor. It can quickly identify single-variable sensors, and add a description of the abnormality according to different single-variable abnormal levels, which can quickly provide a description of the cause of the abnormality explanation.

[0102] Operator algorithm library, online detection and offline monitoring logic diagrams such as Fig.10 As shown, the anomaly detection operator table and the anomaly algorithm table, anomaly detection model, anomaly online detection table, and training record table are associated through the operator ID, the anomaly detection model table is associated with the anomaly algorithm table through the model ID, the dataset ID is associated with the dataset anomaly table and the dataset table, and the anomaly online detection table and the training record table are associated with the anomaly detection model table through the model ID. The anomaly detection operator table 2 has been introduced. The following are the structural details of the anomaly algorithm library table 9, the anomaly detection model table 10, the anomaly online detection table 11, and the anomaly training record table 12. The details are as follows:

[0103] Table 9 adAlgorithm table structure

[0104]

[0105]

[0106] Table 9 is an anomaly detection algorithm table, which specifically includes the algorithm name, algorithm description and algorithm ID. The anomaly detection operator is managed through the operator ID.

[0107] Table 10adModel table structure

[0108] # Fields name Data Types Primary Key Not empty default value 1 adOperator Operator INT √ 2 algorithmId Algorithm ID INT √ √ 3 timeStamp Timestamp DATETIME √ 4 dataSetId Dataset id INT √ 5 device Equipment parameters VARCHAR(255) √ 6 modelConfig Model parameters VARCHAR(255) √ 7 id Model id INT √

[0109] Table 10 is an anomaly detection model table, which specifically includes model parameters, device parameters, data set ID and model ID. The anomaly detection model table is associated with the anomaly detection algorithm library through the algorithm ID.

[0110] Table 11adOnline table structure

[0111] # Fields name Data Types Primary Key Not empty default value 1 adOperator Operator INT √ 2 dataSetId Data id INT √ 3 adLevel Abnormal Level INT √ 4 id Online test results INT √ √ 5 timeStamp Timestamp DATETIME √ 6 batchId Batch Id INT √ 7 modeIId Model id INT √

[0112] Table 11 is an abnormal online detection table, which is responsible for recording the detection results of real-time data collection, associating the model used for real-time data monitoring through the model ID, associating the database table with the data ID, associating the data with the batch ID, and recording whether the data is abnormal and the level of the abnormality. According to the abnormality level corresponding to the data set, the cause of the abnormality is queried.

[0113] Table 12adTrainRecord table structure

[0114] # Fields name Data Types Primary Key Not empty default value 1 adOperator Operator INT √ 2 id Training ID INT √ √ 3 duringTime Duration DATETIME √ 4 GPUMemory GPU Memory DECIMAL(24,6) √ 5 CPUMemory CPU Memory DECIMAL(24,6) √ 6 F1 Model evaluation metrics DECIMAL(24,6) √ 7 AUC Model evaluation metrics DECIMAL(24,6) √ 8 Rec Model evaluation metrics DECIMAL(24,6) √ 9 Pre Model evaluation metrics DECIMAL(24,6) √ 10 NDCG1 Model evaluation metrics DECIMAL(24,6) √ 11 NDCG5 Model evaluation metrics DECIMAL(24,6) √ 12 NDCG10 Model evaluation indicators DECIMAL(24,6) √ 13 HR1 Model evaluation indicators DECIMAL(24,6) √ 14 HR5 Model evaluation metrics DECIMAL(24,6) √ 15 HR10 Model evaluation metrics DECIMAL(24,6) √ 16 learing Learning Rate DECIMAL(24,6) √ 17 timeStamp Timestamp DATETIME √ 18 epoch Iteration number INT √ 19 batchId Batch Id INT √ 20 dataSetaId Dataset Id INT √ 21 modeIId Model id INT √

[0115] Table 12 is the anomaly offline training record table, which is responsible for recording the algorithm associated with the model ID, the data associated with the dataset ID, the data batch associated with the batch ID, the evaluation indicators of the anomaly detection model, the CPU memory occupancy, the GPU memory occupancy and other detailed processes in the anomaly detection model training process, providing better optimization guidance for the anomaly detection model.

[0116] The logic diagram of the operator, anomaly detection model, anomaly diagnosis and anomaly explanation data tables is shown in the figure. Fig.11 As shown, the anomaly detection operator table and the anomaly detection model, anomaly explanation table, and anomaly diagnosis table are associated through the operator ID, the anomaly explanation table is associated through the anomaly detection model ID, and the anomaly diagnosis table is associated through the anomaly detection model ID. The operator table has been introduced in Table 2, and the anomaly detection model has been introduced in Table 11. The following is an introduction to the table structure of the anomaly diagnosis table 13 and the anomaly cause explanation table 14, as follows:

[0117] Table 13 adDiagnose table structure

[0118] # Fields name Data Types Primary Key Not empty default value 1 adOperator Operator INT √ 2 id Explanation ID INT √ √ 3 adLevel Abnormal Level INT √ 4 dataSetId Dataset id INT √ 5 timeStamp Timestamp DATETIME √ 6 Result Diagnosis VARCHAR(255) √ 7 modeIId Diagnostic model id INT √ 8 batchId Batch Id INT √

[0119] Table 13 is the anomaly diagnosis record table, which is responsible for recording the algorithm associated with the anomaly diagnosis model ID, the data associated with the data set ID, the data batch associated with the batch ID, and the evaluation index of the anomaly detection model. The result record of the anomaly diagnosis is in the image format, so this field records the image path of the anomaly result.

[0120] Table 14 adExplain table structure

[0121] # Fields name Data Types Primary Key Not empty default value 1 id Cause ID INT √ √ 2 adLevel Abnormal Level INT √ 3 dataSetId Dataset id INT √ 4 timeStamp Timestamp DATETIME √ 5 Result Cause and effect VARCHAR(255) √ 6 modeIId Cause model id INT √ 7 adOperator Operator INT √ 8 batchId Batch Id INT √

[0122] Table 14 is the anomaly cause explanation record table, which is responsible for recording the algorithm associated with the anomaly cause explanation model ID, the data associated with the data set ID, the data batch associated with the batch ID, and the evaluation index of the anomaly detection model. The result record of the anomaly diagnosis is result. Since the anomaly diagnosis result is in image format, this field records the anomaly cause explanation result image path.

[0123] Specifically, the multivariate data anomaly detection system developed and implemented mainly in Python language, the details of various components of its operating environment and the hardware configuration list are shown in Table 15.

[0124] Table 15 System environment configuration

[0125] Serial number Object Configuration 1 Compile / Run System Windows 11 Professional 2 Python runtime environment Python 3.10 3 Database software MySQL 5.5.29 4 Data interface components Falsk3.0.0 5 Client UI Components PyQt55.19.9 6 Database operation components PyMySQL1.1.0 7 View Components LayUI2.8.18 8 Web Components PyQtWebEngine5.15.6 9 Compile / Run CPU Configuration Intel(R)Xeon(R)E-2244G 10 Compile / Run GPU Configuration NVIDIAQuadroP620

[0126] System development technology route:

[0127] The multivariate anomaly monitoring software development model follows the MVC model with front-end and back-end separation, namely the three modules of object model, business control and view display. Fig.12 shown.

[0128] The object model is a relationship mapping model that establishes the relationship between business operations and database MySQL operations, and completes basic operations such as adding, deleting, querying and modifying the data tables corresponding to the data objects.

[0129] Business control is the business request processing between the client and the server. Based on the Falsk framework, the client establishes a connection with the server through an Ajax request. The server responds to different business requests based on the request parameters, such as anomaly detection task scheduling, timed detection tasks, etc.

[0130] The view display is based on PyQt to provide users with a window-like interactive container for anomaly detection. A streaming adaptive, beautiful and practical view is developed through HTML for hosting. The communication between PyQt and HTML is carried out through QtWebChannel. The beautification of HTML interface elements is achieved through the front-end frameworks LayUI and LayTPL.

[0131] In summary, the multivariate data anomaly detection system is divided into three major development modules, namely the view layer, the control layer and the model layer. They are independent in function but have business connections with each other, and together they realize all the functions of anomaly detection.

[0132] Furthermore, the specific contents of displaying the abnormal causes are as follows:

[0133] Anomaly detection is done by marking abnormal points on the curve of the actual operating sensor data;

[0134] Specifically, first, through the multivariate data anomaly detection configuration, select the detection data set, complete the detection model trained on the data set, and the detection start date. Secondly, click the anomaly detection button to perform anomaly detection. Finally, through the multivariate data anomaly detection result visualization part, the multivariate data anomaly detection results are displayed. The multivariate data anomaly detection module includes two modules: anomaly detection configuration and result display.

[0135] The results of multivariate data anomaly detection are as follows Fig.13 As shown in the figure, the anomaly detection results are shown. The curve graph shows the anomaly point annotations of different sensors in the data set, and the map shows the sensor distribution of the data set.

[0136] Multivariate data anomaly detection records such as Fig.14 As shown, multivariate data anomaly detection records are displayed, and the detection record deletion function is provided.

[0137] Abnormal diagnosis: diagnose abnormal sensors through HR or NDCG indicators;

[0138] Specifically, first, through the multivariate data anomaly diagnosis configuration, select the test data set, complete the detection model trained on the data set, and the detection start date. Secondly, execute the multivariate data anomaly diagnosis button to perform anomaly detection. Finally, through the multivariate data anomaly diagnosis result visualization part, the multivariate data anomaly diagnosis results are displayed. The anomaly diagnosis module includes two modules: anomaly diagnosis configuration and result display.

[0139] The results of multivariate data anomaly diagnosis are as follows Fig.15 As shown in the figure, the results of multivariate data anomaly diagnosis are displayed. The horizontal bar chart shows the anomaly values ​​of different sensors in the data set. The right side shows the comprehensive anomaly values ​​of different sensors that deviate from the normal. The larger the value, the more likely the sensor is to be abnormal.

[0140] Multivariate data anomaly diagnosis record Fig.16 As shown, multivariate data abnormality diagnosis records are displayed, and the diagnosis record deletion function is provided.

[0141] The explanation of the abnormal causes is presented through the topological structure of the multivariable coupling network, and the influence relationship of the variables is analyzed.

[0142] Specifically, first, through the multivariate data anomaly cause configuration, select the test data set, the detection model trained for the data set, and the test start date. Secondly, execute the multivariate data anomaly cause analysis button to perform multivariate data anomaly cause analysis. Finally, through the multivariate data anomaly cause visualization, the multivariate data anomaly cause results are displayed. The multivariate data anomaly cause module includes two modules: multivariate data anomaly cause configuration and result display.

[0143] The results of the causes of multivariate data anomalies are as follows Fig.17 As shown, the results of the causes of multivariate data anomalies are displayed, and the correlation and influence relationships between different sensors are analyzed.

[0144] The causes of multivariate data anomalies are recorded as follows: Fig.18 As shown, the multivariate data abnormal cause record is displayed, and the multivariate data cause record deletion function is provided.

[0145] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0146] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multivariate data anomaly detection method, characterized in that: The following steps are involved: Data collection: used for data collection, classification and storage of information systems in the field of industrial production; Data preprocessing: used to clean the collected data; Anomaly detection algorithm library: performs data anomaly detection, anomaly diagnosis, and anomaly cause explanation on cleaned data; Construction of abnormal standard library: construct abnormal judgment standards and cause descriptions, establish abnormal judgment standard library, and provide processing suggestions and cause descriptions for each abnormal type; Visualization platform: displays the causes of anomalies.

2. A multivariate data anomaly detection method according to claim 1, characterized in that: Data collection includes: real-time data and historical data; Data classification classifies or merges real-time data and historical data according to different business scenarios; Data storage stores real-time data and historical data in the cloud according to sensor types.

3. A multivariate data anomaly detection method according to claim 2, characterized in that: Use database for data storage and data table design for business scenarios.

4. A multivariate data anomaly detection method according to claim 1, characterized in that: Data cleaning includes filling missing values, handling outliers, and data type conversion.

5. The multivariate data anomaly detection method according to claim 1, characterized in that: Data anomaly detection on cleaned data includes: online detection and offline detection. Online detection is for anomaly detection of real-time data collection business. It does not participate in the training process of the algorithm library model and directly performs business applications. Offline detection is for offline training of anomaly detection algorithms and parameter recording for historical data collection business.

6. A multivariate data anomaly detection method according to claim 1, characterized in that: The entity types of the anomaly detection algorithm library are: operator data entity relationship, operator abnormal data entity relationship, operator offline training entity relationship, operator online training entity relationship, operator abnormality diagnosis entity relationship and operator abnormality cause explanation entity relationship.

7. A multivariate data anomaly detection method according to claim 1, characterized in that: The specific contents showing the abnormal causes are as follows: Anomaly detection is done by marking abnormal points on the curve of the actual operating sensor data; Abnormal diagnosis: diagnose abnormal sensors through HR or NDCG indicators; The explanation of the abnormal causes is presented through the topological structure of the multivariable coupling network, and the influence relationship of the variables is analyzed.

Images