Abnormality detection method and device

Abstract

The invention discloses an anomaly detection method and device. The method comprises the steps of obtaining at least one index related to a target industrial system and a sample index data set corresponding to the at least one index; extracting at least one index feature corresponding to the at least one index from the sample index data set, and under the condition that the at least one index feature meets a first condition, performing dimension reduction on the at least one index feature to obtain at least one fusion feature; and selecting at least one feature from the at least one index feature and the at least one fusion feature according to a preset selection rule, and performing anomaly detection on the target industrial system according to the at least one feature.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to an anomaly detection method and device. Background Art

[0002] In the process of industrial wastewater treatment, filter membranes are widely used for solid-liquid separation and water purification. However, as the usage time increases, the filter membranes will be affected by factors such as fouling, contamination, and wear, especially more likely to show anomalies when cleaning is not timely. These problems will lead to a decline in membrane performance, such as a decrease in water permeability and filtration efficiency, thus affecting the operation efficiency of the entire treatment system. Traditional monitoring methods rely on regular inspections or changes in physical / chemical parameters, lacking sufficient real-time performance and accuracy. Currently, there are also supervised learning models such as neural networks or random forests used to predict membrane damage or performance degradation, combined with online monitoring data and empirical rules, and time series analysis is used to discover trend changes. However, the generalization ability of a single model is limited, easily restricted by noise and outliers in the dataset, and difficult to adapt to the ever-changing industrial environment, resulting in low accuracy of anomaly detection. Summary of the Invention

[0003] In view of this, this application provides an anomaly detection method and device.

[0004] The technical solution of the embodiment of this application is realized as follows:

[0005] The embodiment of this application provides an anomaly detection method, and the method includes:

[0006] Obtain at least one metric related to the target industrial system and a sample metric dataset corresponding to the at least one metric;

[0007] Extract at least one metric feature corresponding to the at least one metric from the sample metric dataset, and when the at least one metric feature meets the first condition, perform dimensionality reduction on the at least one metric feature to obtain at least one fused feature;

[0008] Select at least one feature from the at least one metric feature and the at least one fused feature according to a preset selection rule, and perform anomaly detection on the target industrial system according to the at least one feature.

[0009] The embodiment of this application provides an anomaly detection device, including:

[0010] An obtaining module, configured to obtain at least one metric related to the target industrial system and a sample metric dataset corresponding to the at least one metric;

[0011] A processing module, configured to extract at least one metric feature corresponding to at least one metric from a sample metric dataset, and perform dimensionality reduction on the at least one metric feature when the at least one metric feature meets a first condition, so as to obtain at least one fusion feature;

[0012] A detection module, configured to select at least one feature from the at least one metric feature and the at least one fusion feature according to a preset selection rule, and perform anomaly detection on the target industrial system according to the at least one feature.

[0013] An embodiment of the present application provides an anomaly detection device, including: a processor, a memory, and a communication bus; the communication bus is configured to implement a communication connection between the processor and the memory; the processor is configured to execute a computer program stored in the memory to implement the above-mentioned anomaly detection method.

[0014] An embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to implement the above-mentioned anomaly detection method.

[0015] It should be understood that the above general description and subsequent detailed description are only exemplary and explanatory, and do not limit the technical solutions provided by the embodiments of the present application. Description of the Drawings

[0016] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings, where:

[0017] Figure 1 It is a schematic flowchart of an anomaly detection method provided by an embodiment of the present application;

[0018] Figure 2 It is a schematic flowchart of an exemplary method for determining whether the first condition is met provided by an embodiment of the present application;

[0019] Figure 3 It is a schematic flowchart of an exemplary feature selection method provided by an embodiment of the present application Figure 1 ;

[0020] Figure 4 It is a schematic flowchart of an exemplary feature selection method provided by an embodiment of the present application Figure 2 ;

[0021] Figure 5A schematic flowchart of an exemplary method for determining importance parameters provided by an embodiment of the present application;

[0022] Figure 6 A schematic flowchart of an exemplary feature selection method provided by an embodiment of the present application Figure 3 ;

[0023] Figure 7 A schematic flowchart of an exemplary method for determining a preset detection model provided by an embodiment of the present application Figure 1 ;

[0024] Figure 8 A schematic flowchart of an exemplary method for determining a preset detection model provided by an embodiment of the present application Figure 2 ;

[0025] Figure 9 A schematic flowchart of an exemplary method for determining a preset detection model provided by an embodiment of the present application Figure 3 ;

[0026] Figure 10 A schematic flowchart of an exemplary anomaly detection method provided by an embodiment of the present application Figure 1 ;

[0027] Figure 11 A schematic flowchart of an exemplary anomaly detection method provided by an embodiment of the present application Figure 2 ;

[0028] Figure 12 A schematic structural diagram of an anomaly detection device provided by an embodiment of the present application;

[0029] Figure 13 A schematic structural diagram of an anomaly detection device provided by an embodiment of the present application. Detailed implementation manners

[0030] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. It can be understood that the specific embodiments described herein are only used to explain the related application, rather than limiting the application. Additionally, it should be noted that for ease of description, only parts related to the related application are shown in the drawings.

[0031] An embodiment of the present application provides an anomaly detection method, which is implemented by an anomaly detection device. As Figure 1 shown, it includes the following steps S101 to S103:

[0032] Step S101: Obtain at least one metric related to the target industrial system and a sample metric data set corresponding to the at least one metric.

[0033] In an embodiment of the present application, the anomaly detection device is an electronic device with an anomaly detection function, which can be a tablet computer, a laptop computer, a handheld computer, a personal digital assistant (PDA), a desktop computer, etc. There is no specific limitation on the specific anomaly detection device here.

[0034] In an embodiment of the present application, exemplarily, the target industrial system can be an industrial wastewater filtration system, an industrial waste gas filtration system, or other industrial-related systems.

[0035] Exemplarily, when the target industrial system is an industrial wastewater filtration system, at least one index related thereto obtained by the anomaly detection device can be: influent flow rate, product water flow rate, product water rate, PC2, PC3, PC4, PC5, concentrate flow rate, PC1, product water conductivity, desalination rate, heat exchanger outlet temperature, influent pressure, concentrate pressure, inter-stage pressure, differential pressure of the first stage, etc. Of course, it can also be set based on actual requirements and application scenarios, and the present application does not make any limitations in this regard. Among them, PC2, PC3, PC4, PC5, and PC1 refer to different models of scale inhibitors.

[0036] Exemplarily, when the target industrial system is an industrial waste gas filtration system, at least one index related thereto obtained by the anomaly detection device can be: intake air flow rate, product gas flow rate, product gas rate, exhaust gas flow rate, exhaust gas concentration, product gas conductivity, intake air pressure, etc. Of course, it can also be set based on actual requirements and application scenarios, and the present application does not make any limitations in this regard.

[0037] In an embodiment of the present application, after the anomaly detection device obtains at least one index related to the target industrial system, it can obtain a sample index data set corresponding to the at least one index.

[0038] In an embodiment of the present application, after the anomaly detection device obtains the sample index data set, it can preprocess the data in the sample index data set. Exemplarily, the preprocessing can be removing noise, handling missing values, or other operations to improve data quality.

[0039] Step S102: Extract at least one index feature corresponding to at least one index from the sample index data set. When the at least one index feature meets the first condition, perform dimensionality reduction on the at least one index feature to obtain at least one fusion feature.

[0040] In an embodiment of the present application, the anomaly detection device extracts at least one index feature corresponding to at least one index from the sample index data set, and determines whether the at least one index feature meets the first condition. If it meets, perform dimensionality reduction processing on the at least one index feature to obtain at least one fusion feature.

[0041] In an embodiment of the present application, the first condition may be that certain characteristics of the index characteristics among at least one index characteristic satisfy certain conditions. Exemplarily, certain characteristics may be redundancy, correlation, or other index characteristics, and certain conditions may be quantitative limitations, proportional limitations, or other limitations; Exemplarily, the first condition may be that the number of corresponding feature redundancy values greater than 1 among at least one index characteristic is greater than one-half; or the number of corresponding feature redundancy values less than 1 among at least one index characteristic is greater than one-half.

[0042] In an embodiment of the present application, if at least one index characteristic satisfies the first condition, dimensionality reduction is performed on at least one index characteristic to obtain at least one fused feature. Exemplarily, the way for the anomaly detection device to perform dimensionality reduction on at least one index characteristic may be the Principal Component Analysis (PCA), which fuses important feature information to obtain at least one independent fused feature. In this way, the generalization ability of the model on unknown data can be improved.

[0043] Step S103: Select at least one feature from at least one index characteristic and at least one fused feature according to a preset selection rule, and perform anomaly detection on the target industrial system based on the at least one feature.

[0044] In an embodiment of the present application, the anomaly detection device may select at least one feature from at least one index characteristic and at least one fused feature according to a preset selection rule, and perform anomaly detection on the target industrial system based on the at least one feature.

[0045] In an embodiment of the present application, the preset selection rule may be a pre-set selection rule; for example, based on historical experience, a whitelist is set, and the most representative features are selected from at least one index characteristic and at least one fused feature based on the set whitelist, or redundancy calculation is performed on at least one index characteristic and at least one fused feature, and then at least one feature with low redundancy is selected, or, correlation calculation is performed on at least one index characteristic and at least one fused feature, and at least one feature with high correlation is selected.

[0046] In an embodiment of the present application, the anomaly detection device may perform anomaly detection on the target industrial system based on the selected at least one feature.

[0047] Compared with the technical problem of low anomaly detection accuracy in the related art, the present application fuses important features of at least one metric feature, and then determines at least one feature for anomaly detection from a richer feature set of at least one fused feature and at least one metric feature. While ensuring the simplification of the feature set, anomaly detection is performed based on at least one feature with high importance, which can improve the accuracy of anomaly detection.

[0048] In some embodiments, as Figure 2 shown, the anomaly detection device may perform the following steps S201 to S203:

[0049] Step S201: Calculate the redundancy of each metric feature in at least one metric feature to obtain at least one feature redundancy value.

[0050] In the embodiments of the present application, the anomaly detection device may calculate the redundancy of each metric feature in at least one metric feature to obtain at least one feature redundancy value. An exemplary implementation method for determining the feature redundancy value is shown in formula (1):

[0051]

[0052] where cor(X i ,X j ) is the correlation coefficient between feature i and feature j, std(X i ) is the standard deviation of feature i, and y i is the feature redundancy value of feature i.

[0053] In the embodiments of the present application, the anomaly detection device calculates the redundancy between each feature, that is, the degree of overlap of information between features.

[0054] Step S202: Determine the feature redundancy values greater than a preset value among at least one feature redundancy value to obtain at least one target redundancy value.

[0055] In the embodiments of the present application, the preset value may be 1, 2, or other values. Exemplarily, the preset value may be set based on actual requirements and application scenarios.

[0056] In the embodiments of the present application, when the preset value is 1, it indicates that the redundancy between features exceeds 1, indicating a high degree of overlap in information among these features.

[0057] In the embodiments of the present application, if the number of at least one metric feature is 6, then the redundancy of each metric feature in at least one metric feature is calculated to obtain the corresponding 6 feature redundancy values. The feature redundancy values greater than the preset value among these 6 feature redundancy values are determined as the target redundancy values to obtain at least one target redundancy value.

[0058] Step S203: When the number of at least one target redundancy value is greater than the first threshold, determine that at least one index feature meets the first condition.

[0059] In an embodiment of the present application, when the number of at least one target redundancy value determined by the anomaly detection device is greater than the first threshold, it is determined that at least one index feature meets the first condition. Among them, the first threshold can be one-half, one-third, or other values of the total number of at least one index feature. This application does not make a limitation on this, and it can be set based on actual requirements and application scenarios.

[0060] Exemplarily, the first threshold is one-half, that is, if the number of at least one target redundancy value is greater than one-half of the total number of at least one index feature, it is determined that at least one index feature meets the first condition; the first threshold is one-third, that is, if the number of at least one target redundancy value is greater than one-third of the total number of at least one index feature, it is determined that at least one index feature meets the first condition.

[0061] Exemplarily, the algorithm for dimensionality reduction processing is as follows:

[0062] IF sum(redundancy>1) / len(redundancy)>0.5: If the number of at least one target redundancy value (redundancy value greater than 1) determined is greater than one-half;

[0063] New_features=pca(features); Perform dimensionality reduction processing to obtain at least one fused feature

[0064] Combine features:features+new_features; At least one index feature and at least one fused feature

[0065] END IF;

[0066] In an embodiment of the present application, if at least one index feature meets the first condition, it means that the redundancy of at least one index feature is too large, that is, there is a high degree of overlap in information of the features, and data processing is required, that is, dimensionality reduction processing is performed on at least one index feature, and then subsequent anomaly detection is performed based on the data after dimensionality reduction processing.

[0067] In this way, the high quality of the data for anomaly detection can be ensured, and thus the accuracy of anomaly detection can be improved.

[0068] In some embodiments, as Figure 3 shown, when the anomaly detection device executes the above step S103, it can also execute the following steps S301 to S303:

[0069] Step S301: Perform minimum redundancy calculation on each of at least one metric feature and at least one fusion feature to obtain a plurality of minimum redundancy values.

[0070] In an embodiment of the present application, the anomaly detection device performs minimum redundancy calculation on each of at least one metric feature and at least one fusion feature. An exemplary calculation method is shown in formula (2):

[0071]

[0072] Among them, min R(S), R is the minimum redundancy value, and S is the sample metric data set.

[0073] In an embodiment of the present application, minimum redundancy calculation is performed for each feature to obtain corresponding multiple minimum redundancy values.

[0074] Step S302: Perform maximum correlation calculation on each of at least one metric feature and at least one fusion feature to obtain a plurality of maximum correlation values.

[0075] In an embodiment of the present application, the anomaly detection device can perform maximum correlation calculation on each of at least one metric feature and at least one fusion feature. An exemplary calculation method is shown in formula (3):

[0076]

[0077] Among them, maxD(S,c), D is the maximum correlation value, c is the category, and the category is normal or abnormal. I(X i ; c) is the correlation between feature i and the category.

[0078] Step S303: Select at least one feature from at least one metric feature and at least one fusion feature according to the plurality of minimum redundancy values and the plurality of maximum correlation values.

[0079] In an embodiment of the present application, the anomaly detection device selects at least one feature from at least one metric feature and at least one fusion feature according to the plurality of minimum redundancy values and the plurality of maximum correlation values. That is, at least one feature determined in the anomaly detection device is determined based on the redundancy value and the correlation value, so that the determined at least one feature is more accurate, and the anomaly detection based on the accurate at least one feature is also more accurate.

[0080] Thus, through the combination of principal component analysis (PCA) and minimum redundancy maximum relevance (mRMR), the redundancy between features is effectively reduced. PCA reduces the dimensionality of features while maintaining the main information, and mRMR further optimizes feature selection to ensure that the selected features are both highly relevant to the target and have low redundancy. The features are fused and sorted to form an efficient feature set, which significantly improves the training efficiency and prediction accuracy of the model.

[0081] In some embodiments, as Figure 4 shown, when the anomaly detection device executes the above-mentioned step S303, it may further execute the following step S401 and step S402:

[0082] Step S401: Determine the importance parameter of each feature based on multiple minimum redundancy values and multiple maximum correlation values.

[0083] In the embodiments of the present application, the importance parameter may be a parameter characterizing the importance degree of each feature in the anomaly detection process.

[0084] In the embodiments of the present application, for each feature, the anomaly detection device determines the corresponding importance parameter based on the corresponding minimum redundancy value and maximum correlation value.

[0085] Step S402: Select at least one feature from at least one metric feature and at least one fused feature according to the importance parameter of each feature.

[0086] In the embodiments of the present application, after the anomaly detection device determines the importance parameter of each feature, it may, based on the importance parameter of each feature, determine the features with importance parameters greater than the preset value as at least one feature from at least one metric feature and at least one fused feature.

[0087] Thus, the at least one selected feature is a feature with relatively high importance. Through the combination of the principal component analysis method and maximum correlation and minimum redundancy, a more refined and information-rich feature set is formed, providing better input for model training, capable of removing features with low importance and high redundancy, further streamlining the feature set, and improving the generalization and computational efficiency of the model.

[0088] In some embodiments, as Figure 5 shown, when the anomaly detection device executes the above-mentioned step S401, it may further execute the following step S501 and step S502:

[0089] Step S501: For each feature, set a penalty coefficient for the corresponding minimum redundancy value to obtain the corresponding penalty redundancy value.

[0090] In an embodiment of the present application, for each feature, the anomaly detection device sets a penalty coefficient β for the corresponding minimum redundancy value to obtain the corresponding penalty redundancy value βR.

[0091] Step S502: For each feature, determine the difference between the corresponding maximum correlation and the penalty redundancy value as the corresponding importance parameter.

[0092] In an embodiment of the present application, for each feature, the anomaly detection device determines the difference between the corresponding maximum correlation and the penalty redundancy value as the corresponding importance parameter. An exemplary determination method is shown in formula (4):

[0093] Φ = D - βR (4);

[0094] Where D is the maximum correlation value, R is the minimum redundancy value, and β is the penalty coefficient.

[0095] In this way, through Max-Relevance and Min-Redundancy (mRMR), a penalty coefficient β is introduced to control the balance between correlation and redundancy. When the redundancy between features exceeds a certain threshold, a penalty term is added to reduce the weight of redundant features, thereby further screening out the features most relevant to the target variable, while ensuring that the selected features have the minimum redundancy, enhancing the prediction accuracy and generalization of the model.

[0096] In some embodiments, as Figure 6 shown, when the anomaly detection device executes the above step S402, it may also execute the following step S601 and step S602:

[0097] Step S601: Sort at least one index feature and at least one fusion feature according to the importance parameter of each feature to obtain a sorting result.

[0098] In an embodiment of the present application, after the anomaly detection device obtains at least one index feature and at least one fusion feature, it will sort them according to the importance parameter to obtain a sorting result. Exemplarily, the sorting result may be arranged in descending order of importance. Then, the first one is the most important feature.

[0099] Exemplarily, based on the importance parameter of each feature, at least one index feature and at least one fusion feature are sorted from high to low according to importance, and then the first few features are selected from high to low. As for how many features to select can be set according to actual needs and scenarios.

[0100] Exemplarily, the sorting method is shown in formula (5):

[0101] maxΦ(D,R)(5);

[0102] Among them, Φ is the importance parameter, D is the maximum correlation value, and R is the minimum redundancy value.

[0103] Step S602: Sequentially select a preset number of features based on the sorting result, and determine the selected features as at least one feature.

[0104] In the embodiments of the present application, the preset number can be 5, 10, or other values less than the number of at least one index. Exemplarily, the preset number can be set based on actual requirements and application scenarios, and the present application does not limit this.

[0105] In the embodiments of the present application, the anomaly detection device sequentially selects a preset number of features based on the sorting result. Since the selection is made according to the importance degree, the features selected by mRMR can retain the key information related to the target variable to the greatest extent and reduce the influence of noise and irrelevant features. This helps the model have better generalization ability when processing complex industrial data and maintain a high detection accuracy in different environments.

[0106] In this way, it can be ensured that the importance degree of the at least one selected feature is relatively high, and further, the accuracy of anomaly detection based on the at least one feature with a relatively high importance degree will also be higher.

[0107] In some embodiments, as Figure 7 shown, when the anomaly detection device executes "performing anomaly detection on the target industrial system according to at least one feature" in the above step S103, the following steps S701 to S704 may further be included:

[0108] Step S701: Establish a first initial detection model based on at least one feature.

[0109] In the embodiments of the present application, the anomaly detection device may establish a first initial detection model based on at least one feature.

[0110] In the embodiments of the present application, the first initial detection model is a least squares support vector machine model (LSSVM) including a radial basis function kernel (RBF kernel). At this time, the model is an undetected detection model. By introducing the RBF kernel, the non-linear relationship in industrial data is effectively processed, and the prediction accuracy of the model is improved. Moreover, the squared loss function of LSSVM simplifies the solution process of the model, making the model more stable and efficient in high-dimensional data. And, since the abnormal samples in industrial film anomaly detection are usually relatively scarce, by combining mRMR and LS-SVM, good classification performance can still be maintained under small sample data. LS-SVM can still effectively capture the non-linear relationship under a small amount of data, ensuring the robustness of the detection model.

[0111] Step S702: Train the first initial detection model using the first sample index data to obtain the first detection model.

[0112] In the embodiments of the present application, the anomaly detection device trains the first initial detection model using the first sample index data. Among them, the first sample index data is the historical data for training the first initial detection model. In this first sample index data, only the data corresponding to at least one feature is obtained to train the first initial detection model to obtain the first detection model. Among them, the first sample index data is the index data in the sample index dataset.

[0113] It should be noted that when using the index data corresponding to at least one feature in the first sample index data, the index data will be standardized. Exemplarily, the method of standardization is shown in formula (6):

[0114]

[0115] Among them, F is the standardized feature, Xi is feature i, μ is the feature mean, and σ is the feature variance.

[0116] In this way, the selected features are standardized to ensure comparability between features, enhancing the applicability and stability of the model in different environments.

[0117] Step S703: Verify the accuracy of the first detection model using the second sample index data. When the detection accuracy of the first detection model is greater than the second threshold, determine the first detection model as the preset detection model.

[0118] In the embodiments of the present application, the second threshold can be ninety percent, eighty percent, or any other value. Exemplarily, the second threshold can be set based on actual requirements and application scenarios, and the present application does not make any limitations in this regard.

[0119] In the embodiments of the present application, after obtaining the first detection model, the anomaly detection device will use the second sample index data to verify the accuracy of the first detection model. If the detection accuracy of the first detection model is greater than the second threshold, determine the first detection model as the preset detection model. Among them, the first sample index data is the index data in the sample index dataset.

[0120] Step S704: Perform anomaly detection on the target industrial system using the preset detection model.

[0121] In the embodiments of the present application, after determining the preset detection model, the anomaly detection device can perform anomaly detection on the target industrial system based on the preset detection model. It should be noted that the obtained real-time data can include the index data corresponding to at least one feature.

[0122] In this way, the input data of the preset detection model obtained through training is related to at least one feature, which can improve the accuracy of anomaly detection.

[0123] In some embodiments, as Figure 8 shown, the anomaly detection device may further perform the following steps S801 to S804:

[0124] Step S801: When the detection accuracy of the first detection model is not greater than the second threshold, select multiple features from at least one metric feature and at least one fusion feature; the number of features of the multiple features is greater than the number of features of the at least one feature.

[0125] In the embodiments of the present application, if the detection accuracy of the first detection model is not greater than the second threshold, it indicates that the detection model needs to be retrained. In this way, multiple features are selected from at least one metric feature and at least one fusion feature, and the number of the multiple features is greater than the number of features of the at least one feature.

[0126] Exemplarily, if the number of at least one metric is 20 and the number of at least one feature is 10, then at this time, the number of multiple features can be 11 and above, with an upper limit of 20.

[0127] Step S802: Re-establish a second initial detection model based on the multiple features.

[0128] In the embodiments of the present application, the anomaly detection device may re-establish a second initial detection model based on the multiple features.

[0129] Step S803: Use the first sample metric data to train the second initial detection model to obtain a second detection model.

[0130] In the embodiments of the present application, the anomaly detection device uses the first sample metric data to train the second initial detection model. The first sample metric data used here is the same as that for training the first initial detection model, but it is the metric data corresponding to the multiple features, while the metric data used for training the first initial detection model is the metric data corresponding to at least one feature.

[0131] Step S804: Use the second sample metric data to verify the accuracy of the second detection model again until the detection accuracy of the established detection model is greater than the second threshold, and obtain a preset detection model.

[0132] In an embodiment of the present application, the anomaly detection device uses the second sample index data to verify the accuracy of the second detection model until the detection accuracy of the established detection model is greater than the second threshold value. That is to say, if the detection accuracy of the second detection model is greater than the second threshold value, the second detection model will be determined as the preset detection model to perform anomaly detection on the target industrial system.

[0133] In an embodiment of the present application, if the detection accuracy of the second detection model is less than the second threshold value, features will continue to be selected from at least one index feature and at least one fusion feature. The number of features selected here will be greater than the number of multiple features, that is, the number of features used in each training is increased compared to the previous time, until the optimal preset detection model is obtained.

[0134] Exemplarily, the number of features set can be (10, len(features)+1), that is, one more feature is used in each training until the optimal preset detection model is obtained.

[0135] In one embodiment, as Figure 9 shown, when the anomaly detection device executes "performing anomaly detection on the target industrial system according to at least one feature" in the above step S103, the following steps S901 to S905 may further be included:

[0136] Step S901: Starting from the preset quantity as the initial value, with the preset interval number as the step size, and the number of at least one index as the final value, select the corresponding number of features from at least one index feature and at least one fusion feature based on the sorting result to obtain multiple feature groups containing different numbers of features.

[0137] In an embodiment of the present application, if the preset quantity is 5, the preset interval number is 1, and the number of at least one indicator is 10, then, based on the sorting result, the selected quantities are 5, 6, 7, 8, 9, 10, and multiple feature groups with different quantities are obtained. Exemplarily, the sorting results of at least one indicator feature and at least one fusion feature are: water production flow rate, water production conductivity, concentrated water flow rate, heat exchanger outlet temperature, water production rate, desalination rate, inlet water pressure, second-stage pressure difference, water production pressure, concentrated water pressure. Then, the obtained multiple feature groups are: {water production flow rate, water production conductivity, concentrated water flow rate, heat exchanger outlet temperature, water production rate}, {water production flow rate, water production conductivity, concentrated water flow rate, heat exchanger outlet temperature, water production rate, desalination rate}, {water production flow rate, water production conductivity, concentrated water flow rate, heat exchanger outlet temperature, water production rate, desalination rate, inlet water pressure}, {water production flow rate, water production conductivity, concentrated water flow rate, heat exchanger outlet temperature, water production rate, desalination rate, inlet water pressure, second-stage pressure difference}, {water production flow rate, water production conductivity, concentrated water flow rate, heat exchanger outlet temperature, water production rate, desalination rate, inlet water pressure, second-stage pressure difference, water production pressure}, {water production flow rate, water production conductivity, concentrated water flow rate, heat exchanger outlet temperature, water production rate, desalination rate, inlet water pressure, second-stage pressure difference, water production pressure, concentrated water pressure}.

[0138] In an embodiment of the present application, the preset quantity may also be 10, 15, or other values less than the number of at least one indicator, and the preset interval number may also be 1, 2, or other values. Exemplarily, both the preset quantity and the preset interval number can be set based on actual requirements and application scenarios, and the present application does not limit this.

[0139] Step S902: Based on each feature group in the multiple feature groups, construct a corresponding initial detection model respectively.

[0140] In an embodiment of the present application, the anomaly detection device constructs a corresponding initial detection model based on each feature group in the multiple feature groups. If, as in step S901, there are 6 feature groups, then 6 initial detection models will be constructed.

[0141] Step S903: For each feature group, input the first sample indicator data into the corresponding initial detection model for training, and use the second sample indicator data to evaluate the trained detection model obtained by training to obtain a corresponding evaluation value.

[0142] In an embodiment of the present application, in each feature group, the anomaly detection device uses the first sample indicator data to train the corresponding initial detection model, and uses the second sample indicator data to verify the accuracy of the trained detection model obtained by training to obtain the corresponding model accuracy and recall rate. Among them, the first sample indicator data and the second sample data are included in the sample indicator data set.

[0143] In an embodiment of the present application, when the anomaly detection device inputs the first sample index data into the corresponding initial detection model for training, the features will be standardized. For an exemplary standardization method, refer to the formula (6) shown above.

[0144] In an embodiment of the present application, each feature group corresponds to a trained detection model.

[0145] In an embodiment of the present application, for each trained detection model, based on the corresponding model accuracy and recall rate, a corresponding evaluation value is determined. For an exemplary determination method, refer to formula (7):

[0146]

[0147] Where F1 is the evaluation value, precision is the model accuracy, and recall is the recall rate.

[0148] Step S904: Based on the model accuracy corresponding to each feature group, select the trained detection model with the highest evaluation value from the trained detection models corresponding to different feature groups, and determine it as the preset detection model.

[0149] In an embodiment of the present application, the anomaly detection device selects the trained detection model with the highest evaluation value from the trained detection models corresponding to different feature groups based on the model accuracy corresponding to each feature group, and determines it as the preset detection model. For an exemplary selection method, the maximum function Max(F1) can be used.

[0150] Step S905: Use the preset detection model to perform anomaly detection on the target industrial system.

[0151] In an embodiment of the present application, the anomaly detection device can use the preset detection model to perform anomaly detection on the target industrial system.

[0152] Exemplarily, the algorithm for determining the preset detection model can be:

[0153] For num_features in range(10,len(features)+1):

[0154] prediction=lssvm(num_features)

[0155] F1=2*(precision*recall) / (precesion+recall)

[0156] END

[0157] Max(F1);

[0158] Among them, F1 is the evaluation value.

[0159] In an embodiment of the present application, as Figure 10 shown, when the anomaly detection device executes the above-mentioned step S905, the following steps S1001 to step S1003 may also be executed:

[0160] Step S1001: Obtain a real-time index data set corresponding to at least one index.

[0161] In an embodiment of the present application, after the anomaly detection device obtains the real-time index data set corresponding to at least one index, it will perform data preprocessing on the real-time index data set. The exemplary preprocessing methods are discussed above and will not be elaborated here.

[0162] Step S1002: Input the real-time index data set into a preset detection model, and use the preset detection model to perform anomaly detection on the target industrial system based on the real-time index data set to obtain a detection result.

[0163] In an embodiment of the present application, by inputting the real-time index data set into a preset detection model and using the preset detection model to perform anomaly detection on the target industrial system, a detection result can be obtained. It should be noted that if the preset detection model needs to fuse features, then the real-time index data needs to be used to extract the fused features by means of PCA dimensionality reduction before being input into the model. Of course, the dimensionality reduction operation can also be performed by the model.

[0164] Step S1003: When the detection result indicates that there is an abnormal state currently, output a warning message to take corresponding processing measures for the target industrial system based on the warning message.

[0165] In an embodiment of the present application, the warning message may be the abnormal RO membrane number and the cause of the abnormality. Then, the warning message can be provided to professionals, or corresponding processing measures may be stored for different warning messages. In this way, corresponding processing measures can be obtained based on the warning message to take corresponding processing measures for the target industrial system.

[0166] In this way, the anomaly detection of the target industrial system can be realized.

[0167] Exemplarily, as Figure 11 shown, a flow diagram of an exemplary anomaly detection method is provided. It includes three parts: Data Source, Algorithm Merging, and Result, and exemplarily includes the following steps S1101 to step S1103:

[0168] Step S1101: Data collection.

[0169] Here, in the industrial wastewater treatment system, substances are detected through membrane sensors and indicators (MembraneIndicator date by sensor) for real-time data (such as flow rate (desalting rate), pressure (pressure difference), temperature (Temperature), etc.). And preprocess the real-time data. For example, preprocess the data to remove noise and handle missing values to ensure it is suitable for subsequent algorithm processing.

[0170] Step S1102: Algorithm combination.

[0171] The algorithm combination in step S1102 can be split into step S11021: Feature processing and step S11022: Model prediction.

[0172] Step S11021: Feature processing.

[0173] Here, redundancy detection of features (Feature Redundancy Detection) can be performed. Refer to the above steps S201 to S203. If the first condition is met, perform the following steps: a. Include PCA dimensionality reduction: Reduce the dimensionality of the original features and extract 5 highly correlated features to reduce feature redundancy. This process retains most of the information in the new low-dimensional feature space. B. mRMR feature selection: Combine the features extracted by PCA with the original features and use the minimum redundancy maximum relevance (mRMR) (Max Relevance, Min Redundancy) method to screen out the most representative features (Feature Selection). In this process, maximize the correlation with the target variable while minimizing the redundancy between features. Finally, perform feature standardization on the selected features.

[0174] Step S11022: Model prediction.

[0175] Here, input the feature-standardized data (Feature input) into the LSSVM classification model.

[0176] Step S1103: Training and prediction of the LSSVM classification model.

[0177] Here, in the LSSVM classification model, the RBF function (LSSVM RBF Kernel) is introduced. First, historical data is used for model training, parameter optimization, and model evaluation. Based on the selected features, a least squares support vector machine (LSSVM) model is constructed. The model is trained using historical data and cross-validated to optimize the model parameters. After training, the model is used to predict anomalies in new data.

[0178] Step S1104, Anomaly Detection and Warning System.

[0179] Here, the LSSVM model is integrated into the real-time monitoring system to implement continuous data stream monitoring. Once an anomaly is detected, the system will issue an alarm (Anomally Monitoring and Alerting) and record the relevant data for subsequent analysis and decision-making.

[0180] The embodiments of this application can address the common problem of scarce anomaly samples in industrial membrane anomaly detection. Through the optimization of feature selection, the interference of noise data is reduced, and the learning effect of the model under small data sets is improved. The generalization ability of LS-SVM also helps to achieve better performance with limited training data, ensuring the stability and accuracy of the detection model in practical applications.

[0181] Exemplarily, an example is provided:

[0182] Use case: Industrial RO membrane-related metrics;

[0183] Data: Metric data from March 26, 2023 to February 10, 2024, including: 'inlet flow rate', 'product water flow rate', 'concentrate water flow rate', 'inlet pressure', 'inter-stage pressure', 'concentrate water pressure', 'product water pressure', 'product water conductivity', 'product water rate', 'first stage differential pressure','second stage differential pressure','salt rejection rate', 'RO membrane number', 'heat exchanger outlet temperature'.

[0184] Train set: Test Set = 7:3;

[0185] PoC results are shown in Table 1:

[0186] Table 1

[0187]

[0188] As shown in Table 2, when the LSSVM algorithm based on mRMR feature engineering is added, in the anomaly detection of industrial RO membranes, the precision reaches 90%, an 8% improvement compared to the prediction of a single LSSVM model. The F1-score (evaluation value) reaches 95%, a 5% improvement compared to the prediction of a single LSSVM model.

[0189] Table 2

[0190]

[0191] The comparison of the confusion matrix results (LSSVM VS mRMR-LSSVM) is as follows:

[0192]

[0193] It can be seen that the accuracy of using the prediction and detection model of this application is better.

[0194] The embodiment of this application provides an anomaly detection method, which includes: obtaining at least one metric related to the target industrial system and a sample metric dataset corresponding to the at least one metric; extracting at least one metric feature corresponding to the at least one metric from the sample metric dataset, and when the at least one metric feature meets the first condition, performing dimensionality reduction on the at least one metric feature to obtain at least one fused feature; selecting at least one feature from the at least one metric feature and the at least one fused feature according to a preset selection rule, and performing anomaly detection on the target industrial system based on the at least one feature. The anomaly detection method provided by this application can improve the accuracy of anomaly detection by fusing important features of at least one metric feature, and then determining at least one feature for anomaly detection from a richer feature set of the at least one fused feature and the at least one metric feature. While ensuring the simplification of the feature set, anomaly detection is performed based on at least one feature of high importance.

[0195] The embodiment of this application provides an anomaly detection device 12, as Figure 12 shown, including:.

[0196] An acquisition module 1201, configured to obtain at least one metric related to the target industrial system and a sample metric dataset corresponding to the at least one metric;

[0197] A processing module 1202, configured to extract at least one metric feature corresponding to the at least one metric from the sample metric dataset, and when the at least one metric feature meets the first condition, perform dimensionality reduction on the at least one metric feature to obtain at least one fused feature;

[0198] The detection module 1203 is configured to select at least one feature from at least one metric feature and at least one fusion feature according to a preset selection rule, and perform anomaly detection on the target industrial system based on the at least one feature.

[0199] In an embodiment of the present application, the processing module 1201 is further configured to calculate the redundancy of each metric feature in the at least one metric feature to obtain at least one feature redundancy value; determine the feature redundancy values greater than a preset value among the at least one feature redundancy value as target redundancy values, and obtain at least one target redundancy value; and determine that the at least one metric feature satisfies the first condition when the number of the at least one target redundancy value is greater than a first threshold.

[0200] In an embodiment of the present application, the detection module 1203 is further configured to perform minimum redundancy calculation on each feature in the at least one metric feature and the at least one fusion feature to obtain a plurality of minimum redundancy values; perform maximum correlation calculation on each feature in the at least one metric feature and the at least one fusion feature to obtain a plurality of maximum correlation values; and select at least one feature from the at least one metric feature and the at least one fusion feature according to the plurality of minimum redundancy values and the plurality of maximum correlation values.

[0201] In an embodiment of the present application, the detection module 1203 is further configured to determine the importance parameter of each feature based on the plurality of minimum redundancy values and the plurality of maximum correlation values; and select at least one feature from the at least one metric feature and the at least one fusion feature according to the importance parameter of each feature.

[0202] In an embodiment of the present application, the detection module 1203 is further configured to set a penalty coefficient for the corresponding minimum redundancy value for each feature to obtain a corresponding penalty redundancy value; and determine the difference between the corresponding maximum correlation and the penalty redundancy value as the corresponding importance parameter for each feature.

[0203] In an embodiment of the present application, the detection module 1203 is further configured to sort the at least one metric feature and the at least one fusion feature according to the importance parameter of each feature to obtain a sorting result; sequentially select a preset number of features based on the sorting result, and determine the selected features as the at least one feature.

[0204] In an embodiment of the present application, the detection module 1203 is further configured to use a preset quantity as the initial value, a preset interval number as the step size, and the quantity of at least one metric as the final value, and select corresponding quantities of features from at least one metric feature and at least one fusion feature based on the sorting result to obtain multiple feature groups with different quantities; respectively construct corresponding initial detection models based on each feature group in the multiple feature groups; for each feature group, input the first sample metric data into the corresponding initial detection model for training, and use the second sample metric data to evaluate the trained detection model obtained by training to obtain the corresponding evaluation value; select the trained detection model with the evaluation value from the trained detection models corresponding to different feature groups based on the model accuracy corresponding to each feature group, and determine it as the preset detection model; use the preset detection model to perform anomaly detection on the target industrial system.

[0205] In an embodiment of the present application, the detection module 1203 is further configured to obtain a real-time metric data set corresponding to at least one metric; input the real-time metric data set into the preset detection model, and use the preset detection model to perform anomaly detection on the target industrial system based on the real-time metric data set to obtain a detection result; in the case where the detection result indicates the current existence of an abnormal state, output a warning message to take corresponding processing measures for the target industrial system based on the warning message.

[0206] In an embodiment of the present application, the preset detection model is a least squares support vector machine model including a radial basis function kernel.

[0207] An embodiment of the present application provides an anomaly detection device 13, as Figure 13 shown, the anomaly detection device includes: a processor 1301, a memory 1302, and a communication bus 1303;

[0208] The communication bus 1303 is used to implement a communication connection between the processor 1301 and the memory 1302;

[0209] The processor 1301 is configured to execute a computer program stored in the memory 1302 to implement the above anomaly detection method.

[0210] An embodiment of the present application provides an anomaly detection device, which acquires at least one metric related to a target industrial system and a sample metric data set corresponding to the at least one metric; extracts at least one metric feature corresponding to the at least one metric from the sample metric data set, and when the at least one metric feature satisfies a first condition, performs dimensionality reduction on the at least one metric feature to obtain at least one fused feature; selects at least one feature from the at least one metric feature and the at least one fused feature according to a preset selection rule, and performs anomaly detection on the target industrial system according to the at least one feature. The anomaly detection device provided by the present application fuses important features of at least one metric feature, and then determines at least one feature for anomaly detection from a richer feature set of the at least one fused feature and the at least one metric feature. While ensuring the simplification of the feature set, anomaly detection is performed based on at least one feature of high importance, which can improve the accuracy of anomaly detection.

[0211] An embodiment of the present application provides a computer-readable storage medium, which stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to implement the above-mentioned anomaly detection method. The computer-readable storage medium may be a volatile memory, such as a random access memory (RAM); or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); it may also be a respective device including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0212] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) containing computer-usable program codes.

[0213] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a means for implementing the functions specified in one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1 or a means for implementing the functions specified in one or more blocks.

[0214] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction means that implements the functions specified in one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1 or a means for implementing the functions specified in one or more blocks.

[0215] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1 or a means for implementing the functions specified in one or more blocks.

[0216] As mentioned above, the above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An anomaly detection method, the method comprising: Obtaining at least one metric related to a target industrial system and a sample metric dataset corresponding to the at least one metric; Extracting at least one metric feature corresponding to the at least one metric from the sample metric dataset, and when the at least one metric feature satisfies a first condition, performing dimensionality reduction on the at least one metric feature to obtain at least one fused feature; Selecting at least one feature from the at least one metric feature and the at least one fused feature according to a preset selection rule, and performing anomaly detection on the target industrial system according to the at least one feature.

2. The method according to claim 1, the method further comprising: Calculating redundancy for each of the at least one metric feature to obtain at least one feature redundancy value; Determining, as target redundancy values, the feature redundancy values greater than a preset value among the at least one feature redundancy value, to obtain at least one target redundancy value; When the number of the at least one target redundancy value is greater than a first threshold, determining that the at least one metric feature satisfies the first condition.

3. The method according to claim 1 or 2, the step of selecting at least one feature from the at least one metric feature and the at least one fused feature according to a preset selection rule, includes: Performing minimum redundancy calculation on each of the at least one metric feature and the at least one fused feature to obtain a plurality of minimum redundancy values; Performing maximum correlation calculation on each of the at least one metric feature and the at least one fused feature to obtain a plurality of maximum correlation values; Selecting the at least one feature from the at least one metric feature and the at least one fused feature according to the plurality of minimum redundancy values and the plurality of maximum correlation values.

4. The method according to claim 3, the step of selecting the at least one feature from the at least one metric feature and the at least one fused feature according to the plurality of minimum redundancy values and the plurality of maximum correlation values, includes: Determining an importance parameter for each feature based on the plurality of minimum redundancy values and the plurality of maximum correlation values; Selecting the at least one feature from the at least one metric feature and the at least one fused feature according to the importance parameter of each feature.

5. The method according to claim 4, the step of determining an importance parameter for each feature based on the plurality of minimum redundancy values and the plurality of maximum correlation values, includes: For each feature, setting a penalty coefficient for the corresponding minimum redundancy value to obtain a corresponding penalty redundancy value; For each feature, determining the difference between the corresponding maximum correlation and the penalty redundancy value as the corresponding importance parameter.

6. The method according to claim 4, the step of selecting the at least one feature from the at least one metric feature and the at least one fused feature according to the importance parameter of each feature, includes: Sorting the at least one metric feature and the at least one fused feature according to the importance parameter of each feature to obtain a sorting result; Select a preset number of features in sequence based on the sorting result, and determine the selected features as the at least one feature.

7. The method according to claim 6, wherein the performing anomaly detection on the target industrial system according to the at least one feature comprises: Using the preset number as the initial value, the preset number of intervals as the step size, and the number of the at least one metric as the final value, select a corresponding number of features from the at least one metric feature and the at least one fusion feature based on the sorting result to obtain a plurality of feature groups with different numbers of features; Construct corresponding initial detection models respectively based on each of the plurality of feature groups; For each of the feature groups, input the first sample metric data into the corresponding initial detection model for training, and evaluate the trained detection model obtained by training by using the second sample metric data to obtain a corresponding evaluation value; Based on the model accuracy corresponding to each of the feature groups, select the trained detection model with the evaluation value from the trained detection models corresponding to different feature groups and determine it as the preset detection model; Use the preset detection model to perform anomaly detection on the target industrial system.

8. The method according to claim 7, wherein the performing anomaly detection on the target industrial system by using the preset detection model comprises: Obtain a real-time metric data set corresponding to the at least one metric; Input the real-time metric data set into the preset detection model, and use the preset detection model to perform anomaly detection on the target industrial system based on the real-time metric data set to obtain a detection result; In the case where the detection result indicates that an abnormal state exists currently, output a warning message to take corresponding processing measures on the target industrial system based on the warning message.

9. The method according to claim 7 or 8, wherein the preset detection model is a least squares support vector machine model including a radial basis function kernel.

10. An anomaly detection device, comprising: A processor, a memory, and a communication bus; The communication bus is used to implement a communication connection between the processor and the memory; The processor is configured to execute a computer program stored in the memory to implement the anomaly detection method according to any one of claims 1 to 9.

Images