Anomaly detection credibility assessment method and system

Abstract

The application relates to an abnormality detection credibility evaluation method and system, and relates to the intelligent detection field. The method comprises the following steps: receiving to-be-detected data, calculating the signal-to-noise ratio of the to-be-detected data as a data quality index; detecting the to-be-detected data by using an abnormality detection model to obtain a detection result; calculating a multi-layer feature consistency index according to a plurality of abnormality response values; processing the to-be-detected data to obtain a plurality of perturbation data, inputting the plurality of perturbation data into the abnormality detection model for detection to obtain a plurality of perturbation detection results, and calculating a stability index according to the detection result and the plurality of perturbation detection results; obtaining historical abnormality detection results, calculating a historical consistency index according to the detection result and the historical abnormality detection results; and weighting and fusing the data quality index, the multi-layer feature consistency index, the stability index and the historical consistency index to obtain a credibility score, so as to evaluate the credibility of the detection result. The technical effect is that the credibility of the abnormality detection result is evaluated.

Description

Technical Field

[0001] This application relates to the field of intelligent detection, and in particular to a method and system for evaluating the reliability of anomaly detection. Background Technology

[0002] With the continuous evolution of artificial intelligence technology, anomaly detection methods based on deep learning have been widely applied in key fields such as industrial manufacturing, equipment maintenance, and intelligent security. Existing technologies generally rely on deep neural networks to extract high-dimensional data features from normal or abnormal samples, and then use the deviation of the sample to be detected from a predetermined normal distribution to identify and determine abnormal states.

[0003] However, most current anomaly detection systems still stop at outputting a binary conclusion of "whether it is anomaly" and lack a systematic evaluation mechanism for the reliability of the detection results. This shortcoming is particularly prominent in situations such as model failure or data distribution drift—the system cannot perceive the reliability of its own judgment, nor can it trigger early warnings or verifications, leading to a decrease in the accuracy of anomaly detection. Summary of the Invention

[0004] To assess the reliability of anomaly detection results, this application provides an anomaly detection reliability assessment method, system, device, and storage medium.

[0005] Firstly, this application provides a method for evaluating the reliability of anomaly detection, employing the following technical solution:

[0006] Receive the data to be detected and calculate the signal-to-noise ratio of the data to be detected as a data quality indicator; The anomaly detection model is used to detect the data to be detected to obtain the detection result. The detection result includes an anomaly score and anomaly response values ​​output by several network layers in the anomaly detection model. Calculate the multi-layer feature consistency index based on several of the aforementioned abnormal response values; The data to be detected is perturbed to obtain perturbed data. Several perturbed data are input into an anomaly detection model for detection to obtain perturbed detection results. A stability index is calculated based on the detection results and several perturbed detection results. Obtain historical anomaly detection results, combine the current detection results and the historical anomaly detection results to obtain a detection result set, and use the detection result set to calculate the historical consistency index; The credibility score is obtained by weighted fusion of the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index to evaluate the credibility of the detection results.

[0007] Through the above technical solutions, this application comprehensively evaluates the reliability of the detection results from four different dimensions: data quality, consistency of internal model features, anti-disturbance stability, and temporal historical patterns. It expands the original single binary output of "abnormal / normal" into a graded output with a credibility score, providing a quantitative basis for subsequent decision grading, manual review and early warning, and model status monitoring, which significantly improves the reliability and decision security of the anomaly detection system in real industrial environments.

[0008] In one specific implementation scheme, calculating the multi-layer feature consistency index based on a plurality of the aforementioned abnormal response values ​​includes: The anomaly detection model is analyzed to obtain the hierarchical features of several network layers in the anomaly detection model; Retrieve anomaly detection feature requirements, and determine the weight ratio of several network layers in the consistency evaluation based on several hierarchical features and anomaly detection feature requirements; Based on the weighting ratio, a multi-level feature consistency index is obtained by analyzing the variance and standard deviation of several abnormal response values.

[0009] Through the above technical solution, this application realizes the task adaptive configuration of hierarchical weights, enabling the multi-layer feature consistency index to dynamically adjust the degree of attention to each network layer according to different anomaly detection tasks. The smaller the variance and standard deviation, the more consistent the judgment of anomalies is among the layers. The weight ratio ensures that the consistency deviation of key layers is appropriately amplified, thereby improving the representation accuracy and generalization ability of the index in different application scenarios.

[0010] In one specific implementation, the perturbation processing of the data to be detected to obtain perturbed data includes: The sensitivity of the anomaly detection model to several perturbation methods is queried, including image brightness changes, noise addition, geometric transformations, and feature perturbations. Based on the sensitivity, a disturbance strategy is formulated, wherein the disturbance strategy includes disturbance intensity, disturbance type, and disturbance quantity; The data to be detected is perturbed according to several perturbation strategies to obtain perturbed data.

[0011] Through the above technical solution, this application realizes the adaptive formulation of perturbation strategy—applying a small intensity to sensitive perturbation methods can generate effective perturbation, while applying a larger intensity to insensitive perturbation methods to fully test the model stability. This avoids the "under-perturbation" or "over-perturbation" problems caused by a uniform perturbation strategy, and makes the stability index calculated subsequently more accurately reflect the actual performance of the model in the face of various data changes in the real environment.

[0012] In one specific implementation scheme, calculating the stability index based on the detection results and several disturbance detection results includes: The detection results and several disturbance detection results are combined to form a detection result set; A stability index is calculated based on the set of test results. The formula for calculating the stability index is as follows: ; in, Here, P represents the stability index, and P is the set of detection results. The average of all test results. The standard deviation of all test results is given. To prevent extremely small constants with a denominator of 0.

[0013] Through the above technical solution, this application uses the ratio of standard deviation to mean (the idea of ​​coefficient of variation) to eliminate the influence of the absolute value dimension of different detection results, making the stability index a dimensionless relative measure that can be fairly compared between anomaly scores of different magnitudes. The smaller the ratio, the more concentrated the detection results are under each perturbation and the higher the model stability. Conversely, the larger the ratio, the more volatile the detection results are and the poor the stability. This provides an objective, quantifiable and cross-task comparable mathematical tool for stability assessment.

[0014] In one specific implementation scheme, calculating the historical consistency index using the detection result set includes: The detection result set is assigned a time decay weight in chronological order; The historical consistency index is calculated based on the detection result set. The formula for calculating the historical consistency index is as follows: ; in, This represents the historical consistency indicator, where N is the number of detection results. This is the result of the i-th detection. Let I() be the result of the (i+1)th detection, and let I() be the indicator function.

[0015] Through the above technical solution, this application uses a time decay weighting mechanism to make the historical consistency index focus more on the model behavior patterns reflected by recent test results, effectively reducing the interference of outdated historical data on the current credibility assessment. This allows the index to sensitively capture the gradual decline of model performance or the slow drift of data distribution, providing a more timely and accurate basis for subsequent equipment maintenance early warning and model update decisions.

[0016] In a specific feasible implementation, the weighted fusion of the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index to obtain the credibility score includes: The credibility of anomaly detection is evaluated and saved by fusing the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index according to the preset index weight ratio. The formula for calculating the credibility score is as follows: ; in, , , , The preset indicator weights are all non-negative numbers and their sum is 1; For data quality indicators, As a multi-layer feature consistency index, As a stability indicator, This is a historical consistency indicator.

[0017] Through the above technical solution, this application combines four-dimensional indicators into an intuitive numerical credibility score, realizing the efficient integration and dimensionality reduction output of multi-dimensional information. This allows subsequent decision-making processes to directly perform automated logical judgments based on the score (such as threshold comparison, trend monitoring, etc.). At the same time, the preset weight ratio allows the fusion framework to flexibly adjust the priority of each indicator according to the security requirements of different application scenarios—for example, increasing the weight of data quality indicators in a noisy environment and increasing the weight of stability indicators in the early stage of model deployment, thus taking into account both the versatility of the solution and the customizability of the scenario.

[0018] In a specific implementation scheme, after evaluating the credibility of anomaly detection by fusing the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index according to a preset index weight ratio to obtain a credibility score and saving it, the scheme further includes: The credibility score is analyzed based on the preset classification criteria; If the credibility score is within the first credibility range of the preset division criteria, the detection result is determined to be credible, and the detection result is output. If the credibility score is within the second credibility range of the preset division criteria, the detection result is determined to be unreliable, and the anomaly detection model is used to re-detect the data to be detected. If the credibility score falls within the third credibility range of the preset classification criteria, the detection result is determined to be unreliable, and a manual review notification is output.

[0019] Through the above technical solution, this application realizes a hierarchical decision-making mechanism based on credibility scoring. High-credibility detection results are directly adopted to ensure efficiency, medium-credibility results are re-detected to attempt correction and reduce false alarm rate, and low-credibility results are directly transferred to manual review to ensure safety. Through differentiated hierarchical handling, a reasonable balance is achieved between automation efficiency and manual safety, while avoiding the decision-making risks caused by blindly outputting erroneous detection results in the case of low credibility.

[0020] In a specific implementation scheme, after evaluating the credibility score by fusing the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index according to a preset index weight ratio, the method further includes: Several historical credibility scores are obtained, and the time trend of credibility scores is analyzed based on the historical credibility scores and the credibility scores. When the time change trend continues to decline within a preset time period, a device maintenance notification is issued.

[0021] Through the above technical solution, this application expands the credibility score from a single criterion to a time-series monitoring indicator, enabling the system to have predictive maintenance capabilities based on the credibility degradation trend. The continuous decline in the credibility score can serve as an early warning signal for equipment anomalies or model failures, triggering maintenance actions before the system completely fails. This transforms post-event remediation into pre-event prevention, significantly reducing the risk of missed detections and false alarms caused by the failure of the anomaly detection system itself, and improving the operational reliability and maintenance timeliness of the entire detection system.

[0022] In a specific implementation scheme, after evaluating the credibility score by fusing the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index according to a preset index weight ratio, the method further includes: Obtain the true annotation information of the detection results, and calculate the prediction deviation of the credibility score based on the comparison results between the true annotation information and the detection results; When the prediction deviation exceeds the preset deviation threshold, the indicator weight ratio is updated and saved according to the prediction deviation.

[0023] Through the above technical solution, this application introduces an online weight self-learning mechanism based on real feedback, which enables the weight ratio to be optimized and adjusted in a closed loop according to the actual detection performance. When the credibility score continuously overestimates or underestimates the actual credibility of the detection results, the system can automatically identify the deviation and correct the fusion weight of each indicator. This allows the credibility assessment model to continuously evolve with environmental changes after actual deployment, maintaining the long-term applicability and evaluation accuracy of the fusion framework in different stages and scenarios.

[0024] Secondly, this application provides an anomaly detection reliability assessment system, which is based on an anomaly detection model and adopts the following technical solution: The system includes: The data receiving module is used to receive the data to be detected and calculate the signal-to-noise ratio of the data to be detected as a data quality indicator. The detection result module is used to detect the data to be detected using the anomaly detection model to obtain the current detection result. The current detection result includes an anomaly score and anomaly response values ​​output by several network layers in the anomaly detection model. The feature consistency calculation module is used to calculate a multi-layer feature consistency index based on several of the aforementioned abnormal response values. The stability calculation module is used to perturb the data to be detected to obtain several disturbance data, input several disturbance data into the anomaly detection model for detection to obtain several disturbance detection results, and calculate the stability index based on the detection results and several disturbance detection results. The historical consistency calculation module is used to obtain historical anomaly detection results, combine the current detection results and the historical anomaly detection results to obtain a detection result set, and use the detection result set to calculate the historical consistency index. The credibility assessment module is used to weightedly fuse the data quality indicators, the multi-layer feature consistency indicators, the stability indicators, and the historical consistency indicators to obtain a credibility score, thereby assessing the credibility of the detection results.

[0025] In summary, this application has the following beneficial technical effects: This application comprehensively evaluates the reliability of detection results from four different dimensions: data quality, consistency of internal model features, stability against disturbances, and temporal historical patterns. It expands the original single binary output of "abnormal / normal" into a graded output with a credibility score, providing a quantitative basis for subsequent decision grading, manual review and early warning, and model status monitoring, which significantly improves the reliability and decision security of the anomaly detection system in real industrial environments. Attached Figure Description

[0026] Figure 1 This is a flowchart of an anomaly detection reliability assessment method in an embodiment of this application.

[0027] Figure 2 This is a flowchart illustrating a method for evaluating the reliability of anomaly detection in an embodiment of this application.

[0028] Figure 3 This is a structural block diagram of an anomaly detection reliability evaluation method in an embodiment of this application.

[0029] Figure reference numerals: 301, Data receiving module; 302, Detection result module; 303, Feature consistency calculation module; 304, Stability calculation module; 305, Historical consistency calculation module; 306, Credibility assessment module. Detailed Implementation

[0030] The following is in conjunction with the appendix Figures 1-3 This application will be described in further detail.

[0031] This application discloses an anomaly detection reliability evaluation method, which is used to evaluate the reliability of anomaly detection results.

[0032] With the continuous evolution of artificial intelligence technology, anomaly detection methods based on deep learning have been widely applied in key fields such as industrial manufacturing, equipment maintenance, and intelligent security. Existing technologies generally rely on deep neural networks to extract high-dimensional data features from normal or abnormal samples, and then use the deviation of the sample to be detected from a predetermined normal distribution to identify and determine abnormal states.

[0033] The following methods are commonly used in existing technologies for anomaly detection: Currently, the main technical solutions include the following categories: I. Anomaly Detection Methods Based on Supervised Learning This method trains a deep learning classification model by collecting a large number of labeled samples, including normal and abnormal samples. During the detection process: input is the data to be detected; model output includes: anomaly category, classification probability, and anomaly score.

[0034] This method has high detection accuracy, but it relies on a large number of labeled anomaly samples. In real-world industrial scenarios, the limited number of anomaly types and their low probability of occurrence make it difficult to obtain anomaly samples.

[0035] II. Anomaly Detection Methods Based on Single-Class Learning This method primarily utilizes normal samples to establish a normal data distribution model. The training phase includes: 1. Obtaining normal samples; 2. Extracting features; 3. Establishing a normal feature space. The detection phase involves calculating the distance between the sample to be detected and the normal feature space. Examples include: feature distance; reconstruction error; and similarity metrics.

[0036] When the deviation exceeds a threshold, it is considered an anomaly. This method reduces the need for abnormal samples and is therefore widely used in industrial inspection.

[0037] III. Anomaly Scoring Method Based on Deep Feature Representation With the development of deep neural networks, some methods utilize features from intermediate layers of the network as a basis for anomaly detection. Shallow texture features, mid-level structural features, and high-level semantic features can be extracted using convolutional neural networks, and then anomaly scores can be calculated.

[0038] However, most current anomaly detection systems still stop at outputting a binary conclusion of "whether it is anomaly" and lack a systematic evaluation mechanism for the reliability of the detection results. This shortcoming is particularly prominent in situations such as model failure or data distribution drift—the system cannot perceive the reliability of its own judgment, nor can it trigger early warnings or verifications, leading to a decrease in the accuracy of anomaly detection.

[0039] Therefore, this application proposes an anomaly detection credibility assessment method, which is used to assess the credibility of anomaly detection results.

[0040] To facilitate understanding, the following explanations are provided for some key terms: An anomaly detection model is a computational model used to identify anomalous patterns or outliers in data. It is usually built based on machine learning or deep learning techniques and can learn normal behavior patterns from large amounts of data and identify data points that deviate from these patterns.

[0041] The data to be detected refers to the input data that needs to be analyzed and judged for anomalies. It can be in various forms such as images, sensor readings, and network traffic logs, and is the starting point of the anomaly detection process.

[0042] Signal-to-noise ratio (SNR) is a metric that measures the relationship between signal strength and background noise intensity. A higher SNR generally indicates better data quality and a clearer signal. In anomaly detection, SNR serves as an important parameter for evaluating the reliability of input data.

[0043] Data quality metrics are parameters used to quantify the quality of input data, reflecting its performance in terms of completeness, accuracy, and consistency. In this application, the signal-to-noise ratio (SNR) is used as a data quality metric.

[0044] The detection result refers to the judgment information output by the anomaly detection model after processing the input data to be detected. It usually includes an anomaly score, which is used to quantify the degree of data anomaly.

[0045] Anomaly score is a numerical value output by an anomaly detection model, representing the degree to which the data deviates from the normal pattern. Generally, the higher the anomaly score, the greater the likelihood that the data is considered anomaly.

[0046] Anomaly response values ​​refer to intermediate features or activation values ​​generated by different network layers within an anomaly detection model when processing the data to be detected. These values ​​reflect the representation of the data at different levels of abstraction within the model and are of great significance for analyzing the internal working mechanism of the model.

[0047] Network layers refer to modules or layers in a deep learning model that represent different processing stages. Each layer is responsible for extracting features from different levels of data, from low-level features (such as edges and textures) to high-level semantic features.

[0048] The multi-layer feature consistency index is used to evaluate the degree of consistency between the anomaly response values ​​output by different network layers within an anomaly detection model. It reflects whether the model's perception of the same anomaly pattern is coordinated and consistent at different levels of abstraction.

[0049] Perturbation data refers to data samples generated by applying specific perturbations (such as adding noise, changing brightness, etc.) to the original data to be detected. These perturbations are designed to simulate various changes or disturbances that the data may encounter in practical applications.

[0050] The disturbance detection result refers to the detection output obtained after inputting disturbance data into the anomaly detection model. By comparing the original detection result with the disturbance detection result, the robustness of the model can be evaluated.

[0051] Stability metrics are used to measure the sensitivity of anomaly detection models to minor perturbations in the input data. They reflect the model's ability to maintain consistent detection results when faced with data changes.

[0052] Historical anomaly detection results refer to records generated by anomaly detection models when detecting similar data over a past period. This historical data provides a reference for assessing the trend and consistency of current detection results.

[0053] Historical consistency metrics are used to assess the degree of consistency between detection results. They help determine whether current detection results are consistent with the model's historical performance, thereby identifying potential model drift or changes in data distribution.

[0054] The credibility score is a comprehensive score that quantifies the reliability of anomaly detection results by integrating multiple dimensions of indicators (such as data quality, internal model consistency, model stability, and historical performance). This score aims to provide users with an intuitive judgment on whether the detection results are trustworthy.

[0055] like Figure 1 As shown, the method includes: S10: Receive the data to be detected and calculate the signal-to-noise ratio of the data to be detected as a data quality indicator.

[0056] Specifically, such as Figure 2 The diagram shown is a detailed flowchart of this application. The process involves receiving raw data streams (i.e., the data to be detected) from sensors, image acquisition devices, or network interfaces. Statistical methods are used to obtain the signal-to-noise ratio (SNR) of the data by analyzing the power ratio of the signal components to the noise components in the data. For example, for image data, the SNR can be approximated by calculating the ratio of the average pixel value to the standard deviation of the pixel value in the image region.

[0057] S20, the anomaly detection model is used to detect the data to be detected to obtain the detection result. The detection result includes the anomaly score and the anomaly response values ​​output by several network layers in the anomaly detection model.

[0058] Specifically, anomaly detection models can be pre-trained deep neural network models, such as those based on autoencoders, generative adversarial networks, or support vector machines. When the data to be detected is input into the model, it outputs a quantified anomaly score, representing the degree of data anomaly. Simultaneously, during processing, the various network layers within the model generate intermediate features or activation values, which are extracted as anomaly response values. For example, in a convolutional neural network, the output of each convolutional or fully connected layer can be considered an anomaly response value for that network layer. These anomaly response values ​​reflect the representation of the data at different levels of abstraction within the model.

[0059] S30, calculate the multi-level feature consistency index based on several abnormal response values.

[0060] Specifically, statistical analysis is performed on the abnormal response values ​​at each level, calculating their mean and variance, and then comparing the differences of these statistics across different levels. For example, if an abnormal sample is identified as having anomalous features at a shallow level of the model, but exhibits features similar to normal samples at deeper levels, it may indicate inconsistencies within the model.

[0061] S40, the disturbance processing of the data to be detected is used to obtain the disturbance data, the disturbance data is input into the anomaly detection model for detection to obtain the disturbance detection result, and the stability index is calculated based on the detection result and the disturbance detection result.

[0062] Specifically, perturbation can be implemented in various ways. For example, random noise can be added to the original data, or slight geometric transformations (such as rotation or translation) can be applied to the image data. This perturbed data is then re-input into the anomaly detection model to obtain corresponding perturbation detection results. Stability metrics are calculated by comparing the differences between the original detection results and these perturbed detection results. For example, the mean absolute error or mean squared error between the original anomaly score and all perturbed anomaly scores can be calculated. If the model exhibits large fluctuations in results to small perturbations, its stability is low.

[0063] S50: Obtain historical anomaly detection results, combine the current detection results and historical anomaly detection results to obtain a detection result set, and use the detection result set to calculate the historical consistency index.

[0064] Specifically, the historical consistency index can be calculated by comparing the similarity of all test results, including the current test result. In actual industrial testing, data from multiple consecutive periods are temporally correlated. Trend analysis is performed on historical test results to determine whether the current test result deviates from the historical trend. For example, if 10 consecutive tests show abnormalities, the reliability is significantly higher than if abnormal and normal results alternate.

[0065] S60 uses a weighted fusion of data quality indicators, multi-layer feature consistency indicators, stability indicators, and historical consistency indicators to obtain a credibility score, which is used to assess the credibility of the detection results.

[0066] Specifically, the fusion process can be achieved by assigning a preset weight to each indicator, then multiplying each indicator by its weight and summing the results. For example, the importance of each indicator can be assessed based on the actual application scenario, and the corresponding weights can be manually set. This credibility score is a comprehensive numerical value that fully reflects the reliability of anomaly detection results, providing a basis for subsequent decision-making.

[0067] This application comprehensively evaluates the reliability of detection results from four different dimensions: data quality, consistency of internal model features, stability against disturbances, and temporal historical patterns. It expands the original single binary output of "abnormal / normal" into a graded output with a credibility score, providing a quantitative basis for subsequent decision grading, manual review and early warning, and model status monitoring, which significantly improves the reliability and decision security of the anomaly detection system in real industrial environments.

[0068] In one embodiment, to evaluate the reliability of anomaly detection results, the step of calculating a multi-layer feature consistency index based on several anomaly response values ​​can be specifically performed as follows: First, the anomaly detection model is analyzed to obtain hierarchical features of several network layers within the model. Specifically, the anomaly detection model is analyzed to obtain structural and functional information of different network layers within the model, i.e., hierarchical features. These hierarchical features may include, but are not limited to, the type (e.g., convolutional layer, fully connected layer, pooling layer, etc.) and purpose of each layer. For example, first-level features are used to describe edges, textures, and local changes; second-level features are used to describe structure and local morphology; and third-level features are used to describe target semantics and high-level information. Then, the anomaly detection feature requirements are retrieved, and the weight ratio of several network layers in the consistency evaluation is determined based on several layer features and anomaly detection feature requirements. Specifically, according to specific anomaly detection requirements, different importance weights are assigned to the anomaly response values ​​of different network layers. Anomaly detection feature requirements can be requirements for sensitivity to specific types of anomalies, requirements for the degree of feature abstraction, or requirements for model interpretability, etc. For example, for image anomaly detection, shallow features may focus more on texture and local structure, while deep features may focus more on semantics and global structure. According to the anomaly detection feature requirements, shallow features can be assigned higher weights. Next, based on the weighted allocation, a multi-layer feature consistency index is obtained by analyzing the variance and standard deviation of several anomaly response values. Specifically, after determining the weight allocation for each network layer, these weights are applied to the anomaly response values ​​output by each layer. By analyzing the variance and standard deviation of the weighted anomaly response values, the degree of consistency of features at different layers with respect to the same input data can be quantified. Smaller variance or standard deviation usually means that different layers are more consistent in their judgment of anomalies, thus indicating a higher reliability of the detection results. If the anomaly detection model network has three layers and the weights are uniformly distributed across all layers, the formula for calculating the multi-layer feature consistency index can be: ; C1 is the multi-layer feature consistency index, and the higher the index, the more consistent the judgments are across different layers. A1, A2, and A3 are the network layers of the anomaly detection model.

[0069] Another calculation formula (which focuses more on determining whether different network layers point in the same abnormal direction) is:

[0070] C1 is the consistency index of multi-layer features, and F1, F2, and F3 are features of different layers.

[0071] This application implements task-adaptive configuration of hierarchical weights, enabling the multi-layer feature consistency index to dynamically adjust the degree of attention to each network layer according to different anomaly detection tasks. The smaller the variance and standard deviation, the more consistent the judgment of anomalies is among the layers. The weight ratio ensures that the consistency deviation of key layers is appropriately amplified, thereby improving the representation accuracy and generalization ability of the index in different application scenarios.

[0072] In one embodiment, to evaluate the reliability of anomaly detection results, the step of perturbing the data to be detected to obtain perturbed data can be specifically performed as follows: First, the sensitivity of the anomaly detection model to various perturbation methods is queried. These perturbation methods include changes in image brightness, noise addition, geometric transformation, and feature perturbation. Specifically, the degree to which the anomaly detection model's detection results (e.g., anomaly score or anomaly response value) change when faced with specific perturbation methods is obtained. This sensitivity can be obtained by testing and analyzing the model in advance. For example, by observing the changing trend of the model's output under different perturbation intensities, or by quantifying the model's response to input changes through gradient analysis, feature importance analysis, and other methods. Perturbation methods include image brightness variation, which refers to adjusting the overall brightness of image data, such as increasing or decreasing pixel values, to simulate the impact of changes in lighting conditions on model performance; noise addition, which refers to introducing random or specific patterned interference into the data, such as Gaussian noise, salt-and-pepper noise, etc., to simulate the impact of sensor noise or data transmission errors on the model; geometric transformation, which refers to performing operations such as rotation, translation, scaling, and cropping on image data to simulate the appearance of the target under different viewpoints, positions, or sizes, and to evaluate the model's robustness to these changes; and feature perturbation, which refers to directly modifying the features extracted from the data, such as increasing or decreasing the value of a certain dimension of the feature vector, or fine-tuning the feature distribution, to evaluate the model's sensitivity to small changes in input features.

[0073] Then, perturbation strategies are formulated based on the sensitivity level. These strategies include perturbation intensity, perturbation type, and perturbation quantity. Specifically, for perturbation methods with high model sensitivity, more refined or challenging strategies can be designed, while for perturbation methods with low sensitivity, relatively milder strategies can be adopted. The perturbation strategy includes perturbation intensity, which refers to the degree or magnitude of the perturbation applied, such as the percentage change in image brightness, the variance of noise, the angle or proportion of geometric transformation, etc.; perturbation type, which refers to the specific perturbation method used, such as brightness change, noise addition, geometric transformation, or feature perturbation, or a combination thereof; and perturbation quantity, which refers to how many different perturbation data samples are generated for a single data sample to be detected. Next, the data to be detected is perturbed according to the perturbation strategy to obtain perturbation data. Specifically, according to the specific perturbation strategy, the data to be detected is used as the original input, and a series of perturbation data samples that are similar to the original data but have specific differences are generated by applying the selected perturbation type, intensity and quantity.

[0074] This application achieves adaptive formulation of perturbation strategy—applying a small intensity to sensitive perturbation methods can generate effective perturbation, while applying a larger intensity to insensitive perturbation methods to fully test model stability. This avoids the "under-perturbation" or "over-perturbation" problems caused by a uniform perturbation strategy, making the stability index calculated subsequently more accurately reflect the actual performance of the model in the face of various data changes in the real environment.

[0075] In one embodiment, the step of calculating a stability index based on the anomaly detection results and the results of minor disturbance detection to assess the reliability of the anomaly detection results can be specifically performed as follows: The set of detection results and the set of detection results for minor disturbances constitute the detection result set; the stability index is calculated based on the detection result set, and the formula for calculating the stability index is: ; in, Here, P represents the stability index, and P is the set of detection results. The average of all test results. The standard deviation of all test results is given. To prevent extremely small constants with a denominator of 0.

[0076] Specifically, the dataset consists of the detection results obtained by the original data to be detected through the anomaly detection model, and the detection results of minor disturbances obtained by the perturbation data after being processed by different perturbation strategies and then processed by the anomaly detection model.

[0077] Statistical analysis was performed on the detection result set P to calculate its mean μ(P) and standard deviation σ(P). The mean μ(P) represents the overall trend of the model's outlier scores when faced with raw data and various perturbations. The standard deviation σ(P) precisely quantifies the dispersion of these outlier scores, i.e., the volatility of the model's output. A stability index was calculated using the following formula: ; in, Here, P represents the stability index, and P is the set of detection results. The average of all test results. The standard deviation of all test results is given. To prevent extremely small constants with a denominator of 0.

[0078] This application utilizes the ratio of standard deviation to mean (the concept of coefficient of variation) to eliminate the influence of the absolute value dimension of different detection results, making the stability index a dimensionless relative measure that can be fairly compared between anomaly scores of different magnitudes. The smaller the ratio, the more concentrated the detection results are under each perturbation and the higher the model stability. Conversely, the larger the ratio, the more volatile the detection results are and the poor the stability. This provides an objective, quantifiable, and cross-task comparable mathematical tool for stability assessment.

[0079] In one embodiment, the step of calculating a historical consistency index using the detection result set to evaluate the reliability of anomaly detection results can be specifically performed as follows: First, the detection result set, including both current and historical anomaly detection results, is assigned time decay weights according to chronological order. Specifically, when evaluating historical data, data closer to the current time is given higher weights, while data further back is given lower weights. This ensures that when constructing the detection result sequence used to calculate the historical consistency index, more recent and valuable data is prioritized, avoiding the undue influence of outdated data on the evaluation results. This allows the historical consistency index to more accurately reflect recent trends and the current state. The historical consistency index is calculated based on the detection result set. The formula for calculating the historical consistency index is as follows: ; in, This represents the historical consistency indicator, where N is the number of detection results. This is the result of the i-th detection. Let I() be the result of the (i+1)th detection, and let I() be the indicator function.

[0080] Specifically, indicator functions exist and If they are the same, return 1; otherwise, return 0. By calculating the proportion of consecutive identical results out of the total number of consecutive pairs, the stability or continuity of the detection results over time can be reflected. The formula for calculating the historical consistency index is: ; in, This represents the historical consistency indicator, where N is the number of detection results. This is the result of the i-th detection. Let I() be the result of the (i+1)th detection, and let I() be the indicator function.

[0081] This application uses a time decay weighting mechanism to make the historical consistency index focus more on the model behavior patterns reflected by recent test results, effectively reducing the interference of outdated historical data on the current credibility assessment. This allows the index to sensitively capture the gradual decline of model performance or the slow drift of data distribution, providing a more timely and accurate basis for subsequent equipment maintenance early warning and model update decisions.

[0082] In one embodiment, to evaluate the credibility of anomaly detection results, the step of weighted fusion of data quality indicators, multi-level feature consistency indicators, stability indicators, and historical consistency indicators to obtain a credibility score can be specifically performed as follows: The credibility of anomaly detection is evaluated and saved by fusing data quality indicators, multi-layer feature consistency indicators, stability indicators, and historical consistency indicators according to the preset indicator weight ratio. The formula for calculating the credibility score is as follows: ; in, , , , The preset indicator weights are all non-negative numbers and their sum is 1; For data quality indicators, As a multi-layer feature consistency index, As a stability indicator, This is a historical consistency indicator.

[0083] Specifically, before conducting the credibility assessment, relative importance coefficients are assigned to each indicator (data quality indicator C0, multi-level feature consistency indicator C1, stability indicator C2, and historical consistency indicator C3). These weighting ratios allow the system to flexibly adjust the contribution of each indicator to the final credibility score based on specific application scenarios, business needs, or expert experience. For example, in scenarios with extremely high data quality requirements, the weight ω0 of data quality indicator C0 can be appropriately increased.

[0084] Then, based on the preset indicator weight ratio, the reliability of anomaly detection is evaluated by fusing data quality indicators, multi-layer feature consistency indicators, stability indicators, and historical consistency indicators to obtain a reliability score, which is then saved. The formula for calculating the reliability score is as follows: ; in, , , , The preset indicator weights are all non-negative numbers and their sum is 1; For data quality indicators, As a multi-layer feature consistency index, As a stability indicator, This is a historical consistency indicator.

[0085] This application combines four dimensions into an intuitive numerical credibility score, achieving efficient integration and dimensionality reduction of multi-dimensional information. This allows subsequent decision-making processes to directly perform automated logical judgments based on the score (such as threshold comparison and trend monitoring). At the same time, the pre-predictable weighting allows the fusion framework to flexibly adjust the priority of each indicator according to the security requirements of different application scenarios—for example, increasing the weight of data quality indicators in noisy environments and increasing the weight of stability indicators in the early stages of model deployment, thus balancing the universality of the solution with the customizability of the scenario.

[0086] In one embodiment, to evaluate the credibility of anomaly detection results, after assessing the credibility of anomaly detection by fusing data quality indicators, multi-layer feature consistency indicators, stability indicators, and historical consistency indicators according to preset indicator weight ratios, obtaining a credibility score, and saving this score, the following steps can be further performed: First, the credibility score is analyzed based on preset criteria. Specifically, the preset criteria can be a series of numerical ranges, for example, dividing the credibility score from 0 to 100 into multiple continuous or discontinuous intervals. These criteria can be determined based on historical data analysis, expert experience, business needs, or risk preferences. For example, a high credibility interval (e.g., 80-100 points), a medium credibility interval (e.g., 60-79 points), and a low credibility interval (e.g., 0-59 points) can be set. Another approach is to use fuzzy logic or machine learning models to dynamically adjust the criteria, making them more adaptable to constantly changing detection environments and data characteristics. Then, if the confidence score falls within the first confidence range of the preset criteria, the detection result is deemed reliable, and the result is output. Specifically, the first confidence range typically corresponds to a high confidence or high reliability score interval. Determining the detection result to be reliable means that the system has sufficient confidence in the result and can directly output it for subsequent processing or decision-making. The output detection result can be transmitted to other systems, such as early warning systems or automated control systems, via an API interface; or it can be displayed directly on the user interface and marked as "confirmed anomaly" or "high confidence anomaly." If the credibility score falls within the second credibility range of the preset criteria, the detection result is deemed unreliable. The anomaly detection model is then used to re-detect the data. Specifically, the second credibility range typically corresponds to a score interval requiring further confirmation. Determining the detection result as unreliable does not mean a complete rejection, but rather that its reliability is insufficient for direct adoption and requires secondary verification. Re-detecting the data can be achieved by re-inputting the same data into the anomaly detection model to eliminate random factors; alternatively, it can involve preprocessing the data before re-detection, such as denoising, feature enhancement, or using different model parameters to obtain more stable detection results. If the confidence score falls within the third confidence range of the preset criteria, the detection result is deemed unreliable, and a manual review notification is issued. Specifically, the third confidence range typically corresponds to a low-confidence or high-risk scoring interval. Since the detection result is deemed unreliable, and automatic re-detection is unlikely to effectively resolve the issue, manual intervention is necessary. Issuing a manual review notification can be done by sending an email, SMS, or system message to the relevant operator, prompting them to manually review and judge the abnormal detection result; alternatively, the data to be detected and the original output of the anomaly detection model (such as anomaly scores and anomaly response values) can be submitted together to an expert system or manual review platform for comprehensive analysis by manual reviewers.

[0087] This application implements a hierarchical decision-making mechanism based on credibility scoring—high-credibility detection results are directly adopted to ensure efficiency, medium-credibility results are re-detected to attempt correction and reduce false alarm rate, and low-credibility results are directly transferred to manual review to ensure safety. Through differentiated hierarchical handling, a reasonable balance is achieved between automation efficiency and manual safety, while avoiding the decision-making risks caused by blindly outputting erroneous detection results in the case of low credibility.

[0088] In one embodiment, to evaluate the credibility of anomaly detection results, after assessing the credibility of anomaly detection by fusing data quality indicators, multi-layer feature consistency indicators, stability indicators, and historical consistency indicators according to preset indicator weight ratios, obtaining a credibility score, and saving this score, the following steps can be further performed: Obtain several historical credibility scores, and analyze the time trend of credibility scores based on these historical credibility scores and the credibility scores; when the time trend continues to decline within a preset time, issue an equipment maintenance notification.

[0089] Specifically, several historical credibility scores are obtained, which can be stored in a database, log files, or a dedicated data warehouse and can be retrieved based on timestamps, detection task identifiers, or monitored device numbers.

[0090] By analyzing a series of credibility scores arranged chronologically, one can identify patterns or directions of evolution over time. This can be achieved through various statistical methods, such as calculating moving averages, performing linear regression analysis, using exponential smoothing, or simply comparing differences between consecutive scores.

[0091] Within a pre-defined time period, the overall trend of the credibility score shows a continuous and significant decline. The pre-defined time period can be configured according to the actual application scenario, such as the last 24 hours, the last 7 days, or the last N detection cycles. The criteria for judging a continuous decline can include a continuous decrease in the average score within this time period, or multiple consecutive scores being lower than the previous score, or the slope of the trend line being negative and its absolute value exceeding a certain threshold.

[0092] Upon detecting a continuous downward trend in the credibility score, an alarm or message is automatically generated and sent to prompt relevant personnel to inspect and maintain the monitored equipment or the anomaly detection system itself.

[0093] Let's illustrate this with a concrete example. Suppose a critical piece of equipment on an industrial production line has its operating status monitored in real time by an anomaly detection system. The system periodically outputs a reliability score to assess the reliability of each anomaly detection result. To achieve effective long-term monitoring of the equipment's operating status, the system continuously stores these reliability scores in a time-series database. A dedicated monitoring module is configured to query this database hourly for all reliability scores within the most recent 24 hours (i.e., a preset timeframe). This module then performs linear regression analysis on these score data to calculate the slope of the reliability score's trend over time. If three consecutive queries (e.g., within three consecutive hours) show a negative slope, and its absolute value exceeds a preset threshold (e.g., -0.05), the system determines that the reliability score is showing a continuous downward trend. At this point, the system automatically generates an equipment maintenance notification and sends it to the equipment maintenance team via the company's internal instant messaging platform (such as WeChat Work). The notification includes the equipment's unique identifier and information about the declining reliability score, prompting the maintenance team to conduct preventative checks on the equipment (e.g., its anomaly detection model and ambient temperature).

[0094] This application expands the credibility score from a single criterion to a time-series monitoring indicator, enabling the system to have predictive maintenance capabilities based on the credibility degradation trend. The continuous decline in the credibility score can serve as an early warning signal for equipment anomalies or model failures, triggering maintenance actions before the system completely fails. This transforms post-event remediation into pre-event prevention, significantly reducing the risk of missed detections and false alarms caused by the failure of the anomaly detection system itself, and improving the operational reliability and maintenance timeliness of the entire detection system.

[0095] In one embodiment, to evaluate the credibility of anomaly detection results, after assessing the credibility of anomaly detection by fusing data quality indicators, multi-layer feature consistency indicators, stability indicators, and historical consistency indicators according to preset indicator weight ratios, obtaining a credibility score, and saving this score, the following steps can be further performed: Obtain the true annotation information of the detection results, calculate the prediction deviation of the credibility score based on the comparison between the true annotation information and the detection results, and update the index weight ratio according to the prediction deviation and save it when the prediction deviation exceeds the preset deviation threshold.

[0096] Specifically, this involves acquiring label information that accurately reflects the true state of the data and corresponds to the detection results output by the anomaly detection model. This accurate labeling information can be obtained through human experts reviewing and annotating the original data. For example, in industrial inspection, quality inspectors manually check product images and mark them for defects.

[0097] Calculate the classification accuracy, F1 score, precision, recall, and other metrics between the detection results indicated by the credibility score (e.g., whether they are credible or uncredible) and the detection results indicated by the true annotation information (e.g., whether they are actually abnormal). Use the reciprocal of these metrics or the difference from the ideal value (e.g., 1) as the prediction bias.

[0098] When the prediction deviation exceeds a preset deviation threshold, it means that the accuracy of the confidence score has dropped to a certain level, requiring the weight update mechanism to be activated to avoid frequent and unnecessary adjustments. The preset deviation threshold can be a fixed value, for example, triggered when the classification accuracy is below 90%.

[0099] Based on the actual performance of the credibility score, the importance of each indicator in the credibility score calculation can be adaptively adjusted to improve the accuracy of credibility assessment. Optimization algorithms such as gradient descent or reinforcement learning can be used, employing prediction bias as a loss function to iteratively adjust the weights and minimize the prediction bias.

[0100] This application introduces an online weight self-learning mechanism based on real feedback, which enables the weight allocation to be optimized and adjusted in a closed loop according to the actual detection performance. When the credibility score continuously overestimates or underestimates the actual credibility of the detection results, the system can automatically identify the deviation and correct the fusion weight of each indicator. This allows the credibility assessment model to continuously evolve with environmental changes after actual deployment, maintaining the long-term applicability and evaluation accuracy of the fusion framework in different stages and scenarios.

[0101] Based on the above method, this application also discloses an anomaly detection reliability evaluation system. For example... Figure 3 The system includes the following modules: Data receiving module 301 is used to receive the data to be detected and calculate the signal-to-noise ratio of the data to be detected as a data quality indicator. The detection result module 302 is used to detect the data to be detected using the anomaly detection model to obtain the detection result. The detection result includes the anomaly score and the anomaly response values ​​output by several network layers in the anomaly detection model. Feature consistency calculation module 303 is used to calculate multi-level feature consistency index based on several abnormal response values; The stability calculation module 304 is used to process the data to be detected to obtain the disturbance data, input the disturbance data into the anomaly detection model to obtain the disturbance detection result, and calculate the stability index based on the detection result and the disturbance detection result. The historical consistency calculation module 305 is used to obtain historical anomaly detection results, combine the current detection results and historical anomaly detection results to obtain a detection result set, and use the detection result set to calculate the historical consistency index. The credibility assessment module 306 is used to weight and fuse data quality indicators, multi-layer feature consistency indicators, stability indicators, and historical consistency indicators to obtain a credibility score, thereby assessing the credibility of the detection results.

[0102] In one embodiment, the feature consistency calculation module 303 is specifically used to parse the anomaly detection model to obtain the hierarchical features of several network layers in the anomaly detection model; retrieve the anomaly detection feature requirements; determine the weight ratio of several network layers in the consistency evaluation based on the several hierarchical features and the anomaly detection feature requirements; and obtain the multi-layer feature consistency index by analyzing the variance and standard deviation of several anomaly response values ​​based on the weight ratio.

[0103] In one embodiment, the stability calculation module 304 is specifically used to query the sensitivity of the anomaly detection model to minor disturbance methods, including image brightness changes, noise addition, geometric transformations, and feature perturbations; formulate a minor disturbance strategy based on the sensitivity, including disturbance intensity, disturbance type, and disturbance quantity; and process the data to be detected by perturbing according to the minor disturbance strategy to obtain minor disturbance data.

[0104] In one embodiment, the stability calculation module 304 is specifically used to combine the detection results and the detection results of minor disturbances into a set of detection results. The stability index is calculated based on the set of test results. The formula for calculating the stability index is as follows: ; in, Here, P represents the stability index, and P is the set of detection results. The average of all test results. The standard deviation of all test results is given. To prevent extremely small constants with a denominator of 0.

[0105] In one embodiment, the historical consistency calculation module 305 is specifically used to assign time decay weights to the detection result set in chronological order. The historical consistency index is calculated based on the detection result set. The formula for calculating the historical consistency index is as follows: ; in, This represents the historical consistency indicator, where N is the number of detection results. This is the result of the i-th detection. Let I() be the result of the (i+1)th detection, and let I() be the indicator function.

[0106] In one embodiment, the credibility assessment module 306 is specifically used to assess the credibility of anomaly detection by fusing data quality indicators, multi-layer feature consistency indicators, stability indicators, and historical consistency indicators according to a preset indicator weight ratio, obtaining a credibility score and saving it. The formula for calculating the credibility score is as follows: ; in, , , , The preset indicator weights are all non-negative numbers and their sum is 1; For data quality indicators, As a multi-layer feature consistency index, As a stability indicator, This is a historical consistency indicator.

[0107] In one embodiment, the credibility assessment module 306 is specifically used to analyze the credibility score according to a preset division standard; if the credibility score is within the first credibility range of the preset division standard, the detection result is determined to be credible and the detection result is output; if the credibility score is within the second credibility range of the preset division standard, the detection result is determined to be unreliable and the anomaly detection model is used to detect and re-detect the data to be detected; if the credibility score is within the third credibility range of the preset division standard, the detection result is determined to be unreliable and a manual review notification is output.

[0108] In one embodiment, the credibility assessment module 306 is specifically used to obtain several historical credibility scores, analyze the time change trend of the credibility scores based on the several historical credibility scores and the credibility scores, and issue a device maintenance notification when the time change trend continues to decline within a preset time.

[0109] In one embodiment, the confidence assessment module 306 is specifically used to obtain the true annotation information of the detection results, calculate the prediction deviation of the confidence score based on the comparison results between the true annotation information and the detection results, and update the index weight ratio and save it when the prediction deviation exceeds the preset deviation threshold.

[0110] This specific embodiment is merely an explanation of the present invention and is not intended to limit the invention. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of the claims of the present invention.

Claims

1. A method for evaluating the reliability of anomaly detection, characterized in that, The method is based on an anomaly detection model and includes: Receive the data to be detected and calculate the signal-to-noise ratio of the data to be detected as a data quality indicator; The anomaly detection model is used to detect the data to be detected to obtain the detection result. The detection result includes an anomaly score and anomaly response values ​​output by several network layers in the anomaly detection model. Calculate the multi-layer feature consistency index based on several of the aforementioned abnormal response values; The data to be detected is perturbed to obtain perturbed data. Several perturbed data are input into an anomaly detection model for detection to obtain perturbed detection results. A stability index is calculated based on the detection results and several perturbed detection results. Obtain historical anomaly detection results, combine the current detection results and the historical anomaly detection results to obtain a detection result set, and use the detection result set to calculate the historical consistency index; The credibility score is obtained by weighted fusion of the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index to evaluate the credibility of the detection results.

2. The method according to claim 1, characterized in that, The calculation of the multi-layer feature consistency index based on several of the aforementioned abnormal response values ​​includes: The anomaly detection model is analyzed to obtain the hierarchical features of several network layers in the anomaly detection model; Retrieve anomaly detection feature requirements, and determine the weight ratio of several network layers in the consistency evaluation based on several hierarchical features and anomaly detection feature requirements; Based on the weighting ratio, a multi-level feature consistency index is obtained by analyzing the variance and standard deviation of several abnormal response values.

3. The method according to claim 1, characterized in that, The disturbance processing of the data to be detected yields disturbance data including: The sensitivity of the anomaly detection model to several perturbation methods is queried, including image brightness changes, noise addition, geometric transformations, and feature perturbations. Based on the sensitivity, a disturbance strategy is formulated, wherein the disturbance strategy includes disturbance intensity, disturbance type, and disturbance quantity; The data to be detected is perturbed according to several perturbation strategies to obtain perturbed data.

4. The method according to claim 3, characterized in that, The calculation of the stability index based on the detection results and several disturbance detection results includes: The detection results and several disturbance detection results are combined to form a detection result set; A stability index is calculated based on the set of test results. The formula for calculating the stability index is as follows: ； in, Here, P represents the stability index, and P is the set of detection results. The average of all test results. The standard deviation of all test results is given. To prevent extremely small constants with a denominator of 0.

5. The method according to claim 1, characterized in that, The calculation of the historical consistency index using the detection result set includes: The detection result set is assigned a time decay weight in chronological order; The historical consistency index is calculated based on the detection result set. The formula for calculating the historical consistency index is as follows: ； in, This represents the historical consistency indicator, where N is the number of detection results. For the i-th detection result, Let I be the result of the (i+1)th detection, and let I() be the indicator function.

6. The method according to claim 1, characterized in that, The weighted fusion of the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index to obtain the credibility score includes: The credibility of anomaly detection is evaluated and saved by fusing the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index according to the preset index weight ratio. The formula for calculating the credibility score is as follows: ; in, , , , The preset indicator weights are all non-negative numbers and their sum is 1; For data quality indicators, As a multi-layer feature consistency index, As a stability indicator, This is a historical consistency indicator.

7. The method according to claim 6, characterized in that, After evaluating the credibility of anomaly detection by fusing the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index according to a preset index weight ratio, and obtaining and saving the credibility score, the process further includes: The credibility score is analyzed based on the preset classification criteria; If the credibility score is within the first credibility range of the preset division criteria, the detection result is determined to be credible, and the detection result is output. If the credibility score is within the second credibility range of the preset division criteria, the detection result is determined to be unreliable, and the anomaly detection model is used to re-detect the data to be detected. If the credibility score falls within the third credibility range of the preset classification criteria, the detection result is determined to be unreliable, and a manual review notification is output.

8. The method according to claim 6, characterized in that, After evaluating the credibility score of anomaly detection by fusing the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index according to a preset index weight ratio, the following steps are also included: Several historical credibility scores are obtained, and the time trend of credibility scores is analyzed based on the historical credibility scores and the credibility scores. When the time change trend continues to decline within a preset time period, a device maintenance notification is issued.

9. The method according to claim 7, characterized in that, After evaluating the credibility score of anomaly detection by fusing the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index according to a preset index weight ratio, the following steps are also included: Obtain the true annotation information of the detection results, and calculate the prediction deviation of the credibility score based on the comparison results between the true annotation information and the detection results; When the prediction deviation exceeds the preset deviation threshold, the indicator weight ratio is updated and saved according to the prediction deviation.

10. An anomaly detection reliability assessment system, characterized in that, The system is based on an anomaly detection model and includes: The data receiving module (301) is used to receive the data to be detected and calculate the signal-to-noise ratio of the data to be detected as a data quality indicator. The detection result module (302) is used to detect the data to be detected using the anomaly detection model to obtain the current detection result. The current detection result includes anomaly score and anomaly response values ​​output by several network layers in the anomaly detection model. The feature consistency calculation module (303) is used to calculate a multi-layer feature consistency index based on several of the aforementioned abnormal response values; The stability calculation module (304) is used to process the data to be detected by disturbance to obtain disturbance data, input the disturbance data into the anomaly detection model for detection to obtain disturbance detection results, and calculate the stability index based on the detection results and the disturbance detection results. The historical consistency calculation module (305) is used to obtain historical anomaly detection results, combine the current detection results and the historical anomaly detection results to obtain a detection result set, and use the detection result set to calculate the historical consistency index. The credibility assessment module (306) is used to weight and fuse the data quality index, the multi-layer feature consistency index, the stability index, and the historical consistency index to obtain a credibility score, so as to assess the credibility of the detection result.

Images