Abnormal data detection method, system and equipment of power and environment monitoring system and medium

By combining preset thresholds and redundant information filtering, multi-dimensional abnormality detection and missing value calculation are performed, the problem of insufficient data processing capability and abnormality detection accuracy of the existing technology dynamic ring monitoring system is solved, and more efficient Abnormal data detection and fault diagnosis have improved the automation level and fault response speed of data center operations and maintenance.

CN119065919APending Publication Date: 2024-12-03NAT COMP NETWORK & INFORMATION SECURITY MANAGEMENT CENT
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410979112.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing dynamic ring monitoring system has shortcomings in massive data processing and intelligent analysis, especially when quickly and accurately identifying abnormalities in real-time data flows. Traditional methods rely on simple threshold comparisons, lack dynamic adaptability and intelligent processing mechanisms, making it difficult to It deals with complex and changeable operating environments and equipment status changes, resulting in false alarms and missed alarms, and the redundant information and topological relationships are not fully utilized, which affects the accuracy and timeliness of fault diagnosis.

Method used

A method of abnormal data detection of dynamic ring monitoring system is adopted, by obtaining real-time monitoring data, combining preset thresholds and redundant information for filtering, and converting formats based on the northward interface specifications, comparing and analyzing the southward and northward interface logs , using multi-dimensional anomaly detection and missing value calculation, including numerical anomaly detection, rate of change detection, frequency detection and time series prediction model, combined with redundant information and context correlation analysis, output abnormal data detection results.

Benefits of technology

It significantly improves the accuracy and effectiveness of data screening, ensures the integrity and consistency of data during cascading transmission, enhances the self-diagnosis ability of the system, simplifies maintenance work, and is suitable for the diversified monitoring needs of large-scale data centers. , improves the maintainability and scalability of filtering rules, improves the failure response speed and maintenance efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065919A_ABST
    Figure CN119065919A_ABST
Patent Text Reader

Abstract

The invention provides an abnormal data detection method for a power and environment monitoring system, which comprises the following steps of: acquiring real-time monitoring data of a lower-level monitoring module in the power and environment monitoring system, and generating a southbound interface log; performing filtering processing on the real-time monitoring data in combination with a preset threshold value and the redundant information, and performing format conversion based on northbound interface specifications to generate northbound interface logs; and comparing and analyzing the southbound interface log and the northbound interface log, and outputting an abnormal data detection result in combination with the filtered abnormal data. According to the method, the reliability of data transmission from a lower level to an upper level is ensured through intelligent filtering and format unification among levels. Compared with the prior art, errors are effectively recognized and eliminated through comparison of the threshold value and redundant data, finally abnormity is accurately recognized by means of a log comparison mechanism, efficient automatic discovery and accurate diagnosis of faults in data center operation and maintenance are achieved, and the response speed and maintenance efficiency of a power and environment monitoring system are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data center infrastructure operation and maintenance, and more specifically to an abnormal data detection method, system, equipment and medium for a dynamic environment monitoring system. Background Art

[0002] As a key facility supporting artificial intelligence and the digital economy, the continuous and stable operation of data centers is extremely critical, and the reliability of the power system is the cornerstone of ensuring business continuity. To this end, it is inevitable to deploy an efficient dynamic environment monitoring system to achieve real-time monitoring and early warning. In response to the special needs of large-scale data centers, dynamic environment monitoring adopts a layered architecture design, which manages the sensor data of a single or multiple computer rooms by building an on-site monitoring layer, and then gradually converges to the park-level and even enterprise-level comprehensive monitoring platform.

[0003] Existing dynamic environment monitoring systems often focus on data collection, but they are insufficient in the effective processing and intelligent analysis of massive data. In particular, to quickly and accurately identify anomalies in real-time data streams, traditional methods rely on simple threshold comparisons, lack dynamic adaptability and intelligent processing mechanisms, and are difficult to cope with complex and changing operating environments and equipment status changes. Although it is easy to implement, directly applying fixed threshold filtering cannot adapt to the natural changes in the environment or equipment performance over time, and is prone to false alarms and missed alarms. Especially for analog data, whose normal range may change due to external conditions or equipment aging, the fixed threshold method appears rigid and not precise enough.

[0004] At the same time, single-dimensional anomaly detection (such as based only on numerical thresholds) is not enough to cover all types of abnormal situations. Failure to fully consider the correlation between data, time series trends and complex patterns makes fault diagnosis lack depth and accuracy, making it difficult to quickly locate the root cause of the problem.

[0005] In complex monitoring systems, there may be redundant information between different sensors, but traditional methods rarely effectively integrate this information to improve the reliability of anomaly detection. The topological relationship and spatial location information of power, HVAC and other systems are not fully utilized, resulting in the failure to reduce false alarms through mutual verification. In the face of missing data, traditional processing methods are relatively simple, such as directly ignoring or using simple interpolation to fill, which may introduce additional errors and affect the continuity and accuracy of overall monitoring. There is a lack of advanced time series prediction models and context-related intelligent filling strategies.

[0006] In addition, the data exchange standards between the southbound interface (device side) and the northbound interface (management platform side) are different, which makes data parsing, format conversion and integration cumbersome and error-prone. The lack of unified log management and efficient data alignment mechanism affects the efficiency of data processing and the timeliness of anomaly detection.

[0007] Therefore, how to design an abnormal data detection method for the dynamic environment monitoring system, improve the automation level and fault response speed of data center operation and maintenance, and ensure the stable and safe operation of the data center is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0008] In view of this, the present invention provides an abnormal data detection method for a dynamic environment monitoring system, which overcomes the defects of the existing dynamic environment monitoring system, such as limited data processing capacity, low anomaly detection accuracy, low comprehensive analysis efficiency, insufficient use of redundant information, improper processing of missing data, etc. It can ensure the continuous stability and safe operation of the data center and provide a solid guarantee for the infrastructure of the digital economy.

[0009] In order to achieve the above object, the present invention adopts the following technical solution:

[0010] In a first aspect, the present invention provides a method for detecting abnormal data in a dynamic environment monitoring system, comprising the following steps:

[0011] S1. Obtain the real-time monitoring data of the lower-level monitoring module in the dynamic environment monitoring system and generate a southbound interface log;

[0012] S2. Filter the real-time monitoring data in combination with preset thresholds and redundant information, perform format conversion based on the northbound interface specification, and generate a northbound interface log;

[0013] S3. Compare and analyze the southbound interface log and the northbound interface log, and output an abnormal data detection result in combination with the abnormal data after the filtering process.

[0014] Furthermore, the southbound interface log and the northbound interface log include: the name of the monitoring point, the name of the device to which the monitoring point belongs, the space name of the device to which the monitoring point belongs, the monitoring point number, the monitoring indicator value and the receiving time.

[0015] Furthermore, in S2, filtering the real-time monitoring data in combination with a preset threshold value includes:

[0016] For the state quantity data in the real-time monitoring data, a direct transmission strategy is implemented;

[0017] For the analog data in the real-time monitoring data, filtering is performed based on the preset minimum and maximum values; if the monitoring indicator value of a certain monitoring point in the analog data is within the preset minimum and maximum value range, the transmission strategy is implemented, otherwise, the monitoring indicator value is filtered and stored in the abnormal monitoring data list.

[0018] Furthermore, in S2, filtering the real-time monitoring data in combination with redundant information includes:

[0019] S21, constructing a redundant information group according to the power topology connection relationship, the HVAC topology connection relationship, and the spatial position relationship; the redundant information group includes one main information and at least two redundant information;

[0020] S22, based on the real-time monitoring value of the redundant information, the real-time monitoring value of the main information is filtered using a change rate rule, and stored in the abnormal monitoring data list; the change rate rule is:

[0021]

[0022] Among them, X t is the value of the main information at time t, It represents the average value of redundant information at time t; r is the rate of change, which indicates the similarity between the main information and the redundant information.

[0023] Furthermore, the S3 includes:

[0024] S31, parsing and formatting the southbound interface log and the northbound interface log to obtain parsed and formatted log information;

[0025] S32, based on the parsed and formatted log information, unify the time base through a timestamp calibration algorithm, and perform sequence alignment in combination with a nearest neighbor matching strategy to obtain the sequence-aligned log information;

[0026] S33, using the improved hash index to screen log entries of the sequence-aligned log information, and converting the log information into a structured format to obtain log information in a structured format;

[0027] S34: Perform multi-dimensional anomaly detection and missing value inference on the log information in the structured format, and output an abnormal data detection result.

[0028] Furthermore, in said S34, the multi-dimensional anomaly detection includes: numerical anomaly detection, change rate anomaly detection and frequency anomaly detection;

[0029] The numerical anomaly detection determines the anomaly limit based on the standard deviation and the interquartile range to identify outliers in the data; the frequency anomaly detection uses Fourier transform to detect periodic or non-periodic abnormal fluctuations in the data to identify abnormal frequency components.

[0030] Furthermore, in said S34, the missing value imputation includes: missing value imputation based on a time series prediction model and context association analysis;

[0031] The time series prediction model adopts the long short-term memory network LSTM, which memorizes the long-term pattern of the time series through the recurrent neural network structure and predicts the value of the missing data points; the context association analysis adopts the K-nearest neighbor algorithm, which predicts the value of the missing data points through the average value of k neighbors.

[0032] In a second aspect, the present invention provides an abnormal data detection system for a dynamic environment monitoring system, comprising:

[0033] Data acquisition module: used to obtain real-time monitoring data of the lower-level monitoring modules in the dynamic environment monitoring system and generate southbound interface logs;

[0034] Data filtering and format conversion module: used to filter the real-time monitoring data in combination with preset thresholds and redundant information, and perform format conversion based on the northbound interface specification to generate a northbound interface log;

[0035] Log comparison and analysis module: used to compare and analyze the southbound interface log and the northbound interface log, and output the abnormal data detection result in combination with the abnormal data after the filtering process.

[0036] In a third aspect, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned abnormal data detection method when executing the computer program.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program implements the above-mentioned abnormal data detection method when executed by a processor.

[0038] It can be seen from the above technical solution that compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. The technical solution of this application combines two strategies: minimum and maximum value filtering based on the sensor range and redundant information auxiliary filtering. It can not only exclude obvious erroneous values, but also effectively identify those abnormal values ​​that are within the range but do not reflect the actual situation through mutual verification of redundant data, significantly improving the accuracy and effectiveness of data screening.

[0040] 2. By comparing the southbound and northbound interface logs, a self-verification process is introduced to ensure the integrity and consistency of data during cascade transmission. It goes beyond the traditional one-way data transmission verification and forms a closed-loop monitoring feedback system, so that any errors occurring in the data processing link can be discovered in a timely manner, enhancing the system's self-diagnosis capabilities.

[0041] 3. It provides the function of configuring filtering rules by measurement point category and merging similar rules, which greatly simplifies maintenance work and reduces configuration complexity. This flexibility is not only suitable for the diversified monitoring needs of large-scale data centers, but also improves the maintainability and scalability of filtering rules, which is conducive to meeting the needs of future data center monitoring system upgrades or scale expansions. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0043] Figure 1 A flow chart of an abnormal data detection method for a dynamic environment monitoring system provided by an embodiment of the present invention;

[0044] Figure 2 A flowchart of filtering real-time monitoring data in combination with redundant information provided by an embodiment of the present invention;

[0045] Figure 3 A flow chart for comparing and analyzing southbound interface logs and northbound interface logs provided in an embodiment of the present invention;

[0046] Figure 4 A framework diagram of an abnormal data detection system for a dynamic environment monitoring system provided by an embodiment of the present invention;

[0047] Figure 5 A schematic diagram of the structure of an electronic device is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The detection method provided in the embodiment of the present application can be applied to a detection server. The above-mentioned detection server can be hardware or software. When the detection server is hardware, it can be implemented as a distributed server cluster that provides detection services, or it can be implemented as a single server. When the detection server is software, it can be installed in the servers listed above. It can be implemented as multiple software or software modules, or it can be implemented as a single software or software module, which is not specifically limited here.

[0050] Embodiment 1;

[0051] like Figure 1 As shown, this embodiment provides an abnormal data detection method for a dynamic environment monitoring system, comprising the following steps:

[0052] S1. Obtain the real-time monitoring data of the lower-level monitoring module in the dynamic environment monitoring system and generate a southbound interface log;

[0053] S2. Filter the real-time monitoring data in combination with preset thresholds and redundant information, perform format conversion based on the northbound interface specification, and generate a northbound interface log;

[0054] S3. Compare and analyze the southbound interface log and the northbound interface log, and output an abnormal data detection result in combination with the abnormal data after the filtering process.

[0055] This method ensures the reliability of data transmission from the lower level to the upper level through intelligent filtering and format unification between levels. By using thresholds and redundant data comparison, errors can be effectively identified and eliminated. Finally, with the help of the log comparison mechanism, anomalies can be accurately identified, achieving efficient automatic discovery and accurate diagnosis of faults in data center operation and maintenance, and improving the response speed and maintenance efficiency of the dynamic environment monitoring system.

[0056] The above steps and related technical features are further described in detail below;

[0057] In this embodiment, the communication protocol and request response mode of the southbound interface of the system are determined according to the data transmission interface specification, and the real-time monitoring data transmitted by the cascaded lower-level monitoring system is obtained based on this; the received raw monitoring data can be stored in the database for querying historical data. The storage period of the raw monitoring data can be selected based on business needs, but it is usually not suitable to exceed 3 months.

[0058] The southbound interface log and the northbound interface log include: the monitoring point name, the name of the device to which the monitoring point belongs, the space name of the device to which the monitoring point belongs, the monitoring point number, the monitoring indicator value and the receiving time.

[0059] This section emphasizes the detailed content of log records, which provides a rich basis for fault location and trend analysis. This detailed log recording mechanism improves the transparency of monitoring and the ability to trace problems, which helps operation and maintenance personnel quickly locate the source of problems and optimize maintenance efficiency.

[0060] In this embodiment S2, the real-time monitoring data is filtered in combination with a preset threshold value. The filtering mainly solves the obvious unreasonable values ​​caused by data transmission line interference, data parsing errors, software abnormality, etc., including:

[0061] For the state quantity data in the real-time monitoring data, a direct transmission strategy is implemented;

[0062] For the analog data in the real-time monitoring data, filtering is performed based on the preset minimum and maximum values; if the monitoring indicator value of a certain monitoring point in the analog data is within the preset minimum and maximum value range, the transmission strategy is implemented, otherwise, the monitoring indicator value is filtered and stored in the abnormal monitoring data list.

[0063] It refines the data filtering strategy, especially for the different processing methods of state quantity and analog quantity. The direct transmission strategy simplifies the state quantity data processing process, and the analog quantity data is filtered through the preset threshold, which not only avoids false alarms caused by data fluctuations, but also ensures that important anomalies are not missed, improving the pertinence and effectiveness of data processing.

[0064] In addition, the preset minimum and maximum values ​​involved in this step are set as follows:

[0065] For monitoring indicators measured directly, the minimum and maximum values ​​are determined using the sensor range. The minimum value is the lower limit of the range, and the maximum value is the upper limit of the range. Since any measured value exceeding the range is a meaningless value, the filtering has a physical basis, which effectively avoids the false filtering caused by other setting methods. For monitoring indicators obtained by indirect calculation, the minimum and maximum values ​​are determined using the range of the relevant indicators. The minimum value is the calculation result of the lower limit of the relevant indicator range, and the maximum value is the calculation result of the upper limit of the relevant indicator range. The calculation method of the minimum and maximum values ​​is the same as that of the indirect monitoring indicators. For example, the power of single-phase electricity is equal to the voltage multiplied by the current. The minimum value in the power filtering rule is equal to the lower limit of the voltage range multiplied by the lower limit of the current range, and the maximum value is equal to the upper limit of the voltage range multiplied by the upper limit of the current range.

[0066] In order to improve the convenience of filtering rule configuration, it can be configured according to the measurement point category. For example, a data center usually has thousands or even tens of thousands of battery sensors. The temperature measurement points, voltage measurement points, and internal resistance measurement points of the battery sensors can be classified into one category. Once a category is configured, all measurement points under this category will take effect.

[0067] At the same time, in order to simplify the management of filtering rules, filtering rules can be merged. For example, the temperature and humidity sensors in the data center may have multiple brands, and the ranges of different brands may be slightly different. In order to reduce the number of rules and improve filtering efficiency, multiple similar rules can be merged. The minimum value of the merged rule is equal to the minimum value of the minimum values ​​of the original multiple rules, and the maximum value of the merged rule is equal to the maximum value of the maximum values ​​of the original multiple rules.

[0068] like Figure 2As shown, the real-time monitoring data is filtered in combination with redundant information. This filtering mainly solves abnormal values ​​within the range that cannot truly reflect the operation of the power system, including:

[0069] S21, constructing a redundant information group according to the power topology connection relationship, the HVAC topology connection relationship, and the spatial position relationship; the redundant information group includes one main information and at least two redundant information;

[0070] Power topology connection relationship: In the power system, equipment and lines form a network structure according to their physical connections and electrical characteristics. For example, substations, transformers, transmission lines, loads, etc. form a complex power network. There are dependencies between these components. If an abnormality occurs in one link, it may affect other parts connected to it. When building redundant information groups, these electrical connections are taken into account to ensure that the measured values ​​of key data such as current and voltage can be mutually verified through the data of multiple related nodes, thereby improving the credibility of monitoring data.

[0071] HVAC topology connection relationship: In the HVAC system, various devices such as fans, air conditioning units, valves, air ducts and water pipes form a topological structure according to their functions and connections in the air handling and fluid delivery process. By analyzing this connection relationship, we can understand how air flow or water flow is transmitted within the system, as well as the mutual influence between various monitoring points (such as temperature, humidity, and pressure sensors). When constructing redundant information groups, these relationships are used to confirm the consistency of the data. For example, the temperature and humidity changes in adjacent rooms should have a certain correlation.

[0072] Spatial position relationship: It involves the layout of monitoring points in the actual physical space. Some equipment or monitoring points may be affected by similar environmental factors (such as temperature and humidity affected by the external climate) due to their close geographical location, or due to interactions caused by spatial layout (such as the effect of heating of equipment in the computer room on the surrounding temperature). By analyzing the spatial position relationship, data points that should have similar performance or expected correlation can be identified and integrated into redundant information groups for mutual verification.

[0073] In summary, constructing a redundant information group is to identify and utilize these physical and logical connections to form a data group containing the main information and at least two redundant information. The main information is the target monitoring data of direct concern, while the redundant information is other monitoring data inferred through the above relationship, which should theoretically show a consistent or related change trend with the main information. In practical applications, these redundant information can help filter out unreasonable data fluctuations and enhance the accuracy and efficiency of anomaly detection.

[0074] S22, based on the real-time monitoring value of the redundant information, the real-time monitoring value of the main information is filtered using a change rate rule, and stored in the abnormal monitoring data list; the change rate rule is:

[0075]

[0076] Among them, X t is the value of the main information at time t, It represents the average value of redundant information at time t; r is the rate of change, which indicates the similarity between the main information and the redundant information.

[0077] The introduction of redundant information groups and change rate rules further enhances the accuracy of anomaly detection. Through the mutual verification of redundant information in multiple dimensions such as power and HVAC, it can effectively identify misjudgments caused by single point failures and ensure the reliability of monitoring data. The application of change rate rules enables the system to dynamically adapt to environmental changes, reduce false alarms and missed alarms, and improve the intelligence of anomaly detection.

[0078] like Figure 3 As shown, this embodiment S3 includes:

[0079] S31. Parse and format the southbound interface log and the northbound interface log to obtain the parsed and formatted log information; this step aims to convert the original southbound and northbound interface logs into a unified and easy-to-process format. Southbound logs usually contain raw monitoring data, while northbound logs are data that have been preliminarily processed and adapted to upper-layer applications. The parsing process reads and understands the content structure of the logs, and the formatting reorganizes the data according to predefined rules to ensure that all information can be efficiently used in subsequent steps.

[0080] S32. Based on the parsed and formatted log information, the time base is unified through the timestamp calibration algorithm, and the sequence is aligned in combination with the nearest neighbor matching strategy to obtain the log information after sequence alignment; since logs from different sources may use different timestamps or have slight time offsets, this step first adjusts all log records to the same time base through the timestamp calibration algorithm to ensure the temporal consistency of the data. Then, the nearest neighbor matching strategy is used to align the log sequence. Even if there are slight differences in the arrival time of the records, they can be arranged in the correct order by comparing adjacent records, creating coherent time series data for subsequent analysis.

[0081] S33, using the improved hash index to filter the log entries after the sequence alignment, and converting the log information into a structured format to obtain the log information in a structured format; the improved hash index technology is used here to efficiently filter the aligned log entries, involving quickly finding or classifying logs according to key fields (such as device ID, timestamp, etc.), removing duplicates or irrelevant information, and thus streamlining the data set. Subsequently, the filtered data is further converted into a structured format, such as a table or JSON format, so that the data has clear fields and values, which is convenient for algorithm and model processing.

[0082] S34: Perform multi-dimensional anomaly detection and missing value inference on the log information in the structured format, and output an abnormal data detection result.

[0083] Multi-dimensional anomaly detection includes: numerical anomaly detection, change rate anomaly detection and frequency anomaly detection; the multi-dimensional anomaly detection strategy realizes the comprehensive identification and location of monitoring data anomalies by comprehensively using three analysis methods: numerical value, change rate and frequency. In general, this strategy aims to analyze data behavior from different angles to ensure the comprehensiveness and accuracy of anomaly detection.

[0084] Numerical anomaly detection: This process is based on statistical principles and uses standard deviation and interquartile range to quantify the degree of dispersion of data distribution. The standard deviation measures the deviation of data points from the mean, while the interquartile range reflects the span of the middle 50% of the data set. The combination of the two can define the normal data range, thereby identifying outliers outside this range, which often indicate potential abnormal conditions.

[0085] Rate of change anomaly detection: focuses on the speed and pattern of data changes over time. By calculating the rate of change between data points, abnormal sharp rises or falls can be captured.

[0086] Frequency anomaly detection: Using frequency domain analysis techniques, especially Fourier transform, the time domain signal is converted into a frequency domain representation to reveal the periodic and non-periodic fluctuation characteristics in the data. By analyzing the spectral components of the signal, frequency components that do not conform to the normal pattern can be identified. These frequency anomalies may correspond to unusual activities such as equipment failures and external interference.

[0087] The multi-dimensional anomaly detection method in this step combines the advantages of statistical analysis, time series analysis, and frequency domain analysis to form a powerful anomaly detection system that can not only capture abnormal values ​​of data points, but also identify anomalies in data change trends and periodic patterns, providing in-depth and detailed anomaly identification capabilities for dynamic environment monitoring in data centers.

[0088] Missing value imputation, including: missing value imputation based on time series prediction model and context association analysis;

[0089] The time series prediction model adopts the long short-term memory network LSTM, which memorizes the long-term pattern of the time series through the recurrent neural network structure and predicts the value of the missing data points; the context association analysis adopts the K-nearest neighbor algorithm, which predicts the value of the missing data points through the average value of k neighbors.

[0090] LSTM is good at processing continuous data with time dependence, while the K-nearest neighbor algorithm has an advantage in considering the local relationship between data points. By combining the two, not only can the accuracy of missing value estimation be improved, but also the robustness of the model under different data characteristics and missing patterns can be enhanced, providing a more complete and reliable data analysis basis for the dynamic environment monitoring system.

[0091] Finally, this embodiment also includes outputting the abnormal information in multiple ways. When each new abnormal information is added to the abnormal monitoring data list, it is output in multiple ways, such as alarms, sound prompts, pop-up windows, etc., to ensure that maintenance personnel can handle it in time.

[0092] Embodiment 2;

[0093] In this embodiment, real-time monitoring data of the lower-level monitoring module in the dynamic environment monitoring system is obtained, and a southbound interface log is generated; specifically, the following steps are performed:

[0094] Data 1: Temperature sensor 1, air conditioning unit A, server room 1, 001, 23.5℃, 08:00:00;

[0095] Data 2: Temperature sensor 2, air conditioning unit A, server room 1, 002, missing, 08:01:00;

[0096] Data 3: Temperature sensor 3, air conditioning unit A, server room 1, 003, 23.6℃, ​​08:05:00;

[0097] Data 4: Temperature sensor 4, air conditioning unit A, server room 1, 004, 24.7℃, 08:09:00;

[0098] Data 5: Humidity sensor 1, environmental control system, server room 1, 005, 50.2%, 09:00:00;

[0099] Data 6: Humidity sensor 2, environmental control system, server room 1, 006, 50.5%, 09:01:00;

[0100] Data 7: Access status sensor, access control system, entrance area, 007, open, 09:10:00;

[0101] The above data include: monitoring point name, name of the equipment to which the monitoring point belongs, space name of the equipment to which the monitoring point belongs, monitoring point number, monitoring index value and receiving time.

[0102] Filter the real-time monitoring data:

[0103] State quantity data processing: Data such as the monitoring of "access control status sensor" (data 7) belongs to state quantity data. Because it is expressed as a binary state such as on / off and does not involve a numerical range, a direct transmission strategy is implemented to directly record and transmit its status to the relevant system or log.

[0104] Processing of analog data:

[0105] Preset threshold filtering: The preset temperature range is 20℃~26℃, and the humidity range is 40%~60%.

[0106] Data 1 (23.5°C), data 3 (23.6°C), and data 4 (24.7°C) all fall within the temperature threshold and are transmitted normally. Data 5 (50.2%) and data 6 (50.5%) fall within the humidity threshold and are transmitted normally. Data 2 (missing value).

[0107] The real-time monitoring data is filtered in combination with the redundant information, including:

[0108] Constructing a redundant information group: Since Data 1, Data 3, and Data 4 all provide valid temperature readings, are from the same server room 1, and serve the same purpose (monitoring temperature), they can be used to construct a redundant information group.

[0109] Set data 1 as the main information, and data 3 and data 4 as redundant information, main information: data 1, temperature 23.5℃, time 08:00:00, redundant information 1 (Ri1): data 3, temperature 23.6℃, ​​time 08:05:00, redundant information 2 (Ri2): data 4, temperature 24.7℃, time 08:09:00.

[0110] The average value of the redundant information is calculated as:

[0111]

[0112] The preset change rate r is 0.1, and the M value is less than the change rate r. Therefore, the data 1 is transmitted normally. Similarly, setting data 3 or data 4 as the main information can also determine whether to filter the monitoring data.

[0113] In this embodiment, log parsing and formatting, time base unification and sequence alignment, and log screening and structured conversion in the process of comparative analysis of southbound interface logs and northbound interface logs are not described in detail, and the focus is on abnormal data detection and missing value processing.

[0114] Numerical anomaly detection: Valid temperature data are 23.5°C, 23.6°C, and 24.7°C. Calculate the average temperature and standard deviation.

[0115] Average temperature = (23.5 + 23.6 + 24.7) / 3 = 23.9 ° C;

[0116] Standard deviation σ=sqrt(((23.5-23.9)^2+(23.6-23.9)^2+(24.7-23.9)^2) / 2)≈0.4℃;

[0117] The abnormal limit is defined as the mean ± 3σ. 3σ is about 1.2℃, so the normal range is roughly 22.7℃ to 25.1℃. The 24.7℃ in data 4 is close to the upper limit but does not exceed it significantly.

[0118] The long short-term memory network (LSTM) and K-nearest neighbor algorithm (KNN) methods are combined to process the missing values ​​of temperature sensor 2 in data 2:

[0119] In this embodiment, the long short-term memory network (LSTM) is constructed using the Python deep learning library. The long short-term memory network (LSTM) includes multiple LSTM layers, followed by a fully connected layer for outputting a predicted value. The predicted value obtained by inputting data 1 (23.5°C), data 3 (23.6°C), and data 4 (24.7°C) into the trained long short-term memory network (LSTM) is: 23.48°C;

[0120] The K-nearest neighbor algorithm (KNN) method is used to filter out other sensor data points with similar time and type to the missing data from the data set, considering the records of temperature sensors 1, 3, and 4 at similar time points. Set k = 2 (the k value can be adjusted according to actual conditions), and select the two valid data points closest to the time of data 2 as neighbors, namely data 1 and data 3. Calculate the average of the temperature values ​​of these two neighbors as the predicted value. Predicted value = (23.5℃ + 23.6℃) / 2 = 23.55℃.

[0121] The LSTM model has a smaller prediction error on historical data and shows better generalization ability, so a higher weight of 0.6 is defined for it. The KNN model is given a lower weight of 0.4 because it is simple and performs well in some local situations. The combined prediction value is calculated as follows:

[0122] Combined predicted value = 0.6 × 23.48 + 0.4 × 23.55 = 14.088 + 9.42 = 23.508 ° C. Finally, the combined predicted temperature value is 23.51 ° C.

[0123] After collecting the dynamic environment monitoring data in this embodiment, the state quantity is distinguished and directly transmitted, and the analog quantity is filtered according to the threshold. Data redundancy is used to improve reliability, and anomalies are detected by the average value and standard deviation. For missing data, the LSTM and KNN model predictions are combined to finally obtain the accurately processed monitoring information, which improves the system monitoring efficiency and data accuracy.

[0124] Embodiment 3;

[0125] like Figure 4 As shown, this embodiment provides an abnormal data detection system for a dynamic environment monitoring system, including:

[0126] Data acquisition module: used to obtain real-time monitoring data of the lower-level monitoring modules in the dynamic environment monitoring system and generate southbound interface logs;

[0127] Data filtering and format conversion module: used to filter the real-time monitoring data in combination with preset thresholds and redundant information, and perform format conversion based on the northbound interface specification to generate a northbound interface log;

[0128] Log comparison and analysis module: used to compare and analyze the southbound interface log and the northbound interface log, and output the abnormal data detection result in combination with the abnormal data after the filtering process.

[0129] Embodiment 4;

[0130] like Figure 5 As shown, this embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the abnormal data detection method described in the above embodiment when executing the computer program.

[0131] Embodiment 5;

[0132] This embodiment provides a computer-readable storage medium, wherein the storage medium stores a computer program. When the computer program is executed by a processor, the abnormal data detection method described in the above embodiment is implemented.

[0133] In the above embodiments provided in the present application, it should be understood that the disclosed methods, systems, devices and media can be implemented in other ways. The above described methods, systems, devices and media embodiments are only schematic. For example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation. Each functional unit can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0134] The units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0135] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. Computer readable media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium, etc.

[0136] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting abnormal data in a dynamic environment monitoring system. It is characterized in that The following steps are involved: S1. Obtain the real-time monitoring data of the lower-level monitoring module in the dynamic environment monitoring system and generate a southbound interface log; S2. Filter the real-time monitoring data in combination with preset thresholds and redundant information, perform format conversion based on the northbound interface specification, and generate a northbound interface log; S3. Compare and analyze the southbound interface log and the northbound interface log, and output an abnormal data detection result in combination with the abnormal data after the filtering process.

2. The abnormal data detection method of a dynamic environment monitoring system according to claim 1, It is characterized in that The southbound interface log and the northbound interface log include: the name of the monitoring point, the name of the device to which the monitoring point belongs, the space name of the device to which the monitoring point belongs, the monitoring point number, the monitoring indicator value and the receiving time.

3. The abnormal data detection method of a dynamic environment monitoring system according to claim 1, It is characterized in that In S2, filtering the real-time monitoring data in combination with a preset threshold value includes: For the state quantity data in the real-time monitoring data, a direct transmission strategy is implemented; For the analog data in the real-time monitoring data, filtering is performed based on the preset minimum and maximum values; if the monitoring indicator value of a certain monitoring point in the analog data is within the preset minimum and maximum value range, the transmission strategy is implemented, otherwise, the monitoring indicator value is filtered and stored in the abnormal monitoring data list.

4. The abnormal data detection method of a dynamic environment monitoring system according to claim 1, It is characterized in that In S2, filtering the real-time monitoring data in combination with redundant information includes: S21, constructing a redundant information group according to the power topology connection relationship, the HVAC topology connection relationship, and the spatial position relationship; the redundant information group includes one main information and at least two redundant information; S22, based on the real-time monitoring value of the redundant information, the real-time monitoring value of the main information is filtered using a change rate rule, and stored in the abnormal monitoring data list; the change rate rule is: Among them, X t is the value of the main information at time t, It represents the average value of redundant information at time t; r is the rate of change, which indicates the similarity between the main information and the redundant information.

5. The abnormal data detection method of a dynamic environment monitoring system according to claim 1, It is characterized in that The S3 includes: S31, parsing and formatting the southbound interface log and the northbound interface log to obtain parsed and formatted log information; S32, based on the parsed and formatted log information, unify the time base through a timestamp calibration algorithm, and perform sequence alignment in combination with a nearest neighbor matching strategy to obtain the sequence-aligned log information; S33, using the improved hash index to screen log entries of the sequence-aligned log information, and converting the log information into a structured format to obtain log information in a structured format; S34: Perform multi-dimensional anomaly detection and missing value inference on the log information in the structured format, and output an abnormal data detection result.

6. The abnormal data detection method of a dynamic environment monitoring system according to claim 5, It is characterized in that In S34, multi-dimensional anomaly detection includes: numerical anomaly detection, change rate anomaly detection and frequency anomaly detection; The numerical anomaly detection determines the anomaly limit based on the standard deviation and the interquartile range to identify outliers in the data; the frequency anomaly detection uses Fourier transform to detect periodic or non-periodic abnormal fluctuations in the data to identify abnormal frequency components.

7. The abnormal data detection method for a dynamic environment monitoring system according to claim 5, It is characterized in that In said S34, missing value imputation includes: missing value imputation based on time series prediction model and context association analysis; The time series prediction model adopts the long short-term memory network LSTM, which memorizes the long-term pattern of the time series through the recurrent neural network structure and predicts the value of the missing data points; the context association analysis adopts the K-nearest neighbor algorithm, which predicts the value of the missing data points through the average value of k neighbors.

8. An abnormal data detection system for a dynamic environment monitoring system, It is characterized in that include: Data acquisition module: used to obtain real-time monitoring data of the lower-level monitoring modules in the dynamic environment monitoring system and generate southbound interface logs; Data filtering and format conversion module: used to filter the real-time monitoring data in combination with preset thresholds and redundant information, and perform format conversion based on the northbound interface specification to generate a northbound interface log; Log comparison and analysis module: used to compare and analyze the southbound interface log and the northbound interface log, and output the abnormal data detection result in combination with the abnormal data after the filtering process.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the abnormal data detection method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the abnormal data detection method according to any one of claims 1 to 7 is implemented.