Abnormal operation data detection method for IoT big data center

Abstract

The present invention discloses an abnormal operation data detection method applicable to an Internet of Things big data center, which belongs to the field of data detection technology, including: collecting real-time operation data of an Internet of Things big data center; processing the real-time operation data of an Internet of Things big data center to determine the real-time operation characteristic data of the Internet of Things big data center; constructing an abnormal operation data detection model to analyze the real-time operation characteristic data of an Internet of Things big data center and detect abnormal operation data to determine the abnormal operation data detection result; and timely early warning and control of abnormal operation data to ensure the safe operation of the Internet of Things big data center. The present invention solves the existing problem that the abnormal operation data of an Internet of Things big data center cannot be effectively detected, resulting in poor safe operation effect of an Internet of Things big data center. The present invention can effectively detect the abnormal operation data of an Internet of Things big data center, can perform timely early warning and control, and can improve the safe operation effect of an Internet of Things big data center.

Description

Technical Field

[0001] The present invention relates to the field of data detection technology, and in particular to a method for detecting abnormal operation data suitable for an Internet of Things big data center. Background Art

[0002] IoT big data centers are core facilities supporting the storage, processing, and analysis of IoT device data. By integrating cloud computing, big data, and artificial intelligence technologies, they provide efficient, real-time data services for IoT applications. During operation, IoT big data centers generate massive amounts of data, which may contain outliers caused by factors such as device failures, communication interference, malicious attacks, and environmental anomalies. If these outliers are not promptly detected and addressed, they can lead to decreased system performance, deterioration of data quality, and even system crashes.

[0003] Existing technologies cannot effectively detect abnormal operating data in the Internet of Things Big Data Center, and cannot provide timely early warning and control, resulting in poor security operation of the Internet of Things Big Data Center. Summary of the Invention

[0004] The purpose of the present invention is to provide an abnormal operation data detection method suitable for the Internet of Things Big Data Center, which can effectively detect the abnormal operation data of the Internet of Things Big Data Center and carry out timely early warning and control, thereby improving the safe operation effect of the Internet of Things Big Data Center and solving the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] Abnormal operation data detection methods applicable to IoT big data centers include:

[0007] Real-time monitoring of equipment operating status data, environmental monitoring data, network communication data, business processing data, and resource management data during the operation of the IoT Big Data Center, and collection of real-time operating data of the IoT Big Data Center;

[0008] Process the real-time operation data of the Internet of Things Big Data Center, extract key features related to abnormal operation data detection, and determine the real-time operation feature data of the Internet of Things Big Data Center;

[0009] Construct an abnormal operation data detection model to analyze the real-time operation feature data of the IoT big data center and detect abnormal operation data to determine the abnormal operation data detection results;

[0010] Based on the abnormal operation data detection results, timely early warning and control of abnormal operation data can be carried out to ensure the safe operation of the Internet of Things Big Data Center.

[0011] Preferably, the real-time operation characteristic data of the Internet of Things Big Data Center is analyzed and abnormal operation data is detected, and the following operations are performed:

[0012] Deploy the abnormal operation data detection model and deploy it in the actual abnormal operation data detection environment;

[0013] The real-time operation feature data of the Internet of Things Big Data Center is input into the abnormal operation data detection model. The real-time operation feature data of the Internet of Things Big Data Center is analyzed and identified according to the abnormal operation data detection model, and the operation data is effectively detected for anomalies to determine the abnormal operation data detection results.

[0014] Preferably, the real-time operation feature data of the Internet of Things Big Data Center is analyzed and identified according to the abnormal operation data detection model, and the following operations are performed:

[0015] The real-time operation feature data X of the IoT big data center collected at the current time t t ;

[0016] Determine whether the current time t is the first data collection time. If the current time t is the first data collection time, the first data point corresponding to the first data collection time t is assumed to be a normal data point.

[0017] Determine whether the current time t is the second data collection time. If the current time t is the second data collection time, retrieve the real-time operation feature data X of the Internet of Things Big Data Center collected at the previous time t-1 corresponding to the current time t. t-1 ;

[0018] Use the real-time operation feature data X of the Internet of Things Big Data Center collected at the previous time t-1 corresponding to the current time t to t-1 Obtain the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t;

[0019] When the current time t is the third and subsequent data collection time, the real-time operation feature data X of the Internet of Things Big Data Center at the previous time t-1 and the previous two times t-2 are retrieved. t-1 and X t-2 ;

[0020] Utilize the real-time operation characteristic data X of the Internet of Things Big Data Center at the previous moment t-1 and the previous two moments t-2 t-1 and X t-2 Obtain the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t;

[0021] The fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t is used to determine whether there is abnormal data in the real-time operation characteristic data of the Internet of Things Big Data Center.

[0022] Preferably, the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t is used to determine whether there is abnormal data in the real-time operation characteristic data of the Internet of Things Big Data Center, and the following operations are performed:

[0023] Obtain the corresponding information entropy for the real-time operation feature data of the IoT big data center at the current time t;

[0024] Retrieve the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t;

[0025] Using the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t in combination with the information entropy corresponding to the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t, the anomaly score corresponding to the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t is set;

[0026] The anomaly score is compared with a preset anomaly score threshold. When the anomaly score exceeds the preset anomaly score threshold, it is determined that an anomaly exists in the real-time operation feature data of the Internet of Things Big Data Center.

[0027] Preferably, the abnormal operation data is timely managed and controlled based on the abnormal operation data detection results, wherein an abnormal warning is automatically triggered to remind management personnel to pay attention to the operation safety of the Internet of Things Big Data Center, and the abnormal cause of the Internet of Things Big Data Center is determined based on the abnormal operation data. An abnormal resolution plan for the Internet of Things Big Data Center is formulated based on the abnormal cause of the Internet of Things Big Data Center, and the abnormal situation of the Internet of Things Big Data Center is resolved using the abnormal resolution plan for the Internet of Things Big Data Center to ensure the safe operation of the Internet of Things Big Data Center.

[0028] Preferably, real-time operation data of the Internet of Things Big Data Center is collected and the following operations are performed:

[0029] Real-time monitoring of basic status data and performance indicator data during the operation of the IoT big data center, and collection of equipment operation status data;

[0030] Real-time monitoring of the physical environment data and computer room infrastructure data during the operation of the IoT big data center, and collection of environmental monitoring data;

[0031] Real-time monitoring of connection performance data and protocol-related data during the operation of the IoT big data center, and collection of network communication data;

[0032] Monitor data flow indicators and service quality data in real time during the operation of the IoT big data center, and collect business processing data;

[0033] Real-time monitoring of energy consumption data and asset information data during the operation of IoT big data centers, and collection of resource management data;

[0034] The real-time operation data of the Internet of Things Big Data Center is determined based on equipment operation status data, environmental monitoring data, network communication data, business processing data and resource management data.

[0035] Preferably, basic status data includes device online / offline status, power status, working mode and firmware / software version information; performance indicator data includes CPU / GPU usage, memory usage, storage space usage, network bandwidth usage and processing delay indicators;

[0036] Physical environment data includes temperature, humidity, air pressure, air quality, noise level, and vibration amplitude; computer room infrastructure data includes UPS power status, air conditioning operating parameters, cabinet temperature distribution, and fire protection system status;

[0037] Connection performance data includes signal strength, network latency, packet loss rate, retransmission rate, and bandwidth utilization; protocol-related data includes MQTT / CoAP protocol message traffic, TCP / UDP connection count, abnormal disconnection records, and authentication failure logs;

[0038] Data flow indicators include data collection frequency, data throughput, data processing delay, and data backlog; service quality data includes request response time, service success rate, number of concurrent connections, and API call frequency;

[0039] Energy consumption data includes real-time power consumption, energy efficiency ratio, carbon emissions and cooling system energy consumption; asset information data includes equipment life cycle status, maintenance records, spare parts inventory and rental / warranty information.

[0040] Preferably, the real-time operation data of the Internet of Things Big Data Center is processed by performing the following operations:

[0041] Clean the real-time operation data of the IoT big data center, identify the noise data in the real-time operation data of the IoT big data center that is useless for abnormal operation data detection, and remove the identified noise data to reduce the interference of the noise data on abnormal operation data detection;

[0042] The real-time operation data of the Internet of Things Big Data Center is standardized, the real-time operation data of the Internet of Things Big Data Center is converted into a standard normal distribution, the dimensional differences in the real-time operation data of the Internet of Things Big Data Center are removed, and standardized real-time operation data of the Internet of Things Big Data Center is formed.

[0043] Preferably, the real-time operation data of the Internet of Things Big Data Center is processed by performing the following operations:

[0044] Integrate the real-time operation data of the IoT big data center, integrate the real-time operation data of the IoT big data center from different sources into a unified view, and securely store the real-time operation data of the IoT big data center integrated into the unified view;

[0045] Feature extraction is performed on the real-time operation data of the Internet of Things Big Data Center, features related to abnormal operation data detection are extracted from the real-time operation data of the Internet of Things Big Data Center, and real-time operation feature data of the Internet of Things Big Data Center is determined.

[0046] Preferably, a model for detecting abnormal operation data is constructed, and the following operations are performed:

[0047] Collect historical operation data of the Internet of Things Big Data Center and divide the historical operation data of the Internet of Things Big Data Center into training sets and test sets;

[0048] Based on deep learning technology, a training set is used to train the deep learning model, so that the deep learning model can autonomously learn abnormal operation data detection behavior from the training set, effectively detect abnormalities in the operation data, and determine the abnormal operation data detection model based on deep learning;

[0049] Use the test set to test the abnormal operation data detection model based on deep learning, evaluate the performance of the abnormal operation data detection model based on deep learning, determine whether the abnormal operation data detection model based on deep learning can achieve the expected effect of detecting anomalies in operation data, and determine the model test evaluation results;

[0050] When the abnormal operation data detection model based on deep learning cannot achieve the expected effect of detecting anomalies in the operation data, the parameters of the abnormal operation data detection model based on deep learning are adjusted and optimized until the abnormal operation data detection model based on deep learning can achieve the expected effect of detecting anomalies in the operation data, thereby determining the optimal abnormal operation data detection model.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention collects the real-time operation data of the Internet of Things Big Data Center by real-time monitoring of the equipment operation status data, environmental monitoring data, network communication data, business processing data and resource management data during the operation of the Internet of Things Big Data Center. By processing the real-time operation data of the Internet of Things Big Data Center, key features related to abnormal operation data detection are extracted, the real-time operation feature data of the Internet of Things Big Data Center is determined, and the real-time operation feature data of the Internet of Things Big Data Center is analyzed by constructing an abnormal operation data detection model and abnormal operation data is detected to determine the abnormal operation data detection result. According to the abnormal operation data detection result, the abnormal operation data is promptly warned and controlled to ensure the safe operation of the Internet of Things Big Data Center. The abnormal operation data of the Internet of Things Big Data Center can be effectively detected and promptly warned and controlled, thereby improving the safe operation effect of the Internet of Things Big Data Center. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The present invention is a flowchart of a method for detecting abnormal operation data in an Internet of Things big data center. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] In order to solve the existing problem of being unable to effectively detect abnormal operation data of the Internet of Things Big Data Center and unable to conduct timely early warning and control, resulting in poor security operation of the Internet of Things Big Data Center, please refer to Figure 1 , this embodiment provides the following technical solutions:

[0056] Abnormal operation data detection methods applicable to IoT big data centers include:

[0057] Real-time monitoring of equipment operating status data, environmental monitoring data, network communication data, business processing data and resource management data during the operation of the Internet of Things Big Data Center, and collection of real-time operation data of the Internet of Things Big Data Center.

[0058] In this embodiment, real-time operation data of the Internet of Things Big Data Center is collected and the following operations are performed:

[0059] Real-time monitoring of basic status data and performance indicator data during the operation of the IoT big data center, and collection of equipment operation status data;

[0060] It should be noted that basic status data includes the device's online / offline status, power status, operating mode, and firmware / software version information; performance indicator data includes CPU / GPU usage, memory usage, storage space usage, network bandwidth usage, and processing delay indicators.

[0061] Real-time monitoring of the physical environment data and computer room infrastructure data during the operation of the IoT big data center, and collection of environmental monitoring data;

[0062] It should be noted that physical environment data includes temperature, humidity, air pressure, air quality, noise level and vibration amplitude; computer room infrastructure data includes UPS power status, air conditioning operating parameters, cabinet temperature distribution and fire protection system status.

[0063] Real-time monitoring of connection performance data and protocol-related data during the operation of the IoT big data center, and collection of network communication data;

[0064] It should be noted that connection performance data includes signal strength, network latency, packet loss rate, retransmission rate, and bandwidth utilization; protocol-related data includes MQTT / CoAP protocol message traffic, TCP / UDP connection number, abnormal disconnection records, and authentication failure logs.

[0065] Monitor data flow indicators and service quality data in real time during the operation of the IoT big data center, and collect business processing data;

[0066] It should be noted that data flow indicators include data collection frequency, data throughput, data processing delay and data backlog; service quality data includes request response time, service success rate, number of concurrent connections and API call frequency.

[0067] Real-time monitoring of energy consumption data and asset information data during the operation of IoT big data centers, and collection of resource management data;

[0068] It should be noted that energy consumption data includes real-time power consumption, energy efficiency ratio, carbon emissions and cooling system energy consumption; asset information data includes equipment life cycle status, maintenance records, spare parts inventory and rental / warranty information.

[0069] The real-time operation data of the Internet of Things Big Data Center is determined based on equipment operation status data, environmental monitoring data, network communication data, business processing data and resource management data.

[0070] It should be noted that by real-time monitoring of equipment operating status data, environmental monitoring data, network communication data, business processing data and resource management data during the operation of the Internet of Things Big Data Center, and collecting real-time operating data of the Internet of Things Big Data Center, a data basis can be provided for subsequent detection of abnormal operating data of the Internet of Things Big Data Center.

[0071] The real-time operation data of the Internet of Things Big Data Center is processed, key features related to abnormal operation data detection are extracted, and the real-time operation feature data of the Internet of Things Big Data Center is determined.

[0072] In this embodiment, the real-time operation data of the Internet of Things Big Data Center is processed, and the following operations are performed:

[0073] Clean the real-time operation data of the Internet of Things Big Data Center, identify the noise data in the real-time operation data of the Internet of Things Big Data Center that is useless for abnormal operation data detection, and remove the identified noise data to reduce the interference of the noise data on abnormal operation data detection, which can improve the data quality of the real-time operation data of the Internet of Things Big Data Center;

[0074] Standardize the real-time operation data of the Internet of Things Big Data Center and convert it into a standard normal distribution. This removes the dimensional differences in the real-time operation data of the Internet of Things Big Data Center and forms standardized real-time operation data of the Internet of Things Big Data Center to facilitate subsequent analysis of the real-time operation data of the Internet of Things Big Data Center.

[0075] Integrate the real-time operation data of the IoT big data center, integrate the real-time operation data of the IoT big data center from different sources into a unified view, and securely store the real-time operation data of the IoT big data center integrated into the unified view;

[0076] Feature extraction is performed on the real-time operation data of the Internet of Things Big Data Center, features related to abnormal operation data detection are extracted from the real-time operation data of the Internet of Things Big Data Center, and the real-time operation feature data of the Internet of Things Big Data Center is determined, so as to facilitate subsequent better analysis of the real-time operation feature data of the Internet of Things Big Data Center and detection of abnormal operation data.

[0077] An abnormal operation data detection model is constructed to analyze the real-time operation feature data of the Internet of Things Big Data Center and detect abnormal operation data to determine the abnormal operation data detection results.

[0078] In this embodiment, an abnormal operation data detection model is constructed, and the following operations are performed:

[0079] Collect historical operation data of the Internet of Things Big Data Center and divide the historical operation data of the Internet of Things Big Data Center into training sets and test sets;

[0080] Based on deep learning technology, a training set is used to train the deep learning model, so that the deep learning model can autonomously learn abnormal operation data detection behavior from the training set, effectively detect abnormalities in the operation data, and determine the abnormal operation data detection model based on deep learning;

[0081] Use the test set to test the abnormal operation data detection model based on deep learning, evaluate the performance of the abnormal operation data detection model based on deep learning, determine whether the abnormal operation data detection model based on deep learning can achieve the expected effect of detecting anomalies in operation data, and determine the model test evaluation results;

[0082] When the abnormal operation data detection model based on deep learning cannot achieve the expected effect of detecting anomalies in the operation data, the parameters of the abnormal operation data detection model based on deep learning are adjusted and optimized until the abnormal operation data detection model based on deep learning can achieve the expected effect of detecting anomalies in the operation data, thereby determining the optimal abnormal operation data detection model.

[0083] In this embodiment, the real-time operation characteristic data of the Internet of Things Big Data Center is analyzed and abnormal operation data is detected, and the following operations are performed:

[0084] Deploy the abnormal operation data detection model and deploy it in the actual abnormal operation data detection environment;

[0085] The real-time operation feature data of the Internet of Things Big Data Center is input into the abnormal operation data detection model. The real-time operation feature data of the Internet of Things Big Data Center is analyzed and identified according to the abnormal operation data detection model, and the operation data is effectively detected for anomalies to determine the abnormal operation data detection results.

[0086] Based on the abnormal operation data detection results, timely early warning and control of abnormal operation data can be carried out to ensure the safe operation of the Internet of Things Big Data Center.

[0087] In this embodiment, the abnormal operation data is promptly managed and controlled based on the abnormal operation data detection results, wherein an abnormal warning is automatically triggered to remind management personnel to pay attention to the operational safety of the Internet of Things Big Data Center, and the abnormal cause of the Internet of Things Big Data Center is determined based on the abnormal operation data. An abnormal resolution plan for the Internet of Things Big Data Center is formulated based on the abnormal cause of the Internet of Things Big Data Center, and the abnormal situation of the Internet of Things Big Data Center is resolved using the abnormal resolution plan for the Internet of Things Big Data Center to ensure the safe operation of the Internet of Things Big Data Center.

[0088] Specifically, the real-time operation feature data of the IoT big data center is analyzed and identified based on the abnormal operation data detection model, and the following operations are performed:

[0089] The real-time operation feature data X of the IoT big data center collected at the current time t t ;

[0090] Determine whether the current time t is the first data collection time. If the current time t is the first data collection time, since there is no historical data, it is impossible to determine whether it is abnormal (because there is no reference benchmark). According to domain common sense, the system usually has a self-check process when it starts, so the first data point corresponding to the first data collection time t is assumed to be a normal data point;

[0091] Determine whether the current time t is the second data collection time. If the current time t is the second data collection time, retrieve the real-time operation feature data X of the Internet of Things Big Data Center collected at the previous time t-1 corresponding to the current time t. t-1 ;

[0092] Use the real-time operation feature data X of the Internet of Things Big Data Center collected at the previous time t-1 corresponding to the current time t to t-1 Obtain the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t;

[0093] The wave energy operator is obtained by the following formula:

[0094]

[0095] Among them, AFEO 01 The fluctuation energy operator of the real-time operation characteristic data of the IoT big data center when the current time t is the second data collection time;

[0096] When the current time t is the third and subsequent data collection time, the real-time operation feature data X of the Internet of Things Big Data Center at the previous time t-1 and the previous two times t-2 are retrieved. t-1 and X t-2 ;

[0097] Utilize the real-time operation characteristic data X of the Internet of Things Big Data Center at the previous moment t-1 and the previous two moments t-2 t-1 and X t-2 Obtain the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t;

[0098] The wave energy operator is obtained by the following formula:

[0099]

[0100] Among them, AFEO 02The fluctuation energy operator representing the real-time operation characteristic data of the IoT Big Data Center when the current time t is the third and subsequent data collection times;

[0101] The fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t is used to determine whether there is abnormal data in the real-time operation characteristic data of the Internet of Things Big Data Center.

[0102] The technical effect of the above technical solution is: according to the different data collection moments (second, third and subsequent), different wave energy operator formulas (AFEO 01 For the second moment, AFEO 02 For the third and subsequent moments). Given the temporal nature of data collection in IoT big data centers, the amount of historical data available varies at different stages. Phased calculations fully utilize this available historical data, allowing the wave energy operator to more accurately align with the actual data collection process and accurately characterize data fluctuations. For example, at the second moment, only the data from the previous moment can be used, while the data from the third and subsequent moments can be combined. The formula adapts to this data volume difference, improving the targeted nature of the calculation. The first data collection moment is designated as a normal data point by default, aligning with the common knowledge that IoT big data centers typically undergo a self-checking process upon system startup. This prevents subsequent detection processes from being impacted by a lack of historical data. This establishes an initial, normal reference baseline for the entire anomaly detection process, ensuring that the detection process proceeds smoothly from the start.

[0103] In the wave energy operator formula, by introducing the data difference (such as |X t−1 −X t ∣、∣X t −2X t −1+X t−2 ∣) and logarithmic operations can amplify data fluctuations. Logarithmic operations perform nonlinear transformations on data variances, highlighting the degree of fluctuation. This allows the energy operator to more sensitively detect abnormal data fluctuations, helping to accurately identify abnormal operating data and promptly detect abnormal conditions within the IoT big data center, ensuring stable operation. The current fluctuation energy operator is calculated using historical data from the previous and two previous moments, fully exploiting the temporal correlations within the real-time operational data of the IoT big data center. IoT data is typically temporal. This correlation allows for the determination of whether current data is abnormal based on historical data. This consideration of the dynamic nature of data changes makes detection more reliable than is achieved by isolating current data. This allows for the effective identification of different types of anomalies, such as sudden data changes and trend anomalies.

[0104] From case-by-case processing at the moment of data collection, to calculating the fluctuation energy operator at different times, and finally using these operators to identify anomalies, a comprehensive anomaly detection process has been constructed, tailored to the data characteristics of IoT big data centers. This process covers various scenarios from the startup phase to the ongoing data collection phase, continuously analyzing and identifying real-time operational feature data. This meets the 24 / 7 monitoring requirements of IoT big data centers and ensures consistent and effective anomaly detection. Each step is based on clear time determination, data retrieval, and calculation formulas, resulting in clear logic and easy engineering implementation. In actual deployments within IoT big data centers, this process can be used for algorithm coding and system integration to implement anomaly detection functionality, helping operations personnel promptly identify and address anomalies, reducing the risk of system failures and performance degradation caused by anomalies, and improving the reliability and operational efficiency of IoT big data centers.

[0105] Specifically, the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t is used to determine whether there is abnormal data in the real-time operation characteristic data of the Internet of Things Big Data Center, and the following operations are performed:

[0106] Obtain the corresponding information entropy for the real-time operation feature data of the IoT big data center at the current time t;

[0107] The information entropy corresponding to the real-time operation feature data of the Internet of Things Big Data Center at the current time t is obtained by the following formula:

[0108]

[0109] Among them, H t represents the information entropy corresponding to the real-time operation feature data of the IoT Big Data Center at the current time t; k represents the total number of sub-ranges divided when the data in the sliding window is discretized; p i represents the probability that the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t falls into the i-th sub-range after discretization processing; σ represents the standard deviation of the real-time operation characteristic data of the Internet of Things Big Data Center within the sliding window; μ represents the average value of the real-time operation characteristic data of the Internet of Things Big Data Center within the sliding window;

[0110] Retrieve the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t;

[0111] Using the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t in combination with the information entropy corresponding to the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t, the anomaly score corresponding to the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t is set;

[0112] The abnormality score is obtained by the following formula:

[0113]

[0114] Among them, Ascore t represents the anomaly score corresponding to the real-time operation feature data of the IoT big data center at the current time t; α and β represent the first weight and the second weight respectively; AFEO t Fluctuation energy operator representing the real-time operation characteristic data of IoT big data center; AFEO p H represents the change rate of the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center compared with the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center collected at the previous moment; t H represents the information entropy corresponding to the real-time operation feature data of the IoT big data center at the current time t; p represents the rate of change of the information entropy of the real-time operation feature data of the Internet of Things Big Data Center compared to the information entropy of the real-time operation feature data of the Internet of Things Big Data Center collected at the previous moment; δ represents the dynamic fluctuation threshold, and the dynamic fluctuation threshold δ=μ AFEO +r*σ AFEO *(1-e -λ*Ht ), where μ AFEO represents the average value of the fluctuation energy operator of the real-time operation characteristic data of the IoT big data center that has been collected; σ AFEO It represents the standard deviation of the fluctuation energy operator of the collected real-time operation characteristic data of the IoT big data center; r represents the sensitivity factor, which defaults to 2.5; λ is the entropy coupling coefficient, which defaults to 0.8.

[0115] The anomaly score is compared with a preset anomaly score threshold (a value range is 0.6-0.8). When the anomaly score exceeds the preset anomaly score threshold, it is determined that there is an anomaly in the real-time operation feature data of the Internet of Things Big Data Center.

[0116] The technical effect of the above technical solution is: the simultaneous introduction of two types of features: the fluctuation energy operator and the information entropy. The fluctuation energy operator focuses on the instantaneous fluctuation intensity of the data, and the information entropy reflects the complexity of the data distribution. The combination of the two can characterize the data characteristics from the two dimensions of "fluctuation amplitude" and "distribution law". Compared with the existing technology that only uses a single feature (such as relying solely on fluctuation or entropy), it can capture anomalies more comprehensively. For example, it can detect fluctuation anomalies such as data mutations, and can also identify complex anomalies such as chaotic data distribution. Dynamic threshold adaptation: dynamic fluctuation threshold δ = μ AFEO +r*σ AFEO *(1-e -λ*Ht), allowing the threshold to be dynamically adjusted with the complexity of the system. In high entropy states (complex data distribution), the threshold is relaxed to avoid misjudgment; in low entropy states, the threshold is tightened to accurately identify anomalies. In existing technologies, static thresholds are difficult to adapt to dynamic changes in the system. This solution improves the adaptability of the threshold to different operating states, reducing missed detections and false detections; the anomaly scoring formula This approach incorporates the rate of change of the fluctuation energy operator and the rate of change of information entropy, as well as adaptive weights α and β. The rate of change reflects the dynamic trend of the data, and the weights are adaptively adjusted based on data fluctuations and other conditions (for example, when fluctuations are severe, α approaches 1, emphasizing fluctuation detection). This allows anomaly scoring to dynamically reflect the real-time state of the data. Compared to the simple "feature threshold comparison" approach used in existing technologies, this scoring method better captures the essence of data anomalies and achieves higher detection accuracy. Adaptable scoring thresholds: The pre-set anomaly scoring threshold ranges from 0.6 to 0.8 and can be flexibly adjusted based on the actual scenarios (tolerance for missed detections and false detections) of IoT big data centers. Compared to existing technologies that use fixed thresholds or are difficult to adapt to specific scenarios, this solution enhances adaptability to diverse business needs, balancing the risks of missed detections and false detections while ensuring detection effectiveness. Targeting the time series data characteristics of IoT big data centers, the fluctuation energy operator calculation is processed at different moments (the second, third, and subsequent moments), and features are constructed using historical data (the previous and two moments, etc.). This approach leverages the continuous data collection and strong temporal correlations of IoT data. Compared to existing solutions that underutilize temporal characteristics, it can better identify dynamic data anomalies and improve adaptability to IoT scenarios. The default setting of the first data collection moment as the normal point aligns with common sense for system startup self-checks, ensuring that the detection process runs smoothly from the start-up phase. This addresses existing issues with initial system data processing, which can lead to misjudgments due to a lack of historical references. This allows for effective anomaly detection throughout the entire lifecycle of IoT big data centers, including the startup phase.

[0117] At the same time, this technical solution improves accuracy by building a comprehensive and accurate anomaly identification system through multi-dimensional feature fusion and dynamic adaptation mechanisms. On the one hand, it innovatively integrates two types of features: the fluctuation energy operator and the information entropy. The fluctuation energy operator focuses on capturing instantaneous mutations in data, and can keenly perceive data jumps caused by transient equipment failures. The information entropy focuses on characterizing the complexity of data distribution and can issue early warnings for hidden anomalies such as slow changes in data distribution. The two complement each other and cover a variety of anomaly types, including sudden anomalies, hidden anomalies, and complex anomalies (dual anomalies of fluctuation and distribution). This eliminates the blind spots of traditional single-feature detection and lays a solid foundation for accurate anomaly identification from the feature dimension. Furthermore, a dynamic fluctuation threshold is introduced, which automatically adjusts based on system complexity (reflected by information entropy). When the system is in a high-entropy state (i.e., when the data distribution is complex and normal fluctuations are large), the threshold is automatically relaxed to prevent normal fluctuations from being misclassified as anomalies. When the system is in a low-entropy state (i.e., when the data distribution is stable and anomalies are easier to identify), the threshold is automatically tightened to prevent abnormal fluctuations from being missed. Compared to static thresholds, this mechanism significantly reduces the risk of misclassification and missed detection, bringing anomaly identification accuracy to a new level. In terms of the anomaly scoring mechanism, this solution breaks through the limitations of traditional simple threshold comparisons and achieves precise quantification of the degree of anomaly. The anomaly scoring formula cleverly incorporates the rate of change of the fluctuation energy operator and the rate of change of information entropy. This eliminates the focus solely on the current feature value and also considers the "trend" of the data. For example, a sudden acceleration in entropy often indicates a potential anomaly. The introduction of the rate of change makes anomaly identification more proactive. Furthermore, adaptive weighting is set to dynamically adjust based on the actual data fluctuations. When fluctuations are severe, the fluctuation energy operator is prioritized, while when fluctuations are mild, the information entropy is prioritized, ensuring that the scoring closely reflects the nature of the anomaly. This combination of "current value + rate of change + adaptive weighting" transforms anomaly scoring from a simple, crude threshold comparison to a precise quantification of the degree of anomaly, providing a more nuanced and realistic basis for accurate anomaly identification, significantly improving anomaly identification accuracy. Furthermore, to address the unique characteristics of IoT time series data, this technical solution utilizes a phased processing strategy for time series data to achieve efficient adaptation. Taking into account the differences between different stages of IoT data collection—for example, no historical data in the startup phase, only one historical data point in the second phase, and two historical data points in the third phase and beyond—a specifically designed wave energy operator utilizes one historical point in the second phase and two historical points in the third phase and beyond. This phased processing approach effectively avoids the inefficient overhead of forced calculations when no historical data is available, while fully utilizing the minimal amount of historical data for effective calculations. While ensuring anomaly detection functionality, it reduces data storage and computational costs, enabling the anomaly detection process to more efficiently adapt to the full lifecycle of IoT time series data, from startup to ongoing collection, improving overall detection efficiency.

[0118] In summary, by building an abnormal operation data detection model to analyze the real-time operation feature data of the Internet of Things Big Data Center and detect abnormal operation data, the abnormal operation data detection results are determined, and timely early warning and control of abnormal operation data are carried out according to the abnormal operation data detection results to ensure the safe operation of the Internet of Things Big Data Center. The abnormal operation data of the Internet of Things Big Data Center can be effectively detected, and timely early warning and control can be carried out, which can improve the safe operation effect of the Internet of Things Big Data Center.

[0119] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0120] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting abnormal operation data in an Internet of Things big data center, characterized in that: include: Real-time monitoring of equipment operating status data, environmental monitoring data, network communication data, business processing data, and resource management data during the operation of the IoT Big Data Center, and collection of real-time operating data of the IoT Big Data Center; Process the real-time operation data of the Internet of Things Big Data Center, extract key features related to abnormal operation data detection, and determine the real-time operation feature data of the Internet of Things Big Data Center; Construct an abnormal operation data detection model to analyze the real-time operation feature data of the IoT big data center and detect abnormal operation data to determine the abnormal operation data detection results; Analyze and identify the real-time operation feature data of the IoT Big Data Center based on the abnormal operation data detection model, and perform the following operations: The real-time operation feature data X of the IoT big data center collected at the current time t t ; Determine whether the current time t is the first data collection time. If the current time t is the first data collection time, the first data point corresponding to the first data collection time t is assumed to be a normal data point. Determine whether the current time t is the second data collection time. If the current time t is the second data collection time, retrieve the real-time operation feature data X of the Internet of Things Big Data Center collected at the previous time t-1 corresponding to the current time t. t-1 ; Use the real-time operation feature data X of the Internet of Things Big Data Center collected at the previous time t-1 corresponding to the current time t to t-1 Obtain the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t; When the current time t is the third and subsequent data collection time, the real-time operation feature data X of the Internet of Things Big Data Center at the previous time t-1 and the previous two times t-2 are retrieved. t-1 and X t-2 ; Utilize the real-time operation characteristic data X of the Internet of Things Big Data Center at the previous moment t-1 and the previous two moments t-2 t-1 and X t-2 Obtain the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t; Use the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t to determine whether there is abnormal data in the real-time operation characteristic data of the Internet of Things Big Data Center. Perform the following operations: Obtain the corresponding information entropy for the real-time operation feature data of the IoT big data center at the current time t; Retrieve the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t; Using the fluctuation energy operator of the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t in combination with the information entropy corresponding to the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t, the anomaly score corresponding to the real-time operation characteristic data of the Internet of Things Big Data Center at the current time t is set; Comparing the anomaly score with a preset anomaly score threshold, and when the anomaly score exceeds the preset anomaly score threshold, determining that an anomaly exists in the real-time operation feature data of the Internet of Things Big Data Center; Based on the abnormal operation data detection results, timely early warning and control of abnormal operation data can be carried out to ensure the safe operation of the Internet of Things Big Data Center.

2. The abnormal operation data detection method applicable to the Internet of Things Big Data Center according to claim 1 is characterized in that: Analyze the real-time operation feature data of the IoT Big Data Center and detect abnormal operation data. Perform the following operations: Deploy the abnormal operation data detection model and deploy it in the actual abnormal operation data detection environment; The real-time operation feature data of the Internet of Things Big Data Center is input into the abnormal operation data detection model. The real-time operation feature data of the Internet of Things Big Data Center is analyzed and identified according to the abnormal operation data detection model, and the operation data is effectively detected for anomalies to determine the abnormal operation data detection results.

3. The abnormal operation data detection method applicable to the Internet of Things Big Data Center according to claim 2 is characterized in that: According to the abnormal operation data detection results, the abnormal operation data is timely controlled. Among them, the abnormal warning is automatically triggered to remind management personnel to pay attention to the operation safety of the Internet of Things Big Data Center, and the abnormal cause of the Internet of Things Big Data Center is determined based on the abnormal operation data. According to the abnormal cause of the Internet of Things Big Data Center, the abnormal solution plan of the Internet of Things Big Data Center is formulated, and the abnormal situation of the Internet of Things Big Data Center is solved by using the abnormal solution plan of the Internet of Things Big Data Center to ensure the safe operation of the Internet of Things Big Data Center.

4. The abnormal operation data detection method applicable to the Internet of Things Big Data Center according to claim 3 is characterized in that: Collect real-time operation data from the IoT Big Data Center and perform the following operations: Real-time monitoring of basic status data and performance indicator data during the operation of the IoT big data center, and collection of equipment operation status data; Real-time monitoring of the physical environment data and computer room infrastructure data during the operation of the IoT big data center, and collection of environmental monitoring data; Real-time monitoring of connection performance data and protocol-related data during the operation of the IoT big data center, and collection of network communication data; Monitor data flow indicators and service quality data in real time during the operation of the IoT big data center, and collect business processing data; Real-time monitoring of energy consumption data and asset information data during the operation of IoT big data centers, and collection of resource management data; The real-time operation data of the Internet of Things Big Data Center is determined based on equipment operation status data, environmental monitoring data, network communication data, business processing data and resource management data.

5. The abnormal operation data detection method applicable to the Internet of Things Big Data Center according to claim 4 is characterized in that: Basic status data includes device online / offline status, power status, operating mode, and firmware / software version information; performance indicator data includes CPU / GPU usage, memory usage, storage space usage, network bandwidth usage, and processing delay indicators; Physical environment data includes temperature, humidity, air pressure, air quality, noise level, and vibration amplitude; computer room infrastructure data includes UPS power status, air conditioning operating parameters, cabinet temperature distribution, and fire protection system status; Connection performance data includes signal strength, network latency, packet loss rate, retransmission rate, and bandwidth utilization; protocol-related data includes MQTT / CoAP protocol message traffic, TCP / UDP connection count, abnormal disconnection records, and authentication failure logs; Data flow indicators include data collection frequency, data throughput, data processing delay, and data backlog; service quality data includes request response time, service success rate, number of concurrent connections, and API call frequency; Energy consumption data includes real-time power consumption, energy efficiency ratio, carbon emissions and cooling system energy consumption; asset information data includes equipment life cycle status, maintenance records, spare parts inventory and rental / warranty information.

6. The abnormal operation data detection method applicable to the Internet of Things Big Data Center according to claim 5 is characterized in that: Process the real-time operation data of the IoT Big Data Center and perform the following operations: Clean the real-time operation data of the IoT big data center, identify the noise data in the real-time operation data of the IoT big data center that is useless for detecting abnormal operation data, and remove the identified noise data; The real-time operation data of the Internet of Things Big Data Center is standardized, the real-time operation data of the Internet of Things Big Data Center is converted into a standard normal distribution, the dimensional differences in the real-time operation data of the Internet of Things Big Data Center are removed, and standardized real-time operation data of the Internet of Things Big Data Center is formed.

7. The abnormal operation data detection method applicable to the Internet of Things Big Data Center according to claim 6 is characterized in that: Process the real-time operation data of the IoT Big Data Center and perform the following operations: Integrate the real-time operation data of the IoT big data center, integrate the real-time operation data of the IoT big data center from different sources into a unified view, and securely store the real-time operation data of the IoT big data center integrated into the unified view; Feature extraction is performed on the real-time operation data of the Internet of Things Big Data Center, features related to abnormal operation data detection are extracted from the real-time operation data of the Internet of Things Big Data Center, and real-time operation feature data of the Internet of Things Big Data Center is determined.

8. The abnormal operation data detection method applicable to the Internet of Things Big Data Center according to claim 7 is characterized in that: To build an abnormal operation data detection model, perform the following operations: Collect historical operation data of the Internet of Things Big Data Center and divide the historical operation data of the Internet of Things Big Data Center into training sets and test sets; Based on deep learning technology, a training set is used to train the deep learning model, so that the deep learning model can autonomously learn abnormal operation data detection behavior from the training set, effectively detect abnormalities in the operation data, and determine the abnormal operation data detection model based on deep learning; Use the test set to test the abnormal operation data detection model based on deep learning, evaluate the performance of the abnormal operation data detection model based on deep learning, determine whether the abnormal operation data detection model based on deep learning can achieve the expected effect of detecting anomalies in operation data, and determine the model test evaluation results; When the abnormal operation data detection model based on deep learning cannot achieve the expected effect of detecting anomalies in the operation data, the parameters of the abnormal operation data detection model based on deep learning are adjusted and optimized until the abnormal operation data detection model based on deep learning can achieve the expected effect of detecting anomalies in the operation data, thereby determining the optimal abnormal operation data detection model.

Images