A data center monitoring and early warning system and method based on big data analysis

Abstract

The application discloses a data center monitoring and early warning system and method based on big data analysis, which is composed of a sensor data acquisition module, a log data conversion module, a multi-dimensional feature fusion module, a state intelligent diagnosis module and an early warning response processing module. The application builds a core processing chain of intelligent monitoring of the data center, forms a collaborative optimization system of data acquisition-feature fusion-intelligent diagnosis-closed loop response, and significantly improves the capturing ability of transient anomalies and compound fault modes through environmental mutation capture and log semantic enhancement technology at the data input end. The overall technical architecture realizes autonomous optimization through online model evolution and elastic resource scheduling, so that the data center monitoring and early warning system has the core ability to continuously adapt to complex operating environments and equipment performance degradation, and finally achieves the triple improvement of fault early warning foresight, resource allocation efficiency and system operation economy.

Description

Technical Field

[0001] This invention belongs to the field of information technology, and more specifically, relates to a data center monitoring and early warning system based on big data analysis. Furthermore, this invention also relates to a data center monitoring and early warning method based on big data analysis. Background Technology

[0002] Data center infrastructure monitoring systems commonly employ a technical architecture combining sensor networks and log analysis, triggering alarms for abnormal events through expert rules and fixed threshold mechanisms. Multi-source data is processed independently: physical sensor data is directly input into the anomaly detection model after simple filtering, while log data is converted into structured event entries through regular expression matching. The alarm response module executes a linear notification process based on a preset rule base. System parameters and analysis models require offline updates and maintenance. This type of technical solution has been widely applied in data center temperature control and critical equipment health monitoring scenarios.

[0003] However, existing technologies still have significant shortcomings. Static data acquisition strategies cannot capture transient environmental anomalies, leading to a high rate of missed detections for gradient jump-type faults; log processing lacks spatiotemporal correlation analysis, making it impossible to identify recurring event patterns and trace the root causes of faults; rigid alarm response logic easily triggers the risk of cascading alarms; and lagging model iteration causes the baseline to gradually deviate from the actual operating level after equipment performance degradation. Therefore, we propose a data center monitoring and early warning system and method based on big data analytics. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a data center monitoring and early warning system and method based on big data analysis. This system constructs a data processing chain for intelligent monitoring of data centers, forming a collaborative optimization system of data acquisition, feature fusion, intelligent diagnosis, and closed-loop response.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A data center monitoring and early warning system based on big data analytics includes: The sensor data acquisition module captures real-time physical parameters of the data center environment through multiple distributed temperature, humidity, power, and rack airflow physical sensors; it also captures device port traffic and packet loss rate through network traffic probes and obtains device CPU / memory resource data through device interface polling. The log data conversion module collects semi / unstructured text data from device operation logs, operating system event logs, and application software logs through a lightweight agent program deployed on servers and network devices. The multi-dimensional feature fusion module receives raw data from the sensor data acquisition module and event data from the log data conversion module, and performs aggregated counting of similar events within a time window. The intelligent status diagnosis module receives feature vectors output from the multi-dimensional feature fusion module, uses a machine learning model, and combines dynamic base learning results to compare the real-time feature vectors with the expected status range, and performs a comprehensive quantitative score of equipment resource status, environmental health index, and potential performance bottlenecks. The early warning response processing module receives the status evaluation results and fault risk set from the status analysis module, and triggers multi-level responses based on predefined rules and severity.

[0006] Preferably, the system achieves multi-module collaboration through hierarchical data flow and bidirectional control links. The sensor data acquisition module is directly connected to physical sensors and network probes, receiving real-time physical environment parameters and device performance readings, and outputting the cleaned and aligned structured raw data stream to the multi-dimensional feature fusion module. The log data conversion module independently collects device and system logs, receives raw text data through a proxy program or log interface, performs semantic parsing, and outputs a standardized event data stream to the multi-dimensional feature fusion module. Simultaneously, the key real-time alarm tags identified by the module are directly sent to the early warning response processing module for rapid bypass response.

[0007] Preferably, the multidimensional feature fusion module synchronously receives raw data from the sensor data acquisition module and event data from the log data conversion module. After performing feature fusion and statistical calculation, it transmits the generated multidimensional dynamic feature vector to the state analysis module. The state intelligent diagnosis module receives the feature vector and performs the analysis and diagnosis function of the machine learning model, and pushes the state judgment result with quantitative score and diagnostic label to the early warning response processing module.

[0008] Preferably, the log data conversion module has a built-in dynamic event tag extension unit that dynamically generates composite event tags based on the semantic correlation of the logs collected in real time. When a preset keyword combination is parsed, a high-order event tag set is automatically created. This tag set is synchronously input into the multi-dimensional feature fusion module to enhance feature correlation analysis, and at the same time, it triggers the log data conversion module to send an enhanced alarm notification with priority marking to the early warning response processing module.

[0009] Preferably, the multi-dimensional feature fusion module integrates an environmental change capture submodule, which performs non-fixed-period window dynamic segmentation on the physical environment parameter sequence input by the sensor data acquisition module, identifies temperature and humidity gradient jump intervals that exceed the preset change rate, and automatically increases the sampling rate of the original data in the interval by five times to generate high-precision feature fragments.

[0010] Preferably, the early warning response processing module includes a response path intelligent orchestration submodule. When it receives the composite anomaly diagnosis result issued by the status intelligent diagnosis module, it automatically matches the optimal response sequence based on the historical operation record library: priority level 1 events initiate a parallel notification mechanism, priority level 2 events only trigger email notifications and associate the knowledge base recommended processing solutions, and at the same time, all response actions are generated into versioned operation logs and sent back to the log data conversion module.

[0011] Preferably, the multi-dimensional feature fusion module has a built-in cross-source feature fusion unit that performs three-dimensional spatial alignment calculations on temperature and humidity sensor data from the same physical area, CPU load characteristics of the corresponding rack server, and traffic characteristics of associated network devices to generate a device cluster-level collaborative operation status indicator. This indicator serves as an independent feature vector input to the hierarchical diagnostic model of the intelligent diagnostic module.

[0012] Preferably, the state intelligent diagnosis module is coupled with the diagnostic backtracking analyzer. When environmental parameters of a continuous over-alert state are detected, the module automatically reverse-searches the original feature sequence stored in the multi-dimensional feature fusion module, extracts the feature change pattern within a specific time period before the anomaly occurs, and generates an event source map. This map is attached to the alarm notification issued by the early warning response processing module to enrich the evidence chain and assist in human decision-making.

[0013] A data center monitoring and early warning method based on big data analytics, the method being based on the aforementioned system operation, includes the following steps: S1. Real-time sensor data acquisition: Physical environment parameters and key equipment performance parameters, including temperature, humidity, airflow pressure and equipment energy consumption data, are acquired at a dynamic frequency through a distributed sensor node network, generating a timestamp-synchronized structured sensor data stream. S2. Intelligent Log Data Parsing: Collects device and system log text streams, extracts key event entities and operation behavior tags through a semantic analysis engine, and generates structured standard event records. Once a key real-time alarm tag is identified, it is directly sent to the early warning response processing module for rapid bypass response; S3. Multi-source feature fusion calculation: The sensor data stream and the structured standard event records of the log are fused together, cross-dimensional correlation analysis is performed, and a dynamic feature vector with fused spatiotemporal features is generated. S4. Intelligent diagnosis of operating status: Input the dynamic feature vector into the machine learning model and output the equipment status diagnosis results with confidence scores and anomaly type labels; S5, Multi-level Early Warning Dynamic Response: Triggers a graded response protocol based on diagnostic results: Level 1 alarms execute parallel notifications and automatic control, and Level 2 alarms push processing suggestion logs; S6. Closed-loop feedback optimization: Collect early warning response records and manual correction feedback to drive automatic iterative incremental updates of feature calculation strategies and diagnostic models.

[0014] The technical effects and advantages of this invention are as follows: This invention provides a data center monitoring and early warning system based on big data analysis. Compared with existing technologies, this invention adopts a dynamic adaptive sampling frequency algorithm, which dynamically increases the sampling density by real-time analysis of the second-order rate of change of environmental physical quantities, ensuring that gradient jump events are fully captured; through the event aggregation and weighting algorithm of the log time-series compression engine, it fuses the spatiotemporal density features of events to generate high-order semantic tags and establishes a fault transmission causal chain between discrete logs; through the three-dimensional spatial alignment calculation of the cross-source feature fusion unit, it constructs a device cluster-level collaborative status evaluation index to accurately identify systemic resource contention within the single-point monitoring blind zone; based on the dynamic threshold model of alarm adaptive convergence control, it implements an intelligent suppression strategy according to the root cause correlation of events to avoid redundant alarm notifications; and by utilizing the feature importance feedback mechanism of the incremental learning unit, it dynamically iterates and optimizes the diagnostic model weights based on manual verification results, continuously reducing the false alarm rate. Attached Figure Description

[0015] Figure 1 This is a flowchart of the data center monitoring and early warning method based on big data analysis according to the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0017] This invention provides, for example Figure 1 This paper presents a data center monitoring and early warning system and method based on big data analytics. It constructs a core processing chain for intelligent data center monitoring, forming a collaborative optimization system of data acquisition, feature fusion, intelligent diagnosis, and closed-loop response. At the data input end, environmental mutation capture and log semantic enhancement technologies significantly improve the ability to capture transient anomalies and complex fault modes. At the analysis center layer, relying on cross-dimensional feature association models and dynamic baseline learning mechanisms, it achieves accurate identification and fault diagnosis of system-level hidden risks. At the response output end, intelligently orchestrated alarm paths and self-converging control strategies effectively reduce operational interference. The overall technical architecture achieves autonomous optimization through online model evolution and elastic resource scheduling, enabling the data center monitoring and early warning system to continuously adapt to complex operating environments. Ultimately, it achieves a triple improvement in fault early warning foresight, resource allocation efficiency, and system operation and maintenance economy.

[0018] The system consists of a sensor data acquisition module, a log data conversion module, a multi-dimensional feature fusion module, a state intelligent diagnosis module, and an early warning response processing module; The sensor data acquisition module captures real-time physical parameters of the data center environment through multiple distributed physical sensors for temperature, humidity, power consumption, and rack airflow. It also captures device port traffic and packet loss rate via network traffic probes and obtains device CPU / memory resource data through polling of device interfaces. The module includes a built-in data cleaning unit to eliminate invalid readings and abrupt noise, and performs timestamp alignment to form a structured raw data stream. It provides continuous raw physical layer data and network performance data streams to the multi-dimensional feature fusion module and receives sampling frequency and sensor health status monitoring commands set by the system configuration module. Among them, the sensor data acquisition module is equipped with a distributed cache relay unit. When the network communication fails, a local data temporary storage mechanism is activated to store the original physical parameters for up to 72 hours in a ring buffer structure. After the network is restored, the time window data corresponding to the abnormal events marked by the state intelligent diagnosis module is prioritized to ensure that the multi-dimensional feature fusion module obtains a continuous and effective time-series data stream.

[0019] The log data transformation module collects semi / unstructured text data from device operation logs, operating system event logs, and application software logs through lightweight agents deployed on servers and network devices, or through a centralized log server interface. The module has a built-in natural language parsing unit that categorizes the raw logs according to preset rules, such as errors, alarms, and operation records, and converts them into a standard semantic event format with event type, device ID, and timestamp. The converted standardized log event data stream is then input into the feature fusion module. Furthermore, the log data conversion module has a built-in dynamic event tag extension unit that dynamically generates composite event tags based on the semantic correlation of the logs collected in real time. When a preset keyword combination is parsed, a high-order event tag set is automatically created. This tag set is synchronously input into the multi-dimensional feature fusion module to enhance feature correlation analysis, and at the same time triggers the log data conversion module to send an enhanced alarm notification with priority marking to the early warning response processing module. It should be noted that the log data conversion module integrates a log time-series compression engine. For similar event logs that occur repeatedly within a preset period for the same device ID, it automatically merges them to generate aggregated log records with the first occurrence timestamp, the last occurrence timestamp, and the cumulative number of occurrences. These records are converted into a standard semantic event format and input into the multi-dimensional feature fusion module, which significantly reduces the computational load of the multi-dimensional feature fusion module. The log time-series compression engine executes an event aggregation weighting algorithm: ,in, This represents the final weight value of the aggregated events. This is the event type coefficient. As a quantity percentage factor, This is the standardized value of the cumulative number of times the event occurs within the aggregation window. To aggregate time spans, The maximum aggregation time window allowed by the system; In addition, the log data conversion module adds a log credibility verification unit, which verifies the integrity of logs from different collection sources according to the digital signature rules preset by the device manufacturer, filters invalid or tampered logs, and adds credibility tags to the standard event format. The status intelligent diagnosis module can dynamically adjust the weight allocation strategy of feature vectors based on the credibility tags to ensure the reliability of the analysis conclusions.

[0020] The multi-dimensional feature fusion module receives raw data from the sensor data acquisition module and event data from the log data conversion module. For physical environment data, it performs sliding window statistical calculations, such as temperature gradient changes and average / peak power; for network traffic, it calculates rate fluctuations and connection anomaly probabilities within a time window; and for log events, it aggregates and counts similar events within a time window. It constructs a multi-dimensional feature vector containing time-series statistical and correlation features. The input data for this module comes directly from the sensor data acquisition and log data conversion modules, and the generated dynamic multi-dimensional feature vector is output to the state analysis module for in-depth analysis. The processing window size is controlled by the system configuration module. Furthermore, the multi-dimensional feature fusion module integrates an environmental change capture sub-module. For the physical environment parameter sequence input by the sensor data acquisition module, it performs non-fixed periodic window dynamic segmentation, identifies temperature and humidity gradient jump intervals that exceed the preset change rate, and automatically increases the sampling rate of the original data in the interval by several times to generate high-precision feature fragments, significantly enhancing the sensitivity of the state intelligent diagnosis module to detect transient abnormal events. The environmental mutation capture submodule employs a dynamic adaptive sampling frequency algorithm: ,in, The actual sampling frequency at time t. The preset base sampling frequency is β, where β is the environmental sensitivity coefficient. The second derivative of the environment vector parameters. This is a vector of environmental parameters acquired in real time by the sensor data acquisition module.

[0021] In addition, the multi-dimensional feature fusion module has a built-in cross-source feature fusion unit that performs three-dimensional spatial alignment calculations on temperature and humidity sensor data from the same physical area, the CPU load characteristics of the corresponding rack server, and the traffic characteristics of associated network devices to generate a device cluster-level collaborative operation status evaluation index. This index serves as an independent feature vector input to the multi-level diagnostic model of the intelligent diagnostic module, effectively identifying systemic resource contention problems that traditional single-point monitoring cannot detect. The multi-dimensional feature fusion module is configured with a traffic pattern quantizer to identify the traffic fingerprint characteristics of TCP retransmission storms and UDP flooding attacks within a sliding time window for network traffic data. It generates an attack strength index by calculating the packet rate mutation coefficient and the abnormal connection growth slope. This index is independently input into the state intelligent diagnosis module to trigger a special network security diagnosis process.

[0022] The intelligent status diagnosis module receives feature vectors from the multi-dimensional feature fusion module. Utilizing machine learning models, neural network models, and expert systems, it compares the real-time feature vectors with the expected status range based on a dynamic baseline learned from historical data, to perform a comprehensive quantitative score of equipment resource status, environmental health index, and potential performance bottlenecks. This module includes a built-in outlier detection unit to identify patterns exceeding normal thresholds; its core input is the output of the multi-dimensional feature fusion module; and it sends the analyzed quantitative status assessment results, including normal, warning, and danger levels, and key abnormal features, to the early warning response module in real time. Specifically, the state intelligent diagnosis module deploys an incremental learning unit to receive manual confirmation results and false alarm correction records from the early warning response processing module. When the false alarm rate of the same event continues to exceed the threshold, online model parameter fine-tuning is initiated. The recognition strategy and equipment lifespan degradation are synchronously evolved through iterative updates of the dynamic baseline learning results. At the same time, the updated model feature weight parameters are transmitted in reverse to the multi-dimensional feature fusion module to correct the statistical calculation strategy. Furthermore, the intelligent state diagnosis module is coupled with the diagnostic backtracking analyzer. When environmental parameters of a continuous state of being above alert are detected, the module automatically retrieves the original feature sequence stored in the multi-dimensional feature fusion module, extracts the feature change patterns within a specific time period before the anomaly occurs, and generates an event tracing map. This map is attached to the alarm notification issued by the early warning response processing module to enrich the evidence chain and assist in human decision-making.

[0023] The early warning response processing module receives status evaluation results and key detection data from the intelligent status diagnosis module. Based on predefined rules and severity, such as drastic changes in the temperature gradient of the air-conditioned area + exceeding temperature limits = emergency fault, it triggers multi-level response channels: notifying administrators and corresponding maintenance personnel via console messages, SMS / email alarms; automatically executing preset mitigation procedures for specific high-risk events, such as activating redundant power supplies and enhancing heat dissipation; recording complete alarm event information and operation logs; its core input is the judgment result of the intelligent status diagnosis module; its alarm records and operation audit logs are input into the log data conversion module and uniformly merged into the system log stream; its early warning rule updates based on user feedback need to be submitted to the system configuration module for review. It should be noted that the early warning response processing module includes a response path intelligent orchestration submodule. When it receives the composite anomaly diagnosis result issued by the status intelligent diagnosis module, it automatically matches the optimal response sequence based on the historical operation record library: priority level 1 events initiate a parallel notification mechanism (console + SMS + audible and visual alarm), priority level 2 events only trigger email notifications and associate the knowledge base recommended processing solutions, and at the same time, all response actions are generated into versioned operation logs and sent back to the log data conversion module.

[0024] The early warning response processing module implements alarm adaptive convergence control. For cascaded alarm events of multiple devices caused by the same fault, it automatically merges and generates an aggregated fault certificate report, pushes the main event notification only to the upper-level management system, and automatically suppresses derivative alarms of the same type within 24 hours. This convergence rule set is dynamically maintained and updated by the system configuration module.

[0025] Alarm adaptive convergence control uses an alarm suppression threshold model: ,in, Let t be the suppression threshold. The initial suppression threshold baseline value, It is a time decay function. At the moment the main event occurs, Where t is the attenuation coefficient, t is the current time, and C is the cascade gain factor. These are superimposed terms of events with the same root cause. Let be the severity score of the i-th derived alarm. It is a root source indicator.

[0026] The data interaction relationships between the modules described above constitute a unidirectional data flow chain and a closed-loop feedback: Unidirectional data flow chain: Raw physical / network data is collected and organized by the sensor data acquisition module; raw logs are structured and transformed by the log data conversion module; the two data streams converge to the multi-dimensional feature fusion module for fusion processing and extraction of high-order features; the feature data is input to the intelligent diagnosis module to complete the core anomaly detection and status determination; the diagnosis results are handed over to the early warning response processing module to execute alarm and response actions.

[0027] Closed-loop feedback path: Manual confirmation or correction information and response operation records received by the early warning response processing module flow into the log data conversion module as important log events; early warning rule change applications proposed by the early warning response processing module are reviewed and then updated by the system configuration module to unify the configuration; key real-time alarm events detected by the log data conversion module, such as device port status Down messages, can be prioritized by the early warning response processing module without feature extraction and status analysis.

[0028] This embodiment also proposes a data center monitoring and early warning method based on big data analysis. The method is based on the above-mentioned system operation and includes the following steps: S1. Real-time sensor data acquisition: Physical environmental parameters, including temperature, humidity, air pressure and equipment energy consumption data, are acquired at a dynamic frequency through a distributed sensor node network, generating a raw sensor data stream with timestamp synchronization. Specifically, an intelligent resampling strategy is executed based on the equipment cluster topology. Redundant sensor nodes deployed at the junction of hot and cold aisles in the cabinets monitor local environmental gradient changes in real time. When a heat island effect or sudden change in air pressure is detected, a secondary sampling network in the vicinity is automatically triggered to collect airflow disturbance characteristics and temperature fluctuation curves at a rate three times higher than the base frequency. All sensor data streams are embedded with nanosecond-level timestamps by a time synchronization controller, and data integrity verification and outlier marking are performed by edge computing nodes to form an environmental state matrix with quality labels. S2. Intelligent parsing of log data: Collects text streams of device and system logs, extracts key event entities and operation behavior tags through a semantic analysis engine, and generates structured standard event records; It should be noted that a multi-level semantic processing engine is used to achieve adaptive parsing: at the syntax analysis layer, regular expressions are used to match predefined key opcode sequences; at the semantic association layer, an event propagation graph based on device topology is constructed to identify abnormal operation chains across devices; at the context understanding layer, historical alarm records and current running status are integrated to label events with root cause inference tags; the final output structured standard event record includes the original log hash value, semantic tag confidence, and three-dimensional data attributes of cross-device association chain encoding. S3. Multi-source feature fusion calculation: Fusion of raw sensor data streams and structured standard event records, execution of cross-dimensional correlation analysis, and generation of dynamic feature vectors containing spatiotemporal features; S4. Intelligent Diagnosis of Operating Status: Input dynamic feature vectors into a machine learning model and output equipment status diagnosis results with confidence scores and anomaly type labels. Optionally, a dual-channel diagnostic engine architecture can be built: the main diagnostic channel uses a lightweight convolutional neural network to process environmental feature vectors and generate coarse-grained predictions of fault types; the auxiliary verification channel uses a temporal memory network to trace the historical state evolution trajectory of similar equipment and outputs the state transition pattern matching degree; the results from both channels are weighted by an arbitrator to generate a final diagnostic report with confidence coefficients, where the equipment status diagnosis results include three derived indicators: fault level identifier, impact range heatmap, and equipment health decay rate. S5, Multi-level Early Warning Dynamic Response: Triggers a graded response protocol based on diagnostic results: Level 1 alarms execute parallel notifications and automatic control, and Level 2 alarms push processing suggestion logs; S6. Closed-loop feedback optimization: Collect early warning response records and manual correction feedback to drive incremental updates of feature calculation strategies and diagnostic models; A dual-path feedback-driven mechanism is constructed: On the feature calculation optimization path, the weight allocation strategy and feature combination algorithm of cross-dimensional correlation analysis in S3 are dynamically adjusted by analyzing the false alarm feature distribution of manually amended cases; on the model update path, the gradient magnitude and parameter optimization direction of the incremental learning strategy are automatically selected based on the handling delay time and processing effectiveness score in the early warning response record; the optimization results of the two paths are synchronously pushed to the online update interface of S1 data acquisition and S4 diagnostic model through the configuration distribution bus.

[0029] In summary, compared with existing technologies, this invention adopts a dynamic adaptive sampling frequency algorithm, which dynamically increases the sampling density by analyzing the second-order rate of change of environmental physical quantities in real time, ensuring that gradient jump events are fully captured; and through the event aggregation weighting algorithm of the log time-series compression engine, it fuses the spatiotemporal density features of events to generate high-order semantic tags, and establishes a fundamental fault correlation chain between discrete logs. By leveraging the three-dimensional spatial alignment calculation of the cross-source feature fusion unit, a cluster-level collaborative status index for equipment is constructed to accurately identify systemic resource contention within single-point monitoring blind spots. Based on the dynamic threshold model of alarm adaptive convergence control, an intelligent suppression strategy is implemented according to the root cause correlation of events to avoid redundant alarm notifications. Utilizing the feature importance feedback mechanism of the incremental learning unit, the diagnostic model weights are optimized online based on manual verification results to continuously reduce the false alarm rate.

[0030] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A data center monitoring and early warning system based on big data analytics, characterized in that, include: The sensor data acquisition module captures real-time physical parameters of the data center environment through multiple distributed temperature, humidity, power, and rack airflow physical sensors; it also captures device port traffic and packet loss rate through network traffic probes and obtains device CPU / memory resource data through device interface polling. The log data conversion module collects semi / unstructured text data from device operation logs, operating system event logs, and application software logs through a lightweight agent program deployed on servers and network devices. The multi-dimensional feature fusion module receives raw data from the sensor data acquisition module and event data from the log data conversion module, and performs aggregated counting of similar events within a time window. The intelligent status diagnosis module receives feature vectors from the multi-dimensional feature fusion module, uses machine learning models, neural network models and expert systems, and combines the dynamic baseline learning results trained based on historical data to compare the real-time feature vectors with the expected status range, to perform a comprehensive quantitative score of equipment resource status, environmental health index and potential performance bottlenecks, and diagnose the root cause of the fault, the probability of occurrence and the subsequent impact. The early warning response processing module receives status evaluation and fault diagnosis results from the status intelligent diagnosis module, and triggers multi-level responses based on predefined rules and severity.

2. The data center monitoring and early warning system based on big data analysis according to claim 1, characterized in that, The system achieves module collaboration through hierarchical data flow and bidirectional control link. The sensor data acquisition module is directly connected to physical sensors and network probes to receive real-time physical environment parameters and equipment performance readings, and outputs the cleaned and aligned structured raw data flow to the multi-dimensional feature fusion module. The log data conversion module independently collects device and system logs, receives raw text data through a proxy program or log interface, and outputs a standardized event data stream to the multi-dimensional feature fusion module after semantic parsing. At the same time, the key real-time alarm tags it identifies will be directly sent to the early warning response processing module to achieve a rapid bypass response.

3. The data center monitoring and early warning system based on big data analysis according to claim 2, characterized in that, The multidimensional feature fusion module synchronously receives raw data from the sensor data acquisition module and event data from the log data conversion module. After performing feature fusion and statistical calculation, it transmits the generated multidimensional dynamic feature vector of the fused spatiotemporal sequence to the state intelligent diagnosis module. The state intelligent diagnosis module receives the feature vector and performs fault identification and diagnostic analysis, and pushes the state judgment result with quantitative score and diagnostic label to the early warning response processing module.

4. The data center monitoring and early warning system based on big data analysis according to claim 1, characterized in that, The log data conversion module has a built-in dynamic event tag extension unit that dynamically generates composite event tags based on the semantic correlation of the logs collected in real time. When a preset keyword combination is parsed, a high-level event tag set is automatically created. This tag set is synchronously input into the multi-dimensional feature fusion module to enhance feature correlation analysis. At the same time, it triggers the log data conversion module to send an enhanced alarm notification with priority marking to the early warning response processing module.

5. The data center monitoring and early warning system based on big data analysis according to claim 1, characterized in that, The multi-dimensional feature fusion module integrates an environmental change capture submodule. For the physical environment parameter sequence input by the sensor data acquisition module, it performs non-fixed periodic window dynamic segmentation, identifies temperature and humidity gradient jump intervals that exceed the preset change rate, and automatically increases the sampling rate of the original data in the interval by several times to generate high-precision feature fragments.

6. The data center monitoring and early warning system based on big data analysis according to claim 1, characterized in that, The early warning response processing module includes a response path intelligent orchestration submodule. When it receives the composite anomaly diagnosis result issued by the status intelligent diagnosis module, it automatically matches the optimal response sequence based on the historical operation record library: priority level 1 events initiate a parallel notification mechanism, priority level 2 events only trigger email notifications and associate the knowledge base recommended processing solutions, and all response actions are generated into versioned operation logs and sent back to the log data conversion module.

7. The data center monitoring and early warning system based on big data analysis according to claim 1, characterized in that, The multi-dimensional feature fusion module has a built-in cross-source feature fusion unit that performs three-dimensional spatial alignment calculations on temperature and humidity sensor data from the same physical area, CPU load characteristics of the corresponding rack server, and traffic characteristics of associated network devices to generate a device cluster-level collaborative operation status indicator. This indicator serves as an independent feature vector input to the multi-level diagnostic model of the intelligent diagnostic module.

8. The data center monitoring and early warning system based on big data analysis according to claim 1, characterized in that, The state intelligent diagnosis module is coupled with the diagnostic backtracking analyzer. When environmental parameters of a continuous over-alert state are detected, it automatically retrieves the original feature sequence stored in the multi-dimensional feature fusion module, extracts the feature change pattern within a specific time period before the anomaly occurs, and generates an event source map. This map is attached to the alarm notification issued by the early warning response processing module to assist in manual decision-making.

9. A data center monitoring and early warning method based on big data analysis, the method being based on the system operation according to any one of claims 1-8, characterized in that, Includes the following steps: S1. Real-time sensor data acquisition: Physical environmental parameters, including temperature, humidity, air pressure and equipment energy consumption data, are acquired at a dynamic frequency through a distributed sensor node network, generating a raw sensor data stream with timestamp synchronization. S2. Intelligent parsing of log data: Collects text streams of device and system logs, extracts key event entities and operation behavior tags through a semantic analysis engine, and generates structured standard event records; S3. Multi-source feature fusion calculation: The original sensor data stream and structured standard event records are fused together, cross-dimensional correlation analysis is performed, and a dynamic feature vector containing spatiotemporal features is generated. S4. Intelligent diagnosis of operating status: Input the dynamic feature vector into the machine learning model, neural network model and expert system, and output the equipment status fault mode, subsequent impact and abnormal type label with confidence score; S5, Multi-level Early Warning Dynamic Response: Triggers a graded response protocol based on diagnostic results: Level 1 alarms execute parallel notifications and automatic control, and Level 2 alarms push processing suggestion logs; S6. Closed-loop feedback optimization: Collect early warning response records and manual correction feedback to drive incremental updates of feature calculation strategies and diagnostic models.

Images