Multivariable real-time measurement and state test method for data center

Abstract

The invention relates to the technical field of computers and the field of data testing, in particular to a multivariable real-time measurement and state testing method oriented to a data center. The method comprises the following steps: acquiring original data from a sensor, equipment and an application program in a data center, and filtering, cleaning and formatting the original data to generate preprocessed data; storing the preprocessed data to a distributed file system, and constructing a reverse index and B-tree index structure to realize efficient retrieval and redundant backup; intelligent analysis is carried out on the stored data by introducing a machine learning model and an automatic operation and maintenance strategy, the abnormal state of the key performance index is identified, and the resource use condition is predicted; and based on an intelligent analysis result, executing dynamic scheduling and optimal allocation operation of the resources. Through multi-variable real-time monitoring and intelligent scheduling, the resource utilization efficiency and the operation stability of the data center are improved.

Description

Technical Field

[0001] The present invention relates to the fields of computer technology and data testing, and in particular to a multivariable real-time measurement and state testing method for a data center. Background Art

[0002] With the expansion of data center scale and the diversification of application requirements, resource management and operation and maintenance efficiency of data centers have become important research directions. At present, various sensors and devices in data centers provide rich environmental data and performance data, including key indicators such as temperature and humidity, power consumption, CPU utilization, and memory usage. However, traditional data collection and monitoring methods are mostly limited to monitoring a single indicator, and it is difficult to achieve correlation analysis and dynamic scheduling between multiple variables, resulting in low resource utilization efficiency and slow fault response speed, which cannot effectively meet the requirements of modern data centers for high efficiency and high stability.

[0003] To this end, intelligent monitoring and dynamic management technology based on multi-source sensor data has emerged. Through real-time monitoring and intelligent analysis of multiple variables in the data center, combined with machine learning algorithms to predict system state changes and perform intelligent scheduling, resources can be optimally configured. This technology can not only improve the resource utilization of the data center, but also enhance the operational stability of the system, ensuring the reliability and continuity of the data center in complex environments and high-load conditions. However, there is currently a lack of a complete method to achieve efficient data collection, analysis and scheduling to meet the growing management needs of the data center. Summary of the invention

[0004] The present invention provides a multivariable real-time measurement and status testing method for a data center to solve the problem of how to achieve multivariable real-time monitoring, intelligent analysis and dynamic scheduling based on multi-source sensor data and key performance indicators of the data center, so as to improve the resource utilization efficiency and operation stability of the data center.

[0005] A multivariable real-time measurement and status testing method for a data center, comprising:

[0006] Collecting raw data from sensors, devices and applications in the data center, and performing preliminary filtering, cleaning and formatting on the raw data to generate preprocessed data;

[0007] The pre-processed data is stored in a distributed file system, and an inverted index and a B-tree index structure are constructed to perform efficient retrieval and redundant backup of the data;

[0008] By introducing machine learning models and automated operation and maintenance strategies, the storage data is intelligently analyzed to identify abnormal states of key performance indicators in the data center and predict resource usage;

[0009] According to the intelligent analysis results, dynamic scheduling and optimal allocation of resources are performed.

[0010] Furthermore, the multivariable real-time measurement and status testing method for data centers further includes:

[0011] Collect real-time data from devices and applications in the data center through sensor interfaces and APIs;

[0012] The real-time data is preprocessed by edge computing.

[0013] Furthermore, the multivariable real-time measurement and status testing method for data centers further includes:

[0014] The pre-processed data is stored using a distributed file system and database technology, the data is divided into a plurality of data blocks, and a redundant backup is generated for each data block;

[0015] An inverted index and a B-tree index structure are constructed based on the key fields of the data to achieve multi-dimensional retrieval and fast access to the data.

[0016] Furthermore, the multivariable real-time measurement and status testing method for data centers further includes:

[0017] Using a machine learning model to train historical data in the stored data to generate a prediction model for anomaly detection;

[0018] Based on the prediction model, abnormal status identification and resource usage trend prediction are performed on key performance indicators in real-time monitoring data to generate intelligent analysis results.

[0019] Furthermore, the multivariable real-time measurement and status testing method for data centers further includes:

[0020] Intelligently dispatching resources of the data center according to the intelligent analysis results;

[0021] Based on the intelligent scheduling solution, automatic allocation adjustment of resources is performed.

[0022] Furthermore, the multivariable real-time measurement and status testing method for data centers further includes:

[0023] Detecting abnormal data points in the real-time data through a set threshold, and unifying normal data that meets the threshold into a standard format through data cleaning and format conversion;

[0024] The cleaned data is stored in the edge device, and the cleaning result is transmitted to the distributed file system.

[0025] Furthermore, the multivariable real-time measurement and status testing method for data centers further includes:

[0026] Generate inverted index based on timestamp, device ID and other fields to support keyword query of real-time data;

[0027] Build a B-tree index structure based on data access frequency to implement range query and multi-dimensional screening of historical data.

[0028] Furthermore, the multivariable real-time measurement and status testing method for data centers further includes:

[0029] Based on the pattern changes in historical data, trend forecasts for data center resource usage over the next period of time are made;

[0030] Based on the predicted data, resource allocation optimization suggestions are generated for reference in subsequent intelligent scheduling.

[0031] Furthermore, the multivariable real-time measurement and status testing method for data centers further includes:

[0032] Based on the analysis results, automatically adjust network bandwidth allocation, virtual machine instance expansion or resource pool size to meet the actual load requirements of the data center;

[0033] During the resource optimization process, the intelligent scheduling plan is recorded for dynamic adjustment of future resource allocation.

[0034] Furthermore, the multivariable real-time measurement and status testing method for data centers further includes:

[0035] Visualize resource allocation plans and intelligent analysis results and present them to operation and maintenance personnel in the form of charts and reports to conduct real-time monitoring and management of data center resources.

[0036] Key innovations include:

[0037] (1) Multivariate data processing and storage: Data consistency is ensured through data filtering, cleaning, and standardization. Redundant backup and efficient indexing of large-scale data are achieved using a distributed file system, providing support for complex data analysis.

[0038] (2) Intelligent analysis and prediction: Intelligent analysis based on machine learning models can detect and predict anomalies in key performance indicators and identify problems that may affect system stability in advance.

[0039] (3) Dynamic resource scheduling: Adjust the allocation of server, storage and network resources in real time based on the analysis results to ensure the stable operation of the data center under high load conditions, improve resource utilization efficiency and reduce the need for human intervention.

[0040] The beneficial effects of the present invention include at least:

[0041] The present invention adopts a multivariable real-time measurement and status testing method for data centers. The raw data collected by various sensors in the data center are stored in a distributed file system after preliminary filtering, cleaning and formatting. The inverted index and B-tree index structure are used to achieve efficient retrieval and redundant backup of data. Compared with traditional methods, the present invention greatly improves data retrieval efficiency and supports rapid access to multi-dimensional data by constructing an optimized index structure. In addition, the introduced machine learning model and automated operation and maintenance strategy can identify abnormal states of key performance indicators in the data center and make intelligent predictions on resource usage, thereby achieving dynamic scheduling and optimal allocation of resources, effectively reducing the risk of failure and energy consumption burden of the data center. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 The present invention is a flowchart of a multivariable real-time measurement and status testing method for a data center provided by an embodiment of the present invention.

[0043] Figure 2 It is a structural block diagram of a multivariable real-time measurement and status testing method for a data center provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] Reference Figure 1 , is a flow chart of a multivariable real-time measurement and status testing method for a data center according to an embodiment of the present invention. The flow chart may include at least steps S100-S400:

[0046] S100, collecting raw data from sensors, devices and applications in the data center, performing preliminary filtering, cleaning and formatting on the raw data, and generating preprocessed data.

[0047] S200, storing the pre-processed data in a distributed file system, constructing an inverted index and a B-tree index structure, and performing efficient retrieval and redundant backup of data.

[0048] S300, by introducing machine learning models and automated operation and maintenance strategies, intelligent analysis is performed on the storage data, abnormal states of key performance indicators in the data center are identified, and resource usage is predicted.

[0049] S400: Execute dynamic scheduling and optimal allocation of resources according to the intelligent analysis result.

[0050] Step S100 at least includes steps S110-S140:

[0051] S110 automatically collects real-time and historical data from various devices and applications in the data center by integrating multiple sensor interfaces and APIs. Specifically:

[0052] ① Integrate high-precision temperature and humidity sensors to continuously monitor the environmental conditions of various areas in the data center, including the temperature and humidity of the server room, hot and cold channels, and the inlet and outlet of the air conditioning system to ensure that the equipment is in the best working environment; Power monitoring unit (PMU) and uninterruptible power supply (UPS) interface: directly connect to the communication ports of PMU and UPS to obtain real-time current, voltage, power consumption and battery status information for monitoring the energy efficiency and stability of the data center; use standard interfaces such as RESTfulAPI and SNMP (Simple Network Management Protocol) to extract key performance indicators such as CPU utilization, memory usage, disk I / O, network throughput, etc. from network devices such as servers, switches, and routers regularly or on demand.

[0053] ② By setting a reasonable sampling frequency, the module can capture the real-time data stream of each device and sensor. These data streams include the server's CPU load peak, instantaneous network traffic spikes, or sudden changes in ambient temperature. For long-term trend analysis and historical comparison, the module will store data records over a period of time. These data can be daily, weekly, or monthly averages, or snapshots when specific events are triggered, such as device restarts or system failures.

[0054] ③ Before the data is transmitted to the central processing unit, the edge device will perform preliminary standardization processing on the collected raw data, such as unifying data units, converting data formats (such as from binary to JSON), and removing invalid or erroneous readings to ensure data consistency and quality.

[0055] ④ All data are encrypted during transmission to ensure data security and integrity and avoid being intercepted or tampered with during transmission. In addition, sensitive information such as personal identity data is anonymized during the collection stage to comply with data protection regulations.

[0056] ⑤ Edge devices have preliminary anomaly detection capabilities and can identify abnormal data points based on preset thresholds or historical behavior patterns, such as a sudden surge in server CPU utilization to more than 95%, or an ambient temperature exceeding a safe range, thereby triggering alarms or taking corrective measures in a timely manner.

[0057] S120. Use edge computing technology to perform preliminary processing on the collected data near the data source, including data cleaning, anomaly detection, format conversion and preliminary analysis. Specifically:

[0058] ① When data is collected from sensors, devices, and applications in the data center, the edge computing device immediately starts the data cleaning process. This includes checking the integrity of the data, verifying whether the data field exists, and eliminating or correcting erroneous entries, such as processing missing temperature readings or abnormal CPU utilization values. Through data cleaning, the quality and consistency of the data are ensured, preparing for subsequent analysis and storage.

[0059] ② Using pre-set thresholds and dynamic benchmarks, edge computing devices monitor abnormal patterns in data streams in real time. For example, when it is detected that the CPU utilization of a server suddenly soars to more than 90%, and historical data shows that its normal range should be around 60%, the system will mark the event as abnormal. Anomaly detection helps to quickly identify potential problems, such as equipment overload or system failure, so that timely measures can be taken to avoid operational interruptions in the data center.

[0060] ③ The edge computing device is responsible for converting the raw data in different formats and encodings into a unified format for subsequent analysis and storage. For example, the temperature data received from different sensors is converted from Celsius to Fahrenheit, which is uniformly used within the system, or the binary device status information is converted into JSON format for easy reading and processing by the central server.

[0061] ④After data cleaning, anomaly detection, and format conversion, the edge computing device performs preliminary data analysis. This includes calculating basic statistical indicators such as mean, standard deviation, identifying trends and periodic patterns in the data, and performing simple predictive analysis. For example, predicting the power consumption trend for the next hour based on historical data, or assessing the risk of overheating of equipment in the next few hours. These preliminary analysis results not only help with immediate decision support, but also reduce the amount of data that needs to be transmitted to the central server.

[0062] ⑤ To further reduce the burden of network transmission, edge computing devices compress the processed data and generate data summaries. For example, continuous temperature readings are compressed into average values ​​per minute, or a series of device status updates are summarized into a one-time status report. Through data compression and summarization, only useful data that has been processed and filtered is sent to the central server, significantly reducing the delay in data transmission and the required bandwidth.

[0063] ⑥ Before the data leaves the edge device, all transmitted data will be encrypted to protect the security and privacy of the data. Use secure transmission protocols such as TLS / SSL to ensure that the data will not be intercepted or tampered with during transmission from the edge device to the central server.

[0064] S130. According to predefined rules and algorithms, intelligently filter out useless and duplicate data, aggregate similar data to reduce storage requirements, and maintain data integrity. Specifically:

[0065] ① The data center operation and maintenance management system collects a huge amount of data, including but not limited to equipment status, performance indicators (such as CPU utilization, memory usage, disk I / O), environmental monitoring data (such as temperature, humidity), network traffic, log records, etc. The intelligent filtering mechanism first selects data points that are of practical significance to operation and maintenance management and analysis based on predefined rules and algorithms, and removes redundant or irrelevant information. For example, if consecutive CPU utilization readings do not change much in a short period of time, intelligent filtering will only retain one representative value and ignore the remaining repeated or similar data points, thereby reducing the burden of data transmission and storage. In addition, for abnormal values ​​or obviously erroneous data, intelligent filtering will also eliminate them to avoid interference with data analysis.

[0066] ② The aggregation function further integrates similar or related data based on intelligent filtering to reduce the need for storage space. For example, the readings of multiple sensors of the same type in the same time period can be averaged or summed to generate a single aggregate value; for log data, the aggregation operation involves merging similar events into a single event, recording the number of times the event occurred and the timestamp, which not only maintains the integrity of the data but also saves storage resources; during the aggregation process, the system also considers the time series nature of the data to ensure that time-related data is correctly combined to avoid destroying the time continuity of the data.

[0067] ③ Predefined rules and algorithms are the basis for intelligent filtering and aggregation. These rules include but are not limited to the importance level of data, data type, timeliness of data, and specific business logic. For example, for the operating status data of key equipment, even if the data volume is large, complete records must be kept; for non-critical environmental data, storage can be reduced through more relaxed aggregation strategies. In terms of algorithms, machine learning models are used to automatically identify data patterns and predict which data will become important in the future, thereby deciding to retain or discard certain data points.

[0068] ④ Even during the filtering and aggregation process, the system must ensure that the integrity of the data is not destroyed. This means that when the data is simplified or merged, the key attributes and relationships of the original data must be preserved. For example, the aggregation operation should not affect the statistical characteristics of the data, and the filtering operation should not cause the loss of important information. To ensure this, the system will retain sufficient metadata to record the original state and processing process of the data so that it can be traced and restored when necessary.

[0069] S140, monitor changes in key indicators, and immediately trigger the early warning mechanism when an abnormality is detected, notify the operation and maintenance personnel, and automatically start the corresponding emergency procedures. Specifically:

[0070] ① Collect data in real time from various sensors, devices and applications in the data center, including but not limited to device status, performance indicators (such as CPU utilization, memory usage, disk I / O), environmental monitoring data (such as temperature, humidity), network traffic, log records, etc. Before reaching the real-time monitoring and early warning module, these data have been initially filtered and aggregated by the intelligent edge collection and preprocessing module to reduce noise and improve processing efficiency.

[0071] ② Continuously monitor input data using real-time analysis algorithms that can identify patterns and trends in the data, especially in terms of device performance, network conditions, and environmental conditions. Anomaly detection algorithms determine whether current data deviates from the normal range based on historical data and predefined thresholds. For example, if CPU utilization suddenly soars to 95%, this is an early warning sign that the server is overloaded.

[0072] ③ When the anomaly detection algorithm identifies abnormal changes in key indicators, the real-time monitoring and early warning module responds immediately and triggers the early warning mechanism. The early warning mechanism includes but is not limited to sending alarms to the operation and maintenance team, recording abnormal events, automatically adjusting system configurations, or launching pre-set emergency procedures. Alarms can take many forms, such as emails, text messages, phone calls, or push notifications through integrated communication platforms to ensure that operation and maintenance personnel can respond quickly.

[0073] ④ In some cases, the early warning mechanism is not limited to notifications, it also automatically executes preset emergency procedures to alleviate the problem. For example, if the temperature in a certain area is detected to be too high, the system will automatically increase the power of the cooling system or reroute network traffic to reduce the load on the affected server. Automated response can greatly shorten the problem-solving time and prevent small problems from turning into bigger disasters.

[0074] ⑤After the warning occurs, the system will record the details of the abnormal event, including the time, type, severity and response measures of the abnormality. This information will be used for subsequent analysis to help the operation and maintenance team understand the root cause of the problem and optimize the warning algorithm and emergency procedures. Based on the data of historical warning events, the system can also continuously learn and adjust to improve the accuracy of anomaly detection and the efficiency of response.

[0075] Step S200 at least includes steps S210-S250:

[0076] S210, use distributed file system and database technology to achieve efficient storage and redundant backup of large-scale data. Specifically:

[0077] ① Receive data streams that have been initially cleaned, filtered, and aggregated from the intelligent edge acquisition and preprocessing module. These data include but are not limited to device status, performance indicators, environmental monitoring data, network traffic, and log records. Before the data enters the distributed storage system, additional processing is required, such as data format conversion, metadata addition, and consistency checks to ensure data integrity and readability.

[0078] ② Use a distributed file system such as Hadoop HDFS or a distributed database such as Cassandra to divide the data into multiple smaller data blocks, each of which contains a portion of the data and necessary metadata. These data blocks are then distributed and stored on multiple nodes in the network, with each node storing one or more data blocks, depending on the system configuration and the size of the data blocks.

[0079] ③To ensure high availability and persistence of data, the system creates multiple copies of each data block, and each copy is usually stored on a different node. In this way, even if a node fails, the data can still be recovered from the copies of other nodes, thus avoiding data loss. The data persistence strategy ensures that the data is permanently saved and can be restored even after a node restart or failure. This is usually achieved by periodically writing copies of data blocks to a stable storage medium (such as a hard disk).

[0080] ④ The distributed data storage system provides an efficient indexing and query mechanism, allowing operation and maintenance personnel and analysis modules to quickly retrieve the required data. This includes multiple retrieval methods such as keyword search, time range query, and data type screening. The system also supports real-time access to data and backtracking of historical data, allowing the operation and maintenance team to monitor the status of the data center in real time and conduct in-depth analysis of past events.

[0081] ⑤ In a distributed environment, maintaining data consistency is a major challenge. The system uses mechanisms such as consistent hashing, distributed locks, and distributed transactions to ensure data consistency across multiple nodes. Transaction processing capabilities enable the system to perform atomic operations across multiple data blocks, ensuring data integrity and transaction isolation.

[0082] ⑥ The distributed data storage system itself is also equipped with a monitoring mechanism that can monitor the health status of nodes, the distribution of data blocks, and data access patterns in real time. When a node failure or data block damage is detected, the system can automatically trigger a fault recovery process, such as re-copying the lost data block, migrating the data block to a healthy node, or adjusting the data distribution strategy to ensure the continuous operation of the system.

[0083] S220: Build an inverted index and B-tree efficient index structure to optimize data retrieval speed. Specifically:

[0084] ① Before entering the intelligent data indexing module, the input data has been preliminarily cleaned, filtered and aggregated by the intelligent edge acquisition and preprocessing module. These data include but are not limited to device status, performance indicators, environmental monitoring data, network traffic and log records. The index building process begins with an in-depth analysis of the data to identify key fields and frequently queried attributes, such as device ID, timestamp, exception code, etc. These fields will be used as the basis for index construction. The system uses inverted indexes, B-trees or other efficient index structures to create indexes based on the key fields of the data. Inverted indexes are particularly suitable for text searches, which map keywords to a list of all documents containing the word, while B-trees are suitable for range queries and sorting operations.

[0085] ② As new data continues to flow in, the index must be updated in real time to reflect the status of the latest data. The system adopts an incremental update strategy, which only modifies or adds index items corresponding to new data instead of rebuilding the entire index, thereby maintaining the real-time and efficiency of the index. Index maintenance also includes regular reconstruction and optimization to prevent index expansion and performance degradation. For example, when the amount of data increases, the system will automatically adjust the index segmentation strategy to maintain the compactness of the index structure and query efficiency.

[0086] ③ The intelligent data indexing module has a built-in query optimizer that can analyze query requests and select the optimal index path and query strategy to minimize the data scanning range and improve query speed. The query optimizer also uses a cache mechanism to temporarily store frequently queried results to reduce the overhead of repeated queries and further improve response speed.

[0087] ④Operation and maintenance personnel can submit query requests through the graphical interface or command line tools. These requests involve device status within a specific time range, tracking of abnormal events, trend analysis of performance indicators, etc. The system provides flexible query syntax and filtering options, supports exact matching, fuzzy search, range query and aggregation operations, allowing operation and maintenance personnel to customize query conditions according to specific needs.

[0088] ⑤ The query results are quickly returned to the user in the form of data records, statistical summaries, charts and visual reports, which are easy for operation and maintenance personnel to understand and analyze. The system also supports the export and sharing of results, facilitating collaboration and communication between team members.

[0089] S230, dynamically adjust storage and computing resources according to data volume and processing requirements, and support horizontal expansion. Specifically:

[0090] ① The system continuously monitors current resource usage, including storage capacity, CPU utilization, memory utilization, and network bandwidth, while collecting and analyzing the growth trend of data volume and changes in processing requirements. Based on historical data and current trends, the system can predict future resource requirements and provide a basis for dynamic adjustment of resources.

[0091] ② When it is detected that resource usage is close to the threshold or it is predicted that resource demand will exceed the existing capacity, the system automatically triggers the resource expansion mechanism. Based on the demand assessment results, the system can dynamically add storage nodes or computing nodes to expand storage capacity and computing power. For example, when the amount of data surges, the number of DataNodes of Hadoop HDFS or the number of nodes of the Cassandra cluster can be automatically increased to increase storage space. For computing needs, the system adds Worker nodes of Apache Spark or MapReduce, or automatically expands virtual machine instances in the cloud environment to improve the efficiency of data processing and analysis.

[0092] ③ As resources expand, the system automatically performs load balancing to ensure that newly added nodes can evenly share data storage and processing tasks to avoid hot spots and bottlenecks. Data redistribution strategies ensure that data is evenly distributed among nodes to improve data access efficiency and overall system performance. For example, data blocks are redistributed through hash algorithms to ensure that the amount of data on each node is roughly equal.

[0093] ④ When the demand for data processing decreases or the amount of data tends to stabilize, the system can identify excess resources and automatically recycle unnecessary resources to avoid resource waste and reduce operating costs. The resource recovery process includes reducing the number of queued computing nodes, reducing the number of storage nodes, or releasing unused storage space, while ensuring that data integrity and system stability are not affected.

[0094] ⑤ The elastic expansion capability module adopts an adaptive adjustment strategy to automatically increase or decrease resources based on the real-time monitored system status and pre-set strategies. The system can learn and optimize resource allocation strategies, and over time, continuously improve the accuracy and efficiency of resource scheduling through machine learning algorithms.

[0095] S240, integrated big data analysis framework, supports batch and real-time data analysis, and uses statistics, machine learning and deep learning techniques to mine potential patterns and trends in data. Specifically:

[0096] ① Load the data to be analyzed from the distributed storage system, which includes equipment status, performance indicators, environmental monitoring data, network traffic, and log records. Before entering the analysis process, the data needs to be preprocessed, such as missing value filling, outlier detection, and data type conversion, to ensure data quality and consistency.

[0097] ② Use big data analysis frameworks such as ApacheSpark or Flink to perform batch data analysis tasks. These tasks include data cleaning, data aggregation, statistical analysis, and pattern recognition. ApacheSpark's RDD (Resilient Distributed Dataset) and DataFrame API provide efficient data processing capabilities and can handle PB-level data volumes, while Flink excels in stream processing and can process continuously updated data streams in real time. Batch data analysis is often used for in-depth mining of historical data, such as trend analysis, periodic pattern recognition, and anomaly detection, to assist in operation and maintenance decision-making.

[0098] ③ For scenarios that require immediate response, such as monitoring system health or detecting real-time anomalies, real-time data analysis is key. Flink's stream processing engine can process data streams in real time and provide low-latency analysis results, such as real-time monitoring of CPU utilization, network traffic, or temperature changes. Real-time data analysis supports rule-based anomaly detection and instant alerts, ensuring that operations and maintenance personnel can respond quickly to potential problems.

[0099] ④ Combine machine learning and deep learning algorithms to perform pattern recognition and predictive analysis on data. For example, use regression analysis to predict the remaining service life of equipment, or use clustering algorithms to identify energy consumption patterns in data centers. Deep learning techniques, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), can be used to identify complex patterns in images or time series data, such as abnormal vibration patterns before equipment failure or signs of cyber attacks.

[0100] ⑤ The results of data analysis and mining are presented in an intuitive form, such as charts, dashboards or reports, to help operation and maintenance personnel understand the meaning behind the data. The analysis results can be used to optimize resource allocation, predict equipment maintenance needs, improve energy efficiency or enhance data center security. Based on the data analysis results, the decision support system provides automated suggestions or triggers predefined operation and maintenance actions, such as automatically restarting services, adjusting cooling systems or optimizing network traffic.

[0101] S250 provides intuitive data visualization tools and automatically generated reports to help operation and maintenance personnel understand the meaning behind the data and support decision-making. Specifically:

[0102] ① First, extract the processed data from the elastic distributed storage system, which includes but is not limited to device status information, performance indicators, environmental monitoring data, network traffic statistics, and various application logs. Data comes from multiple sources, including sensors, servers, network devices, and applications, so data integration is required to ensure data consistency and integrity.

[0103] ② Before data visualization, the data needs to be further cleaned and converted to meet the requirements of specific visualization tools and report templates. This stage involves operations such as adjusting the data format, scaling the value range, smoothing the time series, and filtering outliers to ensure the quality of the final data presented.

[0104] ③ Use data visualization tools (such as Tableau, Grafana, or Kibana) to design and generate charts, including line charts, bar charts, pie charts, heat maps, scatter plots, and geographic distribution maps. The specific type depends on the data characteristics and analysis objectives. Chart design needs to consider color coding, labels, and annotations to enhance readability and information communication, so that operation and maintenance personnel can quickly identify key trends and anomalies.

[0105] ④ Reports can be automatically generated at a scheduled time or dynamically generated based on user requests, including key performance indicators (KPIs), abnormal event summaries, historical trend comparisons, and predictive analysis. Custom reports support flexible data filtering and grouping, and operation and maintenance personnel can view data based on specific time periods, equipment types, or geographic locations, thereby deeply analyzing operations in specific areas.

[0106] ⑤ Provide interactive exploration functions, allowing operation and maintenance personnel to explore data details and discover hidden associations and patterns through operations such as drilling down, rolling up, and filtering. The data visualization interface and reports support multi-platform access, facilitating information sharing and collaboration among operation and maintenance team members, ensuring that all relevant parties can make decisions based on the latest data.

[0107] ⑥Data visualization and reporting are not only tools for information display, but can also be used as part of a decision support system. By setting thresholds and early warning mechanisms, they can automatically trigger operation and maintenance strategies or notifications, such as equipment maintenance reminders, resource optimization suggestions, or security incident responses.

[0108] Step S300 at least includes steps S310-S350:

[0109] S310. Use machine learning models to predict the probability of equipment failure, plan maintenance strategies in advance, and reduce unplanned downtime. Specifically:

[0110] ① First, the data collected from the DCIM system includes temperature T, humidity H, current I, voltage V, fan speed F, CPU utilization , memory usage , disk I / O speed D, network bandwidth usage N, etc. These data are standardized and decomposed into different frequency band signals using wavelet transform (WT) to identify patterns on different time scales.

[0111] ②For each signal x(t), apply wavelet transform (a,b), where a is the scale parameter and b is the translation parameter. The wavelet coefficients are given as:

[0112]

[0113] in is the complex conjugate of the wavelet function. This step will help extract features from different scales to capture the changing laws of the device state.

[0114] ③ Take the wavelet coefficients and original data as input to build a dynamic model based on partial differential equations (PDE) to predict the state of the equipment. Define a functional J, which represents the change of the equipment state over time:

[0115]

[0116] Where u is the prediction function of the device state, is the time derivative, ∇u is the spatial gradient, α, β, γ are model parameters, and f(x, t) is the external excitation, i.e., the input function consisting of the wavelet coefficients and the original data. u is solved by minimizing the functional J.

[0117] ④ To solve the above functional minimization problem, the calculus of variations and the finite element method (FEM) are used. This involves solving the variational derivatives of the functional J with respect to u and setting them to zero, resulting in a system of partial differential equations. This system of equations is discretized using the new method, and the optimal solution u is found through an iterative algorithm.

[0118] ⑤ With the prediction u of the equipment status, the failure probability can be calculated , by comparing the deviation of the predicted state with the normal operating range. If the deviation exceeds a predefined threshold θ, a maintenance alarm is triggered.

[0119]

[0120] in This is the normal state of the device.

[0121] ⑥ Establish a closed-loop feedback mechanism to adjust the model parameters α, β, and γ according to the results of the maintenance action to improve the accuracy of the prediction. This involves using the Bayesian update rule to update the prior distribution of the model parameters based on the actual effect of the maintenance action.

[0122] S320: Analyze the resource usage of the data center and make optimization suggestions, such as adjusting server configuration, optimizing network traffic or improving energy efficiency. Specifically:

[0123] 1. Algorithm design

[0124] ①Let X = { , ,..., } is a matrix of real-time and historical data collected from the data center, where each Represents indicators such as CPU utilization, memory usage, network traffic, energy consumption, etc. Let n be the number of features and m be the number of observed samples, then X is an m×n matrix.

[0125] ② Use nonlinear dynamic models to describe the changes in data center resource usage over time. Let y(t) represent the resource usage level at time t, and u(t) be the control vector, representing adjustable parameters such as server configuration, network bandwidth allocation, etc.

[0126]

[0127] Here f is a nonlinear function that describes the relationship between resource usage status and control vector over time.

[0128] ③ Define the objective function J, which aims to minimize the resource usage cost C while maximizing the service quality QoS and energy efficiency EE:

[0129]

[0130] in , , are the weight coefficients of cost, service quality and energy efficiency respectively.

[0131] ④Since resources are limited, constraints need to be set to ensure that resources are not over-allocated or below the threshold:

[0132]

[0133] here Represents the i-th constraint, such as the maximum configuration limit of the server, the upper limit of the network bandwidth, etc.

[0134] ⑤Use the Lagrange multiplier method to transform the constrained optimization problem into an unconstrained optimization problem:

[0135]

[0136] The optimal value of u is solved by gradient descent or Newton-Raphson method, that is:

[0137]

[0138] in represents the gradient with respect to u.

[0139] ⑥ The optimal control vector It is implemented in the resource management of the data center, while monitoring the actual effect and adjusting u and weight w to form a closed-loop control system.

[0140] (II) Operation process

[0141] ① Data collection and feature extraction: Obtain real-time and historical data in X from the data center, and perform preprocessing and feature extraction.

[0142] ② Dynamic system modeling: using nonlinear dynamics models Describes resource usage.

[0143] ③Construct objective function: define J(u) to quantify the economy and efficiency of resource use.

[0144] ④Set constraints: Ensure (u)≤0, to prevent excessive use of resources.

[0145] ⑤ Solve the optimal control vector: Use Lagrange multiplier method and optimization algorithm to solve .

[0146] ⑥Implementation and Feedback: Implementation And monitor the results and adjust the strategy when necessary.

[0147] S330: Based on preset rules and thresholds, the system automatically performs operation and maintenance operations, such as restarting services, adjusting resource allocation, or sending alarms. Specifically:

[0148] Data input sources and indicators: A vector representing all sensor data obtained from the data center at time t, including CPU utilization, memory usage, disk I / O rate, network traffic, and ambient temperature. Indicates the result of an action performed by the system or a change in data center status, such as resource allocation, cooling system adjustments, or energy consumption levels.

[0149] ① In order to accurately simulate the dynamic changes inside the data center, a new set of multivariate differential equations is established to describe the state changes of the data center, which takes into account various factors in time t and space s, including the dynamic changes of computing resources, storage resources, network resources and power resources. The equations are as follows:

[0150]

[0151] in It is a nonlinear function that describes the rate of change of the i-th type of data, which includes the physical characteristics of the data center, the impact of operating instructions, and changes in the external environment; (t,s) represents the status of the i-th resource; X(t,s) is a vector containing the status of all resources; Y(t) is the result of the operation performed by the system; t is time; s is the spatial location of the data center.

[0152] ② In order to find the optimal decision function g(X(t,s)), functional analysis is used to find the function g so that a certain target functional J[g] reaches the minimum value, that is:

[0153]

[0154] Where L is the loss function, which measures the effectiveness of the decision, S is the spatial domain of the data center, g(X(t,s)) is the decision function, L is the loss function, which is used to measure the effectiveness of the decision, and J[g] is the target functional, which represents the overall effect of the decision function.

[0155] ③ In order to solve the above differential equations and functional extremum problems, high-order partial derivatives are used to analyze the sensitivity of the system state to time and space, as well as the dependence on the decision function. Numerical analysis methods, such as finite difference method or spectral method, are used to approximately solve these complex equations.

[0156]

[0157] in is a function that describes the second order rate of change.

[0158] ④ Use the reinforcement learning framework combined with the decision tree algorithm to build a new intelligent decision-making model. Reinforcement learning learns a policy function π(a| ), where a is the action, is the state, and the goal is to maximize the long-term reward R. The decision tree helps the system understand the optimal sequence of actions in different states.

[0159]

[0160] in, is the reward at time t, T is the time step; R is the long-term reward.

[0161] ⑤ Dynamic programming is used to and historical decision sequences, find the optimal decision sequence for the future. At the same time, state estimation technology processes noisy data and predicts future states .

[0162]

[0163] Where F is the state transition function, is the control input, is the process noise.

[0164] ⑥ Dynamically adjust algorithm parameters such as learning rate α and exploration rate ϵ based on decision-making effects and environmental feedback to ensure that the model can adapt to the dynamic changes of the data center.

[0165]

[0166] in is the initial learning rate, η is the decay factor, t is the time step, is the learning rate.

[0167] ⑦ In order to enhance the robustness and computational efficiency of the algorithm, a new topology is introduced to describe the network structure of the data center, and the connectivity and distance metrics in graph theory are used to optimize resource scheduling. In addition, group theory in abstract algebra is used to analyze the symmetry and reversibility of data center operations, helping the system understand which operations can offset or simplify each other.

[0168] G(V,E)

[0169] Where G is the graph representation, V is the vertex set, and E is the edge set.

[0170] ⑧Finally, the result of the operation is Feedback is sent back to the system and compared with the predicted results to evaluate the effectiveness of the decision. If the actual results deviate from the expectations, the model parameters are updated through the back-propagation algorithm to achieve self-optimization.

[0171]

[0172] Where θ is the model parameter, Δθ is the parameter update amount, and η is the learning rate.

[0173] S340, combining historical data and real-time analysis results, provides data-based decision-making suggestions to help the operation and maintenance team make more accurate decisions. Specifically:

[0174] (I) Algorithm model formula

[0175] Let D be the log data set of all sensors, devices, and applications in the data center, where each data point Contains multiple attributes such as CPU utilization , memory usage , disk I / O rate , network traffic , and ambient temperature wait.

[0176] ① First, preprocess the data, including standardization and denoising, to ensure data quality. The preprocessed data is represented as .

[0177] ② Map the preprocessed data to a functional space F, where each data point is represented as a function (t), reflecting the resource usage changing over time t. The functional space allows the use of functional analysis tools to process the time series characteristics of the data.

[0178] ③Construct a multivariate regression model R(⋅) to predict the relationship between resource use and environmental variables:

[0179]

[0180] Where y represents the level of resource utilization and ϵ is the error term. The model can be expressed as:

[0181]

[0182] ④ Use differential equations to simulate the trend of resource usage over time. Assuming that resource usage follows certain dynamic laws, it can be described by the following nonlinear differential equation:

[0183]

[0184] where g(⋅) is a nonlinear function that depends on the current resource usage level y and the environmental variables .

[0185] (II) Algorithm model operation process

[0186] ① Collect log data from sensors, devices, and applications in the data center and pre-process it to ensure the accuracy and availability of the data.

[0187] ② Map the data to the functional space F and use functional analysis methods to extract time series features.

[0188] ③ Use historical data to train the multivariate regression model R(⋅) and determine the coefficient , to accurately predict the relationship between resource usage and environmental variables.

[0189] ④Solve differential equations , and obtain the predicted value of resource usage over time.

[0190] ⑤ Based on the prediction results, generate decision recommendations, such as adjusting resource allocation, optimizing workload, or taking preventive measures.

[0191] ⑥Regularly evaluate decision-making results, update model parameters based on new data, and continuously optimize prediction accuracy.

[0192] S350, through continuous learning and self-optimization, the system can gradually improve the accuracy and efficiency of decision-making and adapt to the dynamic changes of the data center. Specifically:

[0193] ① The system continuously collects real-time data from sensors, devices and applications in the data center, including but not limited to key indicators such as CPU utilization, memory usage, disk I / O rate, network traffic and ambient temperature. These data serve as inputs to the algorithm model and form the basis for self-learning and adaptation.

[0194] ②By implementing anomaly detection algorithms, the system can identify indicator values ​​that deviate from the normal range, which indicates potential problems or impending failures. At the same time, using pattern recognition technology, the system can discover hidden patterns in the data and provide a basis for prediction and decision-making.

[0195] ③ Based on functional analysis and multivariate regression analysis, the model automatically adjusts its parameters according to the latest data. For example, in the differential equation for resource usage prediction, the nonlinear function g(⋅) is adjusted over time to reflect actual changes in the data center environment. This dynamic adjustment ensures the continued relevance and prediction accuracy of the model.

[0196] ④ The system uses an online learning mechanism that can update the model instantly as new data flows in. This means that even when the operating conditions of the data center change, the system can adapt quickly and adjust its decision-making strategy without human intervention.

[0197] ⑤ Decision trees and reinforcement learning algorithms are used to develop specific action plans. Decision trees guide the system to respond based on a series of tests and conditions, while reinforcement learning uses a reward mechanism to let the system learn which behaviors are most conducive to optimizing goals, such as maximizing resource utilization efficiency or minimizing energy consumption.

[0198] ⑥ The system continuously evaluates the effectiveness of decisions through a feedback loop mechanism and collects new data after the decision is executed to further optimize the model. This includes monitoring whether resource usage meets the expected goals, as well as evaluating the system's response speed and the accuracy of decisions.

[0199] ⑦ In addition to short-term online learning, the system also has a long-term memory function, which can accumulate historical data and lessons learned to form a knowledge base. This helps the system make faster and more accurate decisions when faced with similar situations.

[0200] ⑧Once a decision is made, the system can automatically perform corresponding actions, such as adjusting cooling system settings, optimizing workload distribution, or starting backup resources to respond to changes within the data center.

[0201] Step S400 at least includes steps S410-S450:

[0202] S410. Continuously monitor the physical and virtual resources of the data center, including computing, storage, network, and power, to ensure the rational allocation and use of resources. Specifically:

[0203] ① The system automatically collects real-time data from all levels of the data center, including but not limited to:; computing resources: CPU utilization, number of processes, thread load, etc.; storage resources: disk usage, I / O read and write rates, cache hit rate, etc.; network resources: bandwidth usage, latency, packet loss rate, etc.; power resources: power consumption, UPS status, generator readiness, etc.; these data are captured by sensors and monitoring equipment and transmitted to the DCIM system for processing.

[0204] ② The collected data is efficiently stored and classified, and the DCIM system uses advanced data processing technology to analyze it. The system uses statistical analysis, trend prediction and anomaly detection algorithms to ensure that any unusual patterns or potential bottlenecks in resource usage can be discovered in a timely manner.

[0205] ③ The system evaluates resource usage based on data analysis results and generates resource usage reports. These reports include real-time status, historical trends, and forecasting models of resources, helping operation and maintenance personnel understand resource usage efficiency and potential optimization opportunities.

[0206] ④ In order to adapt to the dynamic changes of the data center, the system dynamically adjusts the resource usage threshold based on historical data and current load conditions. When resource usage approaches the threshold, the system will trigger an early warning mechanism to notify the operation and maintenance team to take corresponding measures.

[0207] ⑤ Based on the monitoring results, the system can make resource optimization suggestions, such as dynamically adjusting virtual machine configurations, migrating workloads, optimizing storage strategies, or adjusting network architecture to ensure efficient use of resources.

[0208] ⑥For some common resource allocation problems, the system can automatically execute resource allocation strategies, such as automatically expanding computing nodes, dynamically adjusting the storage pool size, or intelligently allocating network bandwidth to reduce the need for human intervention.

[0209] ⑦ In addition to real-time monitoring, the system also supports long-term resource planning. By analyzing historical data, the system can predict future resource needs and help data center managers plan hardware upgrades or capacity expansions in advance.

[0210] ⑧ Resource monitoring not only focuses on resource utilization efficiency, but also regularly checks security policies and compliance requirements to ensure that all resources are managed in accordance with established security standards and industry regulations.

[0211] S420: Dynamically adjust resource allocation based on current load and forecasted demand, such as dynamically expanding or reducing cloud service instances to balance resource utilization and cost. Specifically:

[0212] ① The intelligent scheduling module continuously collects real-time data from multiple dimensions of the data center, including but not limited to: computing resource indicators such as CPU utilization, memory usage, and disk I / O rate; network performance indicators such as network traffic, latency, and packet loss rate; storage resource indicators such as storage space utilization, read and write speeds; physical environment indicators such as power consumption and ambient temperature; these data are collected through sensors and monitoring equipment, and then efficiently processed by the DCIM system to ensure rapid analysis and response of data.

[0213] ② The system uses machine learning algorithms to conduct in-depth analysis of the collected data, identify current load patterns and historical trends, and then predict future resource needs. The prediction model is trained based on historical data sets and can take into account the impact of seasonal, cyclical and sudden business activities.

[0214] ③ Based on the analysis results of the current load and predicted demand, the intelligent scheduling module will formulate a resource allocation strategy. Including: Dynamic expansion: When the system predicts an upcoming high load, it automatically increases the number of cloud service instances, such as starting additional virtual machines or containers, to cope with the upcoming increase in workload; Resource reduction: When idle resources are detected or demand is predicted to decrease, the system automatically reduces cloud service instances to avoid unnecessary cost expenditures.

[0215] ④ After determining the resource adjustment strategy, the intelligent scheduling module will interact directly with the underlying infrastructure to perform dynamic expansion or reduction of resources. This usually involves communicating with the cloud platform's API interface to adjust virtual machine configuration, container orchestration or storage allocation strategy.

[0216] ⑤After resource adjustment, the system continues to monitor the adjustment effect to verify whether the expected performance goals and cost savings are achieved. If expectations are not met, the system will readjust the resource allocation strategy based on the new data, forming a continuous optimization feedback loop.

[0217] ⑥The intelligent scheduling module also regularly generates cost analysis reports to show how resource adjustments affect overall operating costs, helping data center managers understand the economic benefits brought by intelligent scheduling.

[0218] S430. Design and implement fault recovery strategies to ensure that backup resources can be quickly switched to maintain normal operation of the data center in the event of a fault. Specifically:

[0219] ① The system continuously collects various key indicator data through sensors and monitoring devices throughout the data center, including but not limited to: hardware health status: the operating status of servers, storage devices, and network devices; system performance indicators: CPU utilization, memory usage, disk I / O, network throughput; environmental conditions: temperature, humidity, power supply status; these data are analyzed in real time to identify potential abnormal patterns or performance bottlenecks and trigger early warning mechanisms.

[0220] ② Using big data analysis and machine learning technology, the system can predict the likelihood of equipment failure and assess the impact of potential failures on the overall operation of the data center. This includes: Failure prediction: identifying early signs of failure by analyzing historical data; risk quantification: assessing the impact and recovery time under different failure scenarios, and giving priority to high-risk events.

[0221] ③ When a fault is detected, the system immediately starts an automated process to locate the source of the fault and isolate it to prevent the problem from spreading. This involves: automatic isolation: shutting down or disconnecting faulty components to reduce the impact on other systems; alarm notification: sending instant notifications to the operation and maintenance team, providing fault details and preliminary diagnosis.

[0222] ④ The system is designed with a redundant architecture to ensure seamless switching to backup resources when the primary resource fails. This includes: resource switching: automatically transferring the workload to the backup server or storage array; network redundancy: activating the backup network path to ensure the continuity of data transmission.

[0223] ⑤ For major failures or catastrophic events (such as natural disasters), the system implements pre-planned disaster recovery strategies, including: off-site backup and recovery: recovering data and applications from a remote data center; business continuity plan: enabling pre-defined business recovery processes to ensure the continued operation of key business functions.

[0224] ⑥ After the fault is recovered, the system will review and analyze the entire incident, identify the root cause of the fault, and update the fault recovery strategy and disaster recovery plan to improve the ability to deal with similar incidents in the future.

[0225] S440: Analyze system performance bottlenecks and provide tuning suggestions, such as adjusting cache strategies, optimizing database queries, or improving network configuration to improve system response speed and user experience. Specifically:

[0226] ① The performance tuning module collects data from multiple sources in the data center, including but not limited to: computing resources: CPU utilization, memory usage, disk I / O read and write rates; network resources: network bandwidth usage, latency, packet loss rate; storage resources: disk space usage, storage I / O response time; applications: service response time, error rate, transaction processing speed; databases: query response time, index usage efficiency, cache hit rate; these data are first collected and preprocessed to remove noise, fill missing values ​​and standardize the format for subsequent analysis.

[0227] ② The system uses big data analysis technology and machine learning algorithms to conduct in-depth analysis of the collected data and identify performance bottlenecks. This includes: trend analysis: identifying trends in resource usage over time and predicting future demand; anomaly detection: finding patterns that do not match normal behavior due to improper configuration or hardware failure; correlation analysis: exploring the mutual influence between different resources and applications to understand the overall system behavior.

[0228] ③ Based on the analysis results, the system can locate the specific reasons that lead to performance degradation, such as: insufficient computing resources: CPU or memory overload; network congestion: high latency or low bandwidth; inefficient storage: frequent I / O operations or disk fragmentation; database performance issues: inefficient query or missing indexes.

[0229] ④ Based on the diagnosis results, the system generates specific tuning suggestions, including: adjusting cache strategy: increasing cache size, optimizing cache replacement algorithm; optimizing database query: creating or modifying indexes, rewriting query statements; improving network configuration: adjusting routing strategy, optimizing network protocol; resource allocation optimization: dynamically adjusting computing resource allocation, such as CPU and memory; software configuration adjustment: modifying application configuration parameters, such as thread pool size or connection timeout.

[0230] ⑤ The operation and maintenance personnel implement adjustments based on the tuning suggestions provided by the system. The system then monitors the effects of these changes to verify whether the performance bottleneck has been truly resolved and whether the system response speed and user experience have been improved.

[0231] ⑥Performance tuning is an ongoing process. The system continuously monitors the optimized performance indicators and repeats the above steps regularly to adapt to the dynamic changes and growing needs of the data center.

[0232] S450. Identify cost-saving opportunities through resource usage analysis, such as recycling idle resources, optimizing energy usage, and controlling procurement costs. Specifically:

[0233] ① The cost optimization module collects relevant data from multiple angles of the data center, including but not limited to: equipment status: operating status and health indicators of servers, storage units, and network equipment; resource utilization: real-time and historical usage of CPU, memory, storage, and network; energy consumption data: power consumption, energy consumption statistics of cooling systems; environmental conditions: temperature, humidity, airflow distribution, etc.; maintenance records: historical records of equipment maintenance and predictive maintenance needs; cost information: equipment depreciation, electricity charges, maintenance contracts, and service fees; these data are integrated and pre-processed to form a unified data view for subsequent analysis.

[0234] ② Using big data analysis and machine learning technology, the module conducts in-depth analysis of the collected data and establishes a cost model. The analysis focuses on: Resource efficiency: identifying which resources (such as servers, storage) are over-configured or under-utilized; Energy consumption optimization: analyzing energy consumption patterns and identifying energy-saving opportunities, such as adjusting equipment operating hours or improving thermal management; Cost trends: predicting future costs based on historical data and market trends.

[0235] ③ Based on the results of data analysis, the module identifies opportunities to reduce costs: Idle resource recovery: Automatically identify resources that have not been used for a long time, and make suggestions for reallocation or recycling to reduce waste; Optimize energy use: Intelligently adjust power supply and cooling strategies according to equipment load and environmental conditions to reduce energy consumption; Procurement cost control: Through predictive analysis, optimize the timing of purchasing spare parts and new equipment to avoid additional costs caused by inventory backlogs or emergency purchases.

[0236] ④ The operation and maintenance team takes actions based on the recommendations of the cost optimization module, such as: adjusting resource allocation: reallocating or shutting down idle servers to optimize storage resources; improving facility management: implementing more efficient cooling solutions and adjusting data center layouts; adjusting procurement strategies: formulating procurement plans based on predictive analysis and negotiating more favorable contract terms; after implementation, the system continuously monitors changes in cost and resource usage and evaluates the effectiveness of optimization measures.

[0237] ⑤ Cost optimization is an ongoing process. The system will regularly execute the above steps and make dynamic adjustments based on the operating conditions of the data center and external market conditions to ensure that the cost control strategy is always effective.

[0238] Key innovations include:

[0239] (1) Multivariate data processing and storage: Data consistency is ensured through data filtering, cleaning, and standardization. Redundant backup and efficient indexing of large-scale data are achieved using a distributed file system, providing support for complex data analysis.

[0240] (2) Intelligent analysis and prediction: Intelligent analysis based on machine learning models can detect and predict anomalies in key performance indicators and identify problems that may affect system stability in advance.

[0241] (3) Dynamic resource scheduling: Adjust the allocation of server, storage and network resources in real time based on the analysis results to ensure the stable operation of the data center under high load conditions, improve resource utilization efficiency and reduce the need for human intervention.

[0242] The beneficial effects of the present invention include at least:

[0243] The present invention adopts a multivariable real-time measurement and status testing method for data centers. The raw data collected by various sensors in the data center are stored in a distributed file system after preliminary filtering, cleaning and formatting. The inverted index and B-tree index structure are used to achieve efficient retrieval and redundant backup of data. Compared with traditional methods, the present invention greatly improves data retrieval efficiency and supports rapid access to multi-dimensional data by constructing an optimized index structure. In addition, the introduced machine learning model and automated operation and maintenance strategy can identify abnormal states of key performance indicators in the data center and make intelligent predictions on resource usage, thereby achieving dynamic scheduling and optimal allocation of resources, effectively reducing the risk of failure and energy consumption burden of the data center.

[0244] Figure 2 This is a structural block diagram of a multivariable real-time measurement and status testing method for a data center provided by an embodiment of the present invention. The system may include the following modules:

[0245] Data acquisition module 10, the data acquisition module is used to obtain real-time status information from various data sources in the data center, including temperature and humidity sensors, power monitoring units, equipment performance data (such as CPU utilization, memory usage, disk I / O) and network traffic. Through standardized interfaces (such as RESTful API, SNMP), it ensures that the system can automatically collect environmental and performance parameters from various devices and applications, providing a diverse and timely data source for subsequent data processing.

[0246] Edge computing and preprocessing module 20 : The edge computing and preprocessing module is used to perform preliminary processing on the collected data near the data source, including data cleaning, anomaly detection, format conversion and other operations. Invalid or abnormal data is screened out through intelligent algorithms, effectively reducing the amount of data transmission while ensuring data quality, and ensuring that the data entering the storage and analysis stage is consistent and accurate.

[0247] Data storage and indexing module 30, the data storage and indexing module uses a distributed storage system to efficiently store the processed data, and builds an inverted index and a B-tree structure to optimize the rapid retrieval capability of the data. The system performs distributed storage and redundant backup of the data to ensure high availability of the data in the data center, and supports multi-dimensional queries based on time, device type, and data type.

[0248] The machine learning prediction and anomaly detection module 40 is responsible for predicting the trend of key performance indicators (such as energy consumption, equipment utilization, etc.) of the data center through the trained machine learning model, and performing real-time anomaly detection based on historical data and preset thresholds. When an abnormal change is detected, the system will automatically trigger an early warning to prompt the operation and maintenance personnel to handle it in time to ensure the safe operation of the data center.

[0249] Intelligent scheduling and resource optimization module 50, the intelligent scheduling and resource optimization module is used to intelligently adjust the resource configuration of the data center according to the real-time monitoring data and prediction results. The system can dynamically expand or reduce cloud service instances, adjust load distribution, and optimize resource allocation through adaptive strategies to improve the resource utilization of the data center, reduce energy consumption, and reduce operating costs.

[0250] Data visualization and report generation module 60, the data visualization and report generation module provides an intuitive user interface and visualization tools to display multivariate data and analysis results to users in the form of charts, dashboards, etc. The system supports the generation of real-time and historical reports to help operation and maintenance personnel and managers fully understand the resource usage, performance bottlenecks and optimization suggestions of the data center.

[0251] System monitoring and adaptive module 70, the system monitoring and adaptive module continuously monitors the operating status of each module, and optimizes the system parameters using adaptive learning technology to ensure that the system operates efficiently and stably in a constantly changing environment. The module can adjust system settings based on feedback, improve the accuracy and efficiency of decision-making, and adapt to the dynamic needs of the data center.

[0252] The invention realizes multi-variable real-time measurement and status testing of data centers through the collaborative work of multiple modules. The system not only improves the efficiency of data collection and processing, optimizes resource management and scheduling, and reduces operation and maintenance costs, but also improves the stability and security of data centers through intelligent monitoring and anomaly detection. The data visualization function provides users with a clear and intuitive management interface, helping managers to quickly understand and apply data analysis results, thereby significantly improving the operation and maintenance efficiency and resource utilization of data centers.

[0253] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A multivariable real-time measurement and status testing method for a data center, characterized in that: The following steps are involved: Collecting raw data from sensors, devices and applications in the data center, and performing preliminary filtering, cleaning and formatting on the raw data to generate preprocessed data; The pre-processed data is stored in a distributed file system, and an inverted index and a B-tree index structure are constructed to perform efficient retrieval and redundant backup of the data; By introducing machine learning models and automated operation and maintenance strategies, the storage data is intelligently analyzed to identify abnormal states of key performance indicators in the data center and predict resource usage; According to the intelligent analysis results, dynamic scheduling and optimization allocation operations of resources are performed.

2. The multivariable real-time measurement and status testing method for data center according to claim 1 is characterized in that: The content also includes: Collect real-time data from devices and applications in the data center through sensor interfaces and APIs; The real-time data is preprocessed by edge computing.

3. The multivariable real-time measurement and status testing method for data center according to claim 1 is characterized in that: The content also includes: The pre-processed data is stored using a distributed file system and database technology, the data is divided into a plurality of data blocks, and a redundant backup is generated for each data block; An inverted index and a B-tree index structure are constructed based on the key fields of the data to achieve multi-dimensional retrieval and fast access to the data.

4. The multivariable real-time measurement and status testing method for data centers according to claim 1 is characterized in that: The content also includes: Using a machine learning model to train historical data in the stored data to generate a prediction model for anomaly detection; Based on the prediction model, abnormal status identification and resource usage trend prediction are performed on key performance indicators in real-time monitoring data to generate intelligent analysis results.

5. The multivariable real-time measurement and status testing method for data center according to claim 1 is characterized in that: The content also includes: Intelligently dispatching resources of the data center according to the intelligent analysis results; Based on the intelligent scheduling solution, automatic allocation adjustment of resources is performed.

6. The multivariable real-time measurement and status testing method for data center according to claim 2 is characterized in that: The content also includes: Detecting abnormal data points in the real-time data through a set threshold, and unifying normal data that meets the threshold into a standard format through data cleaning and format conversion; The cleaned data is stored in the edge device, and the cleaning result is transmitted to the distributed file system.

7. The multivariable real-time measurement and status testing method for data center according to claim 3 is characterized in that: The content also includes: Generate inverted index based on timestamp, device ID and other fields to support keyword query of real-time data; Build a B-tree index structure based on data access frequency to implement range query and multi-dimensional screening of historical data.

8. The multivariable real-time measurement and status testing method for data centers according to claim 4 is characterized in that: The content also includes: Based on the pattern changes in historical data, trend forecasts for data center resource usage over the next period of time are made; Based on the predicted data, resource allocation optimization suggestions are generated for reference in subsequent intelligent scheduling.

9. The multivariable real-time measurement and status testing method for data center according to claim 5, characterized in that: The content also includes: Based on the analysis results, automatically adjust network bandwidth allocation, virtual machine instance expansion or resource pool size to meet the actual load requirements of the data center; During the resource optimization process, the intelligent scheduling plan is recorded for dynamic adjustment of future resource allocation.

10. The multivariable real-time measurement and status testing method for data center according to claim 5, characterized in that: The content also includes: Visualize resource allocation plans and intelligent analysis results and present them to operation and maintenance personnel in the form of charts and reports to conduct real-time monitoring and management of data center resources.

Images