Data center user information dynamic management system based on distributed architecture

Abstract

The invention discloses a data center user information dynamic management system based on a distributed architecture. The system comprises a distributed architecture layer, a core function module layer and a data interaction layer, the distributed architecture layer provides distributed computing, storage and communication support for the system. According to the method, tasks are processed in parallel through the distributed computing node cluster, and requests are dynamically distributed in a load balancing manner, so that the problems of low processing efficiency and response lag under the condition of sharp increase of equipment and data volume are solved; based on distributed storage fragmentation storage and redundant backup, the single-point fault risk of concentrated data storage is avoided, and stable operation of the system is guaranteed; cross-regional data real-time synchronization is realized by means of a distributed message queue, and unified management and control of multi-regional data centers are supported; and meanwhile, asset full life cycle traceability is realized through a block chain, the fault early warning accuracy is improved through an LSTM model, energy consumption is optimized through multi-dimensional accounting and an algorithm, and the efficient, safe and energy-saving management requirements of a modern data center are comprehensively met.

Description

Technical Field

[0001] This invention relates to the field of dynamic management technology for data center user information, and in particular to a dynamic management system for data center user information based on a distributed architecture. Background Technology

[0002] With the continuous expansion of data center scale and the surge in the number of devices, cross-regional data center deployment has become a trend. Traditional centralized data center management systems face numerous challenges, including: 1. In a centralized architecture, computing and storage resources are centrally deployed. When the number of devices and the amount of data reach a certain scale, system processing efficiency drops significantly, and response speed lags. 2. Centralized data storage poses a single point of failure risk. Once a core storage device fails, the entire system will be paralyzed. 3. Data synchronization delays across regional data centers make unified management impossible. 4. Traditional systems have limited asset traceability capabilities, insufficient accuracy in fault early warning, and low precision in energy consumption management, making it difficult to meet the efficient, secure, and energy-saving management needs of modern data centers. In light of the above, this application proposes a dynamic data center user information management system based on a distributed architecture. Summary of the Invention

[0003] Based on the technical problems existing in the background technology, this invention proposes a dynamic management system for data center user information based on a distributed architecture.

[0004] The data center user information dynamic management system based on a distributed architecture proposed in this invention includes a distributed architecture layer, a core functional module layer, and a data interaction layer.

[0005] The distributed architecture layer provides distributed computing, storage and communication support for the system, including distributed computing nodes, distributed storage nodes, load balancing nodes and data synchronization nodes;

[0006] The core functional module layer is based on a distributed architecture layer to realize the full life cycle management function of the data center, which includes an asset management and life cycle tracking module, a multi-dimensional monitoring module, an intelligent alarm module, an energy consumption management and optimization module, an automated operation and maintenance and work order system module, and a security and access control module.

[0007] The data interaction layer is responsible for bidirectional data transmission between modules and between the system and external devices and tools.

[0008] Preferably, the distributed computing nodes adopt a cluster deployment mode, splitting the computing tasks of each functional module into multiple nodes for parallel processing; the distributed storage nodes adopt a sharded storage mechanism, splitting and storing equipment asset data, monitoring data, and operation logs according to preset rules, and ensuring data security through redundant backups; the load balancing nodes monitor the load status of each computing node and storage node in real time and dynamically allocate task requests; the data synchronization nodes achieve real-time data synchronization across nodes and regions through distributed message queues.

[0009] Preferably, the asset management and lifecycle tracking module collects data such as model, serial number, procurement information, and maintenance records of cabinet, server, and switch equipment through distributed crawlers and interface interfaces, establishes a distributed asset ledger, and records the entire process of equipment operation from procurement, warehousing, shelving, operation, maintenance to scrapping based on blockchain technology, generating an immutable lifecycle traceability chain that supports cross-node query of asset change history;

[0010] The multi-dimensional monitoring module is used to deploy monitoring agents on distributed computing nodes to collect real-time data on the overall temperature and humidity of the data center, CPU utilization, memory usage, hard disk read / write speed, and network traffic metrics for each rack and individual device. The distributed computing nodes preprocess the collected raw data, filtering out valid data for transmission to the core storage node, thus reducing data transmission bandwidth consumption. The specific logical steps are as follows:

[0011] S101: Based on the device type, match the appropriate agent from the system's "Device-Agent Mapping Library" and automatically distribute it to the target node via Ansible. The agent reads the parameters from the distributed configuration center to determine the collection rules and anomaly thresholds. At the same time, the agent sends a heartbeat packet to the edge node every 10 seconds. If no "acknowledgment frame" is received for 3 consecutive times, an "agent offline alarm" is triggered and the node ID is recorded.

[0012] S102: Collects temperature data and hardware specifications, and encapsulates the collected data into JSON;

[0013] S103: For 10 consecutive data points of the same indicator from a single device, first calculate the mean and standard deviation: If the data satisfies |x k If -μ|>3σ, then replace it with the mean of the first 3 valid data: Next, a moving average with a window size of N=5 is used to eliminate noise. The filtered data is as follows: If y k Data within the normal range and with a difference Δy from the previous valid data less than the minimum change threshold is considered redundant and discarded. k Exceeding the threshold or rate of change The data was determined to be valid.

[0014] S104: Compress the effective data using the Zstandard algorithm and calculate the compression ratio according to the formula: Where S org For the original size, S com This is the compressed size; the actual compression rate can reach 75%.

[0015] S105: High-priority alarm data is transmitted using WebSocket, while normal data is transmitted in batches using HTTP / 2. Distributed computing nodes obtain storage node load through load balancing nodes and calculate node weights according to the formula. And transmit the data to the node with the highest weight, where R i For storage utilization, T i For write response time;

[0016] S106: Distributed computing nodes calculate the MD5 value of compressed data and encapsulate it together with the data into a transmission packet; distributed storage nodes receive the packet and recalculate the MD5 value. If they match, the packet is decompressed; otherwise, it is retransmitted. Distributed storage nodes store data in fragments according to "device type-collection date-metric type" and simultaneously establish a distributed index containing "device ID, timestamp, and storage location" to support fast querying.

[0017] S107: After detecting that the agent is offline, automatically restart the agent remotely via SSH; if the restart fails, generate an "Agent Repair Work Order" and assign it to the operation and maintenance personnel. When the distributed computing node and the distributed storage node are disconnected, the valid data will be temporarily stored on the local SSD. After the connection is restored, the data will be re-uploaded in batches according to the timestamp in ascending order. After the re-upload is completed, the local cache will be deleted.

[0018] Preferably, the intelligent alarm module analyzes monitoring data based on a distributed machine learning model, trains a prediction model using historical fault data, and achieves early warning of equipment faults. It establishes a multi-dimensional alarm association rule base; when multiple related indicators such as rising temperature and decreasing fan speed simultaneously trigger alarms, the alarm level is automatically escalated. Through a distributed message push mechanism, the alarm information is pushed to the corresponding responsible person's terminal in real time, and the entire process of alarm confirmation, assignment, processing, and closure is recorded, forming a closed-loop management system. The specific logical steps are as follows:

[0019] S201: Collect equipment monitoring data and fault records from the past 3 years, use the 3σ principle to remove outliers, remove data where |x-μ|>3σ, and standardize valid data to the [0,1] interval using a formula.

[0020] S202: Divide the standardized data into a training set / test set in a 7:3 ratio using the SparkMLlib framework, construct an LSTM network, and optimize the weights W and biases b through backpropagation during the training process to make the fault risk probability output by the model more accurate. The core prediction formula is: where H t is the output of the LSTM hidden layer, P is the fault risk probability for the next hour, and use the test set to verify the model. If the accuracy then the model can be used;

[0021] S203: Receive the preprocessed data from the multi-dimensional monitoring module through the Kafka message queue, classify and store it in the Redis cache according to the device ID. For each device, extract the time series data in Redis for the most recent 5 minutes and input it into the trained LSTM model, calculate the fault risk probability P according to the formula. If P ≥ 80%, trigger a "Level 1 Predictive Alarm"; if 50% < P < 80%, trigger a "Level 3 Warning Prompt"; if P ≤ 50%, do not trigger an alarm;

[0022] S204: Compare the real-time metrics with the preset thresholds. If a single metric exceeds the threshold, trigger a "Level 2 Single-Dimension Alarm". When the system receives multiple single-dimension alarms simultaneously, automatically query the association rule library and match the association rules;

[0023] S205: Package the alarm information in JSON format, and select the push channel according to the device ownership and alarm level: Push Level 1 alarms to the operation and maintenance supervisor + engineer through WebSocket, and send text messages at the same time; Push Level 3 warnings only to the corresponding engineer through Enterprise WeChat. The push content includes the packaged alarm JSON data to ensure that the responsible person receives it in real time.

[0024] Preferably, the energy consumption management and optimization module collects the total energy consumption of the data center, the energy consumption of IT devices, and the energy consumption of the refrigeration system in real time through distributed metering nodes, combines the parallel computing capabilities of distributed computing nodes, calculates the PUE value in real time, and generates daily, weekly, and monthly energy consumption trend charts based on the historical energy consumption data stored in a distributed manner; supports customizing the energy consumption statistical dimensions, calculates the energy cost according to the computer room, cabinet, and enterprise dimensions, and gives energy-saving suggestions through a distributed optimization algorithm. The specific logical steps are as follows:

[0025] S301: Deploy intelligent metering nodes beside the main distribution room of the data center, IT device cabinets, and refrigeration systems to collect energy consumption data in real time, including the total energy consumption E total 、the energy consumption of IT devices E IT 、the energy consumption of the refrigeration system E coolThe data collection frequency is set to once per minute. The metering node transmits the raw data to the distributed computing node via the LoRaWAN protocol. The distributed computing node verifies the collected data, removes outliers, and fills in missing data using the average of the first 5 valid data points. The formula used is as follows: Where E fill To complete the energy consumption value, E t-1 -E t-5 The first 5 valid data points;

[0026] S302: Encapsulate the verified energy consumption data according to the "collection time-device type-location" label and store it in a distributed database for subsequent calculations;

[0027] S303: Distributed computing nodes calculate PUE in real time according to the formula: Where E total E represents the total energy consumption of the data center. IT For IT equipment energy consumption, the closer the PUE is to 1, the higher the energy efficiency, and the computing node will synchronize the PUE value to the distributed cache every 5 minutes;

[0028] S304: Calls historical energy consumption data from distributed storage, uses the ARIMA time series algorithm to analyze trends, and uses ECharts to generate daily / weekly / monthly energy consumption trend line charts to visually display peak energy consumption periods;

[0029] S305: Calculate the average PUE value μ over the past 7 days PUE With standard deviation σ PUE If the PUE on that day satisfies PUE>μ PUE +1.5σ PUE The system was determined to have an abnormally high PUE, triggering an "energy efficiency warning".

[0030] S306: Calculate energy consumption across various dimensions, including data center energy consumption and enterprise energy consumption. The calculation formula is as follows: Data Center Energy Consumption: Where M represents the number of server racks in the data center. Energy consumption per rack;

[0031] Enterprise energy consumption: Where N is the number of servers rented by the company. Energy consumption per server;

[0032] S307: Enter the unit price of electricity for different time periods and calculate the cost of each dimension according to the formula C=E×P, where C is the energy cost, E is the energy consumption of the statistical dimension, and P is the unit price of electricity for the corresponding time period;

[0033] S308: Employing a genetic algorithm to optimize IT equipment operation strategies and cooling system parameters with the objectives of "minimizing PUE" and "minimizing cost," it calculates the energy-saving potential of each optimization scheme using the formula: ΔE = E current -E optimize E current For current energy consumption, E optimize To optimize the predicted energy consumption, ΔE is the potential energy-saving value. Based on the optimization results, suggestions are output and simultaneously pushed to the operation and maintenance terminal. The changes in energy consumption after the suggestions are implemented are recorded to verify the energy-saving effect in a closed loop.

[0034] Preferably, the automated operation and maintenance and work order system module integrates a distributed automation tool cluster. Through the distributed deployment of Ansible and Puppet tools, it realizes the automated operation of operating system installation, application deployment, and configuration distribution for newly racked servers. Based on preset fault handling rules, it automatically repairs simple and predictable faults through edge computing nodes, and automatically generates operation and maintenance work orders for complex faults. The work order system adopts a distributed flow mechanism, supporting the creation, approval, assignment, and execution tracking of fault reporting, equipment removal and removal applications, and asset changes. The work order status is synchronized to all relevant nodes in real time.

[0035] Preferably, the security and access control module adopts a role-based distributed access control model, which subdivides multiple roles such as administrators, maintenance personnel, visitors, and auditors, and configures different cross-node access permissions for each role; through a distributed log collection system, it records all user login, configuration modification, and data query operations, and the log data is stored in fragments and encrypted; it supports the integration of a distributed video surveillance system to realize cross-node viewing of real-time data center footage, thereby enhancing physical security control.

[0036] Preferably, the data interaction layer adopts a communication method combining RESTful API, WebSocket, and distributed message queues to support bidirectional data interaction between the system and data center equipment, third-party operation and maintenance tools, and user terminals; it uses an encrypted transmission protocol to encrypt the data transmission process to ensure data transmission security, supports adaptive protocol conversion, and is compatible with the communication protocols of different devices and tools.

[0037] Compared with existing technologies, the beneficial effects of this invention are:

[0038] 1. The system adopts a distributed computing node cluster deployment mode, which splits computing tasks such as PUE calculation, LSTM fault prediction, and multi-dimensional monitoring data preprocessing into multiple nodes for parallel processing, avoiding performance bottlenecks caused by centralized computing resources and significantly improving data processing efficiency. At the same time, combined with load balancing nodes to dynamically allocate task requests, it ensures that the load of each node is balanced, so that the system can still maintain millisecond-level response speed in scenarios with more than 10,000 devices and daily data volume of TB, effectively improving the problem of slow response speed caused by large data volume in centralized architecture.

[0039] 2. The distributed storage nodes adopt a sharding storage mechanism, which splits and stores equipment asset data, monitoring data, operation logs, etc. according to preset rules, and performs redundant backups on 3 or more different nodes. There is no single core storage device. Even if a storage node fails, the system can quickly obtain data through other backup nodes, avoiding the risk of "one loss causing the whole system to fail" in centralized storage, and ensuring data availability and system stability.

[0040] 3. By combining data synchronization nodes with RabbitMQ distributed message queues, asset change records, alarm information, and energy consumption data of cross-regional data centers can be synchronized in real time with a synchronization delay of less than 1 second. At the same time, relying on a unified distributed management interface, operation and maintenance personnel can view the equipment status, work order progress, and energy consumption trends of all regional data centers on a single terminal, breaking the limitations of traditional cross-regional management of "data silos" and "dispersed control", and realizing unified scheduling and collaborative management of data centers.

[0041] 4. Regarding asset traceability, blockchain technology records the entire process of equipment operation from procurement, warehousing, shelving to disposal, generating an immutable lifecycle traceability chain. It supports cross-node querying of asset change history, solving the problems of "unclear asset flow and lack of audit basis" in traditional systems. Regarding fault early warning, LSTM models are trained on historical fault data, achieving a fault prediction accuracy of ≥90%. Combined with a multi-dimensional alarm association rule library, it can provide early warning of equipment faults, and automatically upgrade the alarm level when multiple indicators are linked, significantly reducing the fault outbreak rate and troubleshooting time. Regarding energy consumption management, it calculates PUE values ​​in real time, calculates energy costs by data center / rack / enterprise multi-dimensional dimensions, and provides energy-saving optimization suggestions through genetic algorithms, effectively reducing data center energy consumption and fully meeting the management needs of modern data centers for "efficient operation and maintenance, safe and controllable, and green energy saving".

[0042] This invention solves the problems of low processing efficiency and delayed response under the surge in equipment and data volume by using a distributed computing node cluster to process tasks in parallel and dynamically allocate requests with load balancing. Relying on distributed storage sharding and redundant backup, it avoids the single point of failure risk of centralized data storage and ensures stable system operation. It uses a distributed message queue to achieve real-time data synchronization across regions and supports unified management and control of multiple regional data centers. At the same time, it uses blockchain to achieve full lifecycle traceability of assets, LSTM model to improve the accuracy of fault early warning, and multi-dimensional accounting and algorithm optimization to optimize energy consumption, fully meeting the management needs of modern data centers for high efficiency, security and energy saving. Attached Figure Description

[0043] Figure 1 This is a block diagram of the data center user information dynamic management system based on a distributed architecture proposed in this invention. Detailed Implementation

[0044] The present invention will be further explained below with reference to specific embodiments.

[0045] Example

[0046] Reference Figure 1 This embodiment proposes a data center user information dynamic management system based on a distributed architecture, including a distributed architecture layer, a core functional module layer, and a data interaction layer;

[0047] The distributed architecture layer provides distributed computing, storage and communication support for the system, including distributed computing nodes, distributed storage nodes, load balancing nodes and data synchronization nodes;

[0048] The distributed computing nodes adopt a cluster deployment mode, splitting the computing tasks of each functional module into multiple nodes for parallel processing; the distributed storage nodes adopt a sharded storage mechanism, splitting and storing equipment asset data, monitoring data, and operation logs according to preset rules, and ensuring data security through redundant backups; the load balancing nodes monitor the load status of each computing node and storage node in real time and dynamically allocate task requests; the data synchronization nodes achieve real-time data synchronization across nodes and regions through distributed message queues.

[0049] The core functional module layer is based on a distributed architecture layer to realize the full lifecycle management function of the data center, which includes an asset management and lifecycle tracking module, a multi-dimensional monitoring module, an intelligent alarm module, an energy consumption management and optimization module, an automated operation and maintenance and work order system module, and a security and access control module.

[0050] The asset management and lifecycle tracking module collects data such as model, serial number, procurement information, and maintenance records of cabinet, server, and switch equipment through distributed crawlers and interface connections. It establishes a distributed asset ledger and records the entire process of equipment operation from procurement, warehousing, shelving, operation, maintenance to scrapping based on blockchain technology, generating an immutable lifecycle traceability chain that supports cross-node querying of asset change history.

[0051] The multi-dimensional monitoring module is used to deploy monitoring agents on distributed computing nodes to collect real-time data on the overall temperature and humidity of the data center, CPU utilization, memory usage, hard drive read / write speed, and network traffic metrics for each rack and individual device. The distributed computing nodes preprocess the collected raw data, filtering out valid data for transmission to the core storage nodes, thus reducing data transmission bandwidth consumption. The specific logical steps are as follows:

[0052] S101: Based on the device type, match the appropriate agent from the system's "Device-Agent Mapping Library" and automatically distribute it to the target node via Ansible. The agent reads the parameters from the distributed configuration center to determine the collection rules and anomaly thresholds. At the same time, the agent sends a heartbeat packet to the edge node every 10 seconds. If no "acknowledgment frame" is received for 3 consecutive times, an "agent offline alarm" is triggered and the node ID is recorded.

[0053] S102: Collects temperature data and hardware specifications, and encapsulates the collected data into JSON;

[0054] S103: For 10 consecutive data points of the same indicator from a single device, first calculate the mean and standard deviation: If the data satisfies |x k If -μ|>3σ, then replace it with the mean of the first 3 valid data: Next, a moving average with a window size of N=5 is used to eliminate noise. The filtered data is as follows: If y k Data within the normal range and with a difference Δy from the previous valid data less than the minimum change threshold is considered redundant and discarded. k Exceeding the threshold or rate of change The data was determined to be valid.

[0055] S104: Compress the effective data using the Zstandard algorithm and calculate the compression ratio according to the formula: Where S org For the original size, S com This is the compressed size; the actual compression rate can reach 75%.

[0056] S105: High-priority alarm data is transmitted using WebSocket, while normal data is transmitted in batches using HTTP / 2. Distributed computing nodes obtain storage node load through load balancing nodes and calculate node weights according to the formula. And transmit the data to the node with the highest weight, where R i For storage utilization, T i For write response time;

[0057] S106: Distributed computing nodes calculate the MD5 value of compressed data and encapsulate it together with the data into a transmission packet; distributed storage nodes receive the packet and recalculate the MD5 value. If they match, the packet is decompressed; otherwise, it is retransmitted. Distributed storage nodes store data in fragments according to "device type-collection date-metric type" and simultaneously establish a distributed index containing "device ID, timestamp, and storage location" to support fast querying.

[0058] S107: After detecting that the agent is offline, automatically restart the agent remotely via SSH; if the restart fails, generate an "agent repair work order" and assign it to the operation and maintenance personnel. When the distributed computing node and the distributed storage node are disconnected, the valid data will be temporarily stored on the local SSD. After the connection is restored, the data will be re-uploaded in batches according to the timestamp in ascending order. After the re-upload is completed, the local cache will be deleted.

[0059] The intelligent alarm module analyzes monitoring data based on a distributed machine learning model, trains a prediction model using historical fault data, and provides early warnings of equipment failures. It establishes a multi-dimensional alarm association rule base; when multiple related indicators, such as rising temperature and decreasing fan speed, simultaneously trigger alarms, the alarm level is automatically escalated. Through a distributed message push mechanism, alarm information is pushed to the corresponding responsible personnel's terminals in real time, and the entire process of alarm confirmation, assignment, processing, and closure is recorded, forming a closed-loop management system. The specific logical steps are as follows:

[0060] S201: Collect equipment monitoring data and fault records from the past 3 years, use the 3σ principle to remove outliers, remove data where |x-μ|>3σ, and standardize valid data to the [0,1] interval using a formula.

[0061] S202: Using the SparkMLlib framework, standardized data is divided into training and test sets in a 7:3 ratio. An LSTM network is constructed, and during training, backpropagation is used to optimize the weights W and bias b, making the model's output fault risk probability more accurate. The core prediction formula is: Where H t The output of the LSTM hidden layer is given, where P is the probability of failure risk in the next hour. The model is validated using a test set, and the accuracy is [not specified]. The model is then usable;

[0062] S203: Receive the preprocessed data from the multi-dimensional monitoring module through the Kafka message queue, classify and store it in the Redis cache by device ID. For each device, extract the time-series data in Redis for the most recent 5 minutes, and input it into the trained LSTM model. Calculate the fault risk probability P according to the formula. If P ≥ 80%, trigger a "Level 1 predictive alarm"; if 50% < P < 80%, trigger a "Level 3 warning prompt"; if P ≤ 50%, do not trigger an alarm.

[0063] S204: Compare the real-time metrics with the preset thresholds. If a single metric exceeds the threshold, trigger a "Level 2 single-dimensional alarm". When the system receives multiple single-dimensional alarms simultaneously, automatically query the association rule library and match the association rules.

[0064] S205: Package the alarm information in JSON format, and select the push channels according to the device ownership and alarm level: Push Level 1 alarms to the operation and maintenance supervisor + engineer through WebSocket, and send text messages at the same time; Push Level 3 warnings only to the corresponding engineer through enterprise WeChat. The push content includes the packaged alarm JSON data to ensure that the responsible person receives it in real time.

[0065] The energy consumption management and optimization module collects the total energy consumption of the data center, the energy consumption of IT equipment, and the energy consumption of the refrigeration system in real time through distributed metering nodes. Combining the parallel computing capabilities of distributed computing nodes, it calculates the PUE value in real time, and generates daily, weekly, and monthly energy consumption trend charts based on historical energy consumption data stored in a distributed manner using time series analysis algorithms; supports customizing energy consumption statistical dimensions, calculates the energy cost by computer room, cabinet, and enterprise dimensions, and gives energy-saving suggestions through distributed optimization algorithms. The specific logical steps are as follows:

[0066] S301: Deploy intelligent metering nodes beside the main distribution room of the data center, IT equipment cabinets, and refrigeration systems to collect energy consumption data in real time, including the total energy consumption E total , the energy consumption E IT of IT equipment, and the energy consumption E cool of the refrigeration system. Set the collection frequency to 1 minute / time. The metering nodes transmit the original data to the distributed computing node through the LoRaWAN protocol. The distributed computing node verifies the collected data, eliminates outliers, and fills in the missing data with the average value of the previous 5 valid data. The formula used is: Where E fill is the energy consumption value after filling, and E t-1 - E t-5 are the previous 5 valid data.

[0067] S302: Package the verified energy consumption data with tags of "collection time - device type - location" and store it in the distributed database for subsequent calculation and invocation.

[0068] S303: Distributed computing nodes calculate PUE in real time according to the formula: Where E total E represents the total energy consumption of the data center. IT For IT equipment energy consumption, the closer the PUE is to 1, the higher the energy efficiency, and the computing node will synchronize the PUE value to the distributed cache every 5 minutes;

[0069] S304: Calls historical energy consumption data from distributed storage, uses the ARIMA time series algorithm to analyze trends, and uses ECharts to generate daily / weekly / monthly energy consumption trend line charts to visually display peak energy consumption periods;

[0070] S305: Calculate the average PUE value μ over the past 7 days PUE With standard deviation σ PUE If the PUE on that day satisfies PUE>μ PUE +1.5σ PUE The system was determined to have an abnormally high PUE, triggering an "energy efficiency warning".

[0071] S306: Calculate energy consumption across various dimensions, including data center energy consumption and enterprise energy consumption. The calculation formula is as follows: Data Center Energy Consumption: Where M represents the number of server racks in the data center. Energy consumption per rack;

[0072] Enterprise energy consumption: Where N is the number of servers rented by the company. Energy consumption per server;

[0073] S307: Enter the unit price of electricity for different time periods and calculate the cost of each dimension according to the formula C=E×P, where C is the energy cost, E is the energy consumption of the statistical dimension, and P is the unit price of electricity for the corresponding time period;

[0074] S308: Employing a genetic algorithm to optimize IT equipment operation strategies and cooling system parameters with the objectives of "minimizing PUE" and "minimizing cost," it calculates the energy-saving potential of each optimization scheme using the formula: ΔE = E current -E optimize E current For current energy consumption, E optimize To optimize the predicted energy consumption, ΔE is the potential energy-saving value. Based on the optimization results, suggestions are output and simultaneously pushed to the operation and maintenance terminal. The changes in energy consumption after the suggestions are implemented are recorded to verify the energy-saving effect in a closed loop.

[0075] The automated operation and maintenance and work order system module integrates a distributed automation tool cluster. Through the distributed deployment of Ansible and Puppet tools, it automates the installation of operating systems, deployment of applications, and distribution of configurations for newly racked servers. Based on preset fault handling rules, it automatically repairs simple and predictable faults through edge computing nodes, and automatically generates operation and maintenance work orders for complex faults. The work order system adopts a distributed workflow mechanism, supporting the creation, approval, assignment, and execution tracking of fault reports, equipment removal and removal applications, and asset changes. The work order status is synchronized to all relevant nodes in real time.

[0076] The security and access control module adopts a role-based distributed access control model, which subdivides multiple roles such as administrators, operations and maintenance personnel, visitors, and auditors, and configures different cross-node access permissions for each role; through a distributed log collection system, it records all user logins, configuration modifications, and data query operations, and the log data is stored in fragments and encrypted; it supports integration with a distributed video surveillance system to enable cross-node viewing of real-time data center footage, enhancing physical security control;

[0077] The data interaction layer is responsible for bidirectional data transmission between modules and between the system and external devices and tools;

[0078] The data interaction layer adopts a communication method that combines RESTful API, WebSocket and distributed message queue, supporting bidirectional data interaction between the system and data center equipment, third-party operation and maintenance tools and user terminals; it uses an encrypted transmission protocol to encrypt the data transmission process to ensure data transmission security, supports adaptive protocol conversion, and is compatible with the communication protocols of different devices and tools;

[0079] This embodiment solves the problems of low processing efficiency and delayed response under the surge in equipment and data volume by using a distributed computing node cluster to process tasks in parallel and dynamically allocate requests with load balancing. Relying on distributed storage sharding and redundant backup, it avoids the single point of failure risk of centralized data storage and ensures stable system operation. It uses a distributed message queue to achieve real-time data synchronization across regions and supports unified management and control of multiple regional data centers. At the same time, it uses blockchain to achieve full lifecycle traceability of assets, LSTM model to improve the accuracy of fault early warning, and multi-dimensional accounting and algorithm optimization to optimize energy consumption, fully meeting the management needs of modern data centers for high efficiency, security and energy saving.

[0080] In this embodiment, firstly, the distributed architecture layer serves as the foundation for system operation, constructing a stable and efficient support system through four types of nodes. The distributed computing nodes are deployed using a Kubernetes cluster, splitting PUE calculation, fault prediction, and data cleaning calculation tasks into multiple nodes for parallel processing, significantly improving computing efficiency. The distributed storage nodes are based on GlusterFS and employ a sharded storage mechanism, splitting asset data, monitoring data, etc., for storage according to "device type-data type-time dimension," and avoiding single points of failure through 3-node redundancy backup. The load balancing nodes use an Nginx weighted round-robin algorithm to monitor the load status of each node in real time and dynamically allocate task requests to ensure load balancing. The data synchronization nodes rely on the RabbitMQ distributed message queue to achieve real-time data synchronization across nodes and regions, ensuring data consistency across multiple regions.

[0081] Building upon this foundation, the core functional modules revolve around six dimensions—asset management, monitoring, alarms, energy consumption, operation and maintenance, and security—to provide full lifecycle management. The asset management and lifecycle tracing module collects basic device information via distributed crawlers and SNMP protocols, establishing a distributed asset ledger. Simultaneously, it records the entire process of device operations from procurement, warehousing, shelving, to disposal based on a private Ethereum blockchain, generating an immutable blockchain hash value that supports cross-node tracing. The multi-dimensional monitoring module matches and adapts agents according to device type, automatically deploying them via Ansible. The agents collect data such as temperature, humidity, and CPU usage at preset frequencies, and then analyze the 3σ outlier data. Cleaning, moving average filtering, and Zstandard compression are performed, and the data is distributed to low-load storage nodes by the load balancer. If the agent goes offline, an alarm is automatically triggered and a remote restart is attempted. The intelligent alarm module first trains an LSTM model using three years of historical data through SparkMLlib. When running online, it receives monitoring data through Kafka, inputs the model, outputs the fault risk probability P, and matches it against a rule base. When multiple indicators are linked, the alarm level is automatically increased, and the alarm is pushed to the responsible person via WebSocket and SMS. The entire process status is recorded in the MySQL cluster. The energy consumption management and optimization module obtains the total energy consumption E through smart meters. total IT energy consumption E IT Data is collected, missing data is filled in with the average of the first 5 times, PUE is calculated in parallel, and energy consumption trend charts are generated based on the ARIMA algorithm. Costs are calculated by data center, and then the operation strategy is optimized using a genetic algorithm to output energy-saving suggestions.

[0082] The automated operation and maintenance and work order system module integrates Ansible to achieve automated server deployment, automatic repair of simple faults, and generation and distributed workflow of work orders for complex faults, with real-time status synchronization. The security and access control module uses the Shiro framework to subdivide role permissions, records operations through the ELK logging system and stores them in encrypted shards, and integrates video surveillance to achieve physical security control. Finally, the data interaction layer acts as a communication bridge, using a combination of WebSocket, HTTP / 2, and MQTT protocols, combined with TLS 1.3 encryption and MD5 verification to ensure data transmission security. At the same time, it connects to third-party tools and user terminals through RESTful APIs, breaking down information silos and ultimately achieving efficient, secure, and energy-saving dynamic management of the data center.

[0083] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A data center user information dynamic management system based on a distributed architecture, characterized in that, It includes a distributed architecture layer, a core functional module layer, and a data interaction layer; The distributed architecture layer provides distributed computing, storage and communication support for the system, including distributed computing nodes, distributed storage nodes, load balancing nodes and data synchronization nodes; The core functional module layer is based on a distributed architecture layer to realize the full life cycle management function of the data center, which includes an asset management and life cycle tracking module, a multi-dimensional monitoring module, an intelligent alarm module, an energy consumption management and optimization module, an automated operation and maintenance and work order system module, and a security and access control module. The data interaction layer is responsible for bidirectional data transmission between modules and between the system and external devices and tools.

2. The data center user information dynamic management system based on a distributed architecture as described in claim 1, characterized in that, The distributed computing nodes adopt a cluster deployment mode, which splits the computing tasks of each functional module into multiple nodes for parallel processing. The distributed storage nodes adopt a sharding storage mechanism to split and store equipment asset data, monitoring data, and operation logs according to preset rules, and ensure data security through redundant backups; The load balancing node monitors the load status of each compute node and storage node in real time and dynamically allocates task requests. Data synchronization nodes achieve real-time data synchronization across nodes and regions through distributed message queues.

3. The data center user information dynamic management system based on a distributed architecture as described in claim 1, characterized in that, The asset management and lifecycle tracking module collects data such as model, serial number, procurement information, and maintenance records of cabinet, server, and switch equipment through distributed crawlers and interface interfaces, establishes a distributed asset ledger, and records the entire process of equipment operation from procurement, warehousing, shelving, operation, maintenance to scrapping based on blockchain technology, generating an immutable lifecycle traceability chain that supports cross-node query of asset change history. The multi-dimensional monitoring module is used to deploy monitoring agents on distributed computing nodes to collect real-time data on the overall temperature and humidity of the data center, CPU utilization, memory usage, hard disk read / write speed, and network traffic metrics for each rack and individual device. The distributed computing nodes preprocess the collected raw data, filtering out valid data for transmission to the core storage node, thus reducing data transmission bandwidth consumption. The specific logical steps are as follows: S101: Based on the device type, match the appropriate agent from the system's "Device-Agent Mapping Library" and automatically distribute it to the target node via Ansible. The agent reads the parameters from the distributed configuration center to determine the collection rules and anomaly thresholds. At the same time, the agent sends a heartbeat packet to the edge node every 10 seconds. If no "acknowledgment frame" is received for 3 consecutive times, an "agent offline alarm" is triggered and the node ID is recorded. S102: Collects temperature data and hardware specifications, and encapsulates the collected data into JSON; S103: For 10 consecutive data points of the same indicator from a single device, first calculate the mean and standard deviation: If the data satisfies |x k If -μ|>3σ, then replace it with the mean of the first 3 valid data: Next, a moving average with a window size of N=5 is used to eliminate noise. The filtered data is as follows: If y k Data within the normal range and with a difference Δy from the previous valid data less than the minimum change threshold is considered redundant and discarded. k Exceeding the threshold or rate of change The data was determined to be valid. S104: Compress the effective data using the Zstandard algorithm and calculate the compression ratio according to the formula: Where S org For the original size, S com This is the compressed size; the actual compression rate can reach 75%. S105: High-priority alarm data is transmitted using WebSocket, while normal data is transmitted in batches using HTTP / 2. Distributed computing nodes obtain storage node load through load balancing nodes and calculate node weights according to the formula. And transmit the data to the node with the highest weight, where R i For storage utilization, T i For write response time; S106: Distributed computing nodes calculate the MD5 value of compressed data and encapsulate it together with the data into a transmission packet; distributed storage nodes receive the data and recalculate the MD5. If they match, the data is decompressed; otherwise, it is retransmitted. Distributed storage nodes store data in fragments according to "device type-collection date-metric type" and simultaneously establish a distributed index containing "device ID, timestamp, and storage location" to support fast querying. S107: After detecting that the proxy is offline, automatically restart the proxy remotely via SSH; if the restart fails, generate a "proxy repair work order" and assign it to the operation and maintenance personnel. When the distributed computing node is disconnected from the distributed storage node, temporarily store the valid data in the local SSD, and after the connection is restored, batch upload it in ascending order of the timestamp. After the upload is completed, delete the local cache.

4. The data center user information dynamic management system based on a distributed architecture according to claim 1, characterized in that, The intelligent alarm module analyzes the monitoring data based on a distributed machine learning model, trains a prediction model with historical fault data, and realizes early warning of equipment faults. Establish a multi-dimensional alarm association rule library. When multiple related indicators such as rising temperature and decreasing fan speed trigger alarms simultaneously, automatically raise the alarm level. Through the distributed message push mechanism, push the alarm information to the corresponding responsible person's terminal in real time, and record the entire process of alarm confirmation, assignment, handling, and closing to form a closed-loop management. The specific logical steps are as follows: S201: Collect equipment monitoring data and fault records from the past 3 years, use the 3σ principle to remove outliers, remove data where |x-μ|>3σ, and standardize valid data to the [0,1] interval using a formula. S202: Using the SparkMLlib framework, standardized data is divided into training and test sets in a 7:3 ratio. An LSTM network is constructed, and during training, backpropagation is used to optimize the weights W and bias b, making the model's output fault risk probability more accurate. The core prediction formula is: Where H t The output of the LSTM hidden layer is given, where P is the probability of failure risk in the next hour. The model is validated using a test set, and the accuracy is [not specified]. The model is then usable; S203: Receive the preprocessed data from the multi-dimensional monitoring module through the Kafka message queue, classify and store it in the Redis cache according to the device ID. For each device, extract the time series data in the Redis in the last 5 minutes, and input it into the trained LSTM model. Calculate the fault risk probability P according to the formula. If P≥80%, trigger a "level 1 predictive alarm"; if 50%<P<80%, trigger a "level 3 warning prompt"; if P≤50%, do not trigger an alarm. S204: Compare the real-time indicators with the preset thresholds. If a single indicator exceeds the threshold, trigger a "level 2 single-dimensional alarm". When the system receives multiple single-dimensional alarms simultaneously, automatically query the association rule library and match the association rules. S205: Package the alarm information in JSON format, and select the push channel according to the device ownership and alarm level: Push the level 1 alarm to the operation and maintenance supervisor + engineer via WebSocket and send a text message at the same time; Push the level 3 warning only to the corresponding engineer via enterprise WeChat. The push content includes the packaged alarm JSON data to ensure that the responsible person receives it in real time.

5. The data center user information dynamic management system based on a distributed architecture as described in claim 1, characterized in that, The energy consumption management and optimization module collects the total energy consumption of the data center, the energy consumption of IT equipment, and the energy consumption of the refrigeration system in real time through distributed metering nodes. Combining with the parallel computing ability of the distributed computing node, calculate the PUE value in real time, and generate daily, weekly, and monthly energy consumption trend charts based on the historical energy consumption data stored in a distributed manner using the time series analysis algorithm; Support customizing the energy consumption statistical dimension, calculating the energy cost according to the computer room, cabinet, and enterprise dimensions, and giving energy-saving suggestions through the distributed optimization algorithm. The specific logical steps are as follows: S301: Deploy smart metering nodes near the data center's main power distribution room, IT equipment cabinets, and cooling systems to collect energy consumption data in real time, including total energy consumption E. total IT equipment energy consumption E IT Energy consumption of refrigeration system E cool The data collection frequency is set to once per minute. The metering node transmits the raw data to the distributed computing node via the LoRaWAN protocol. The distributed computing node verifies the collected data, removes outliers, and fills in missing data using the average of the first 5 valid data points. The formula used is as follows: Where E fill To complete the energy consumption value, E t-1 -E t-5 The first 5 valid data points; S302: Package the verified energy consumption data with the label of "collection time - device type - location" and store it in the distributed database for subsequent calculation and call. S303: Distributed computing nodes calculate PUE in real time according to the formula: Where E total E represents the total energy consumption of the data center. IT For IT equipment energy consumption, the closer the PUE is to 1, the higher the energy efficiency, and the computing node will synchronize the PUE value to the distributed cache every 5 minutes; S304: Call the historical energy consumption data stored in a distributed manner, analyze the trend using the ARIMA time series algorithm, and generate a line chart of daily / weekly / monthly energy consumption trends using ECharts to visually display the peak energy consumption period. S305: Calculate the average PUE value μ over the past 7 days PUE With standard deviation σ PUE If the PUE on that day satisfies PUE>μ PUE +1.5σ PUE The system was determined to have an abnormally high PUE, triggering an "energy efficiency warning". S306: Calculate energy consumption across various dimensions, including data center energy consumption and enterprise energy consumption. The calculation formula is as follows: Data Center Energy Consumption: Where M represents the number of server racks in the data center. Energy consumption per rack; Enterprise energy consumption: Where N is the number of servers rented by the company. Energy consumption per server; S307: Enter the electricity price per unit at different times, and calculate the cost C = E×P for each dimension according to the formula, where C is the energy cost, E is the energy consumption of the statistical dimension, and P is the electricity price per unit at the corresponding time. S308: A genetic algorithm is used to optimize IT equipment operation strategies and cooling system parameters with the goals of "minimizing PUE" and "minimizing cost," and the energy-saving potential of each optimization scheme is calculated using the formula: ΔE = E current -E optimize E current For current energy consumption, E optimize To optimize the predicted energy consumption, ΔE is the potential energy-saving value. Based on the optimization results, suggestions are output and simultaneously pushed to the operation and maintenance terminal. The changes in energy consumption after the suggestions are implemented are recorded to verify the energy-saving effect in a closed loop.

6. The data center user information dynamic management system based on a distributed architecture according to claim 1, characterized in that, The automated operation and maintenance and work order system module integrates a distributed automation tool cluster. Through the distributed deployment of Ansible and Puppet tools, it realizes the automated operation of operating system installation, application deployment and configuration distribution for newly racked servers. Based on preset fault handling rules, simple and predictable faults are automatically repaired through edge computing nodes, and maintenance work orders are automatically generated for complex faults. The work order system adopts a distributed workflow mechanism, supporting the creation, approval, assignment, and execution tracking of fault reporting, equipment removal and removal applications, and asset changes. The work order status is synchronized to all relevant nodes in real time.

7. The data center user information dynamic management system based on a distributed architecture according to claim 1, characterized in that, The security and access control module adopts a role-based distributed access control model, which subdivides multiple roles such as administrators, maintenance personnel, visitors, and auditors, and configures different cross-node access permissions for each role; through a distributed log collection system, it records all user logins, configuration modifications, and data query operations, and the log data is stored in fragments and encrypted; it supports integration with a distributed video surveillance system to enable cross-node viewing of real-time data center footage, thereby enhancing physical security control.

8. The data center user information dynamic management system based on a distributed architecture according to claim 1, characterized in that, The data interaction layer adopts a communication method that combines RESTful API, WebSocket and distributed message queue, supporting bidirectional data interaction between the system and data center equipment, third-party operation and maintenance tools and user terminals; it uses an encrypted transmission protocol to encrypt the data transmission process to ensure data transmission security, supports adaptive protocol conversion, and is compatible with the communication protocols of different devices and tools.

Images