Data center infrastructure operation and maintenance management method, device and digital platform

Abstract

The present invention relates to the field of data center operation and maintenance management technology, and in particular to a data center infrastructure operation and maintenance management method, device and digital platform, wherein the method comprises: calculating the total resource utilization of the data center at the current moment according to the resource utilization of each node in the data center at the current moment, if the total resource utilization is less than or equal to the first preset utilization, and the actual number of resource utilizations in each node of the data center greater than the second preset utilization is less than or equal to the preset number, statically managing the infrastructure operation and maintenance of each node in the data center based on the pre-allocated weights of each node in the data center; otherwise, calculating the target allocation weight of each node in the data center according to the resource utilization of each node in the data center under the current task, and dynamically managing the infrastructure operation and maintenance of each node in the data center based on the target allocation weight of each node in the data center. Thus, the problem of how to improve the overall efficiency and response speed of the data center infrastructure operation and maintenance is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of data center operation and maintenance management, and in particular to a data center infrastructure operation and maintenance management method, device and digital platform. Background Art

[0002] As the scale of data centers continues to expand, the operation and maintenance management of power, HVAC and other infrastructure in single data centers has become increasingly complex, and cross-regional operation and maintenance management has also become more challenging.

[0003] In related technologies, the operation and maintenance management platform is limited by the original support system design and cannot meet the current operation and maintenance management model. It relies on a large amount of manpower and has problems such as untimely monitoring, slow response, and low efficiency. Summary of the invention

[0004] The present invention provides a data center infrastructure operation and maintenance management method, device and digital platform to solve the problem of how to improve the overall efficiency and response speed of data center infrastructure operation and maintenance.

[0005] The first aspect of the present invention provides a data center infrastructure operation and maintenance management method, comprising the following steps: obtaining the operating status of the infrastructure of each node in the data center at the current moment; calculating the resource utilization of each node in the data center at the current moment according to the operating status of the infrastructure of each node in the data center at the current moment, and calculating the total resource utilization of the data center at the current moment according to the resource utilization of each node in the data center at the current moment; if the total resource utilization is less than or equal to the first preset utilization, and the actual number of nodes in the data center whose resource utilization is greater than the second preset utilization is less than or equal to the preset number, then obtaining the pre-allocated weight of each node in the data center, and statically managing the infrastructure operation and maintenance of each node in the data center based on the pre-allocated weight of each node in the data center; if the total resource utilization is greater than the first preset utilization, or the actual number of nodes in the data center whose resource utilization is greater than the second preset utilization is greater than the preset number, then obtaining the resource utilization of each node in the data center when executing the current task, calculating the target allocation weight of each node in the data center according to the resource utilization of each node in the data center when executing the current task, and dynamically managing the infrastructure operation and maintenance of each node in the data center based on the target allocation weight of each node in the data center. .

[0006] Optionally, the target allocation weight of each node in the data center is calculated based on the resource utilization of each node in the data center when executing the current task, including: obtaining the total number of nodes in the data center; inputting the total number of nodes and the resource utilization of each node in the data center into a load balancing algorithm, and the load balancing algorithm outputs the total allocation weight of the data center; calculating the target allocation weight of each node in the data center based on the resource utilization of each node in the data center and the total allocation weight.

[0007] Optionally, the calculation formula of the load balancing algorithm is:

[0008]

[0009] Among them, f(x) is the total allocation weight, r i (x) is the resource utilization of the i-th node when executing the current task x, and n is the total number of nodes;

[0010] The target allocation weight is calculated as:

[0011] R = f(x)r i (x) -1 ,

[0012] Among them, R is the target allocation weight.

[0013] Optionally, dynamically managing the operation and maintenance of infrastructure of each node in the data center based on the target allocation weight of each node in the data center includes: obtaining a task queue and a priority of each target task in the task queue;

[0014] Generate a priority queue based on the task queue and the priority of each target task;

[0015] Nodes are allocated to target tasks in the priority queue based on a greedy algorithm and target allocation weights, wherein nodes with large target allocation weights are allocated to target tasks with high priorities.

[0016] Optionally, the greedy algorithm includes: traversing the priority queue in sequence until the priority queue is empty, and performing the following operations each time traversal: selecting the target task with the highest priority from the priority queue; traversing all nodes to find the node that can accommodate the target task with the highest priority, and assigning the target task with the highest priority to the node with the largest target allocation weight; if the node that can accommodate the target task with the highest priority is not found, determining the minimum number of required nodes based on the resources required for the target task with the highest priority and the resource utilization of all nodes when executing the current task, and assigning the target task with the highest priority according to the target allocation weight of the selected node; updating the resource utilization of the selected node when executing the current task, and updating the target allocation weights of all nodes based on the updated resource utilization.

[0017] Optionally, the target task with the highest priority is allocated according to the target allocation weight of the selected node, including: dividing the target task with the highest priority into multiple subtasks; identifying whether the number of allocated subtasks for each selected node is an integer; if the number of allocated subtasks is not an integer, performing a rounding operation, and optimizing the number of allocated subtasks for each selected node after performing the rounding operation.

[0018] Optionally, obtaining the pre-allocated weights of each node in the data center includes: obtaining historical data in the data and pre-processing the historical data; performing statistical analysis on the pre-processed data to obtain data distribution, visualizing the pre-processed data to obtain data trends, extracting the pre-processed data according to the data distribution and data trends to generate a data set, and dividing the data set into a training set and a test set; using at least one of a linear regression model, a decision tree, a random forest, a support vector machine, a neural network, and a time series analysis model as a prediction model, iteratively training the prediction model using the training set, and testing the prediction performance of the prediction model using the test set; optimizing the parameters of the prediction model according to the test results, stopping the training until the number of iterative training reaches the target number or the prediction performance of the prediction model meets the target performance, and using the trained prediction model to predict the pre-allocated weights of each node in the data center.

[0019] Optionally, before statically managing the infrastructure operation and maintenance of each node in the data center based on the pre-allocated weights of each node in the data center, it also includes: identifying nodes whose resource utilization is greater than a second preset utilization; optimizing the pre-allocated weights according to the nodes whose resource utilization is greater than the second preset utilization, wherein the weights of the nodes whose resource utilization is greater than the second preset utilization are reduced, and the weights of the nodes whose resource utilization is less than or equal to the second preset utilization are increased.

[0020] The second aspect of the present invention provides an infrastructure operation and maintenance management device for a data center, including: an acquisition module, used to acquire the operating status of the infrastructure of each node in the data center at the current moment; a calculation module, used to calculate the resource utilization of each node in the data center at the current moment according to the operating status of the infrastructure of each node in the data center at the current moment, and calculate the total resource utilization of the data center at the current moment according to the resource utilization of each node in the data center at the current moment; a static management module, used to obtain the pre-allocated weights of each node in the data center if the total resource utilization is less than or equal to a first preset utilization and the actual number of nodes in the data center whose resource utilization is greater than the second preset utilization is less than or equal to a preset number, and statically manage the infrastructure operation and maintenance of each node in the data center based on the pre-allocated weights of each node in the data center; a dynamic management module, used to obtain the resource utilization of each node in the data center when executing the current task if the total resource utilization is greater than the first preset utilization, or the actual number of nodes in the data center whose resource utilization is greater than the second preset utilization is greater than the preset number, calculate the target allocation weight of each node in the data center according to the resource utilization of each node in the data center when executing the current task, and dynamically manage the infrastructure operation and maintenance of each node in the data center based on the target allocation weight of each node in the data center.

[0021] The third aspect of the present invention provides a digital platform, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the above-mentioned data center infrastructure operation and maintenance management method.

[0022] Therefore, the embodiments of the present invention can significantly improve the efficiency and reliability of data center infrastructure operation and maintenance, reduce operation and maintenance costs, and improve user experience by building an efficient operation and maintenance network computing power support system. The main beneficial effects are as follows:

[0023] 1. Improve operation and maintenance efficiency: Through intelligent scheduling and resource optimization, the processing efficiency of data center infrastructure operation and maintenance tasks can be significantly improved;

[0024] 2. Resource utilization optimization: Dynamic scheduling is used to balance the load of each node in the data center, maximize the use of computing resources, and avoid resource waste;

[0025] 3. Reduce operation and maintenance costs: Reduce overall operation and maintenance costs by improving resource utilization and operation and maintenance efficiency.

[0026] 4. Enhance the stability of the entire data center system: through real-time monitoring and dynamic scheduling, ensure the stable operation of the system under high load conditions;

[0027] 5. Improve user experience: Improve user satisfaction with network services by quickly responding and efficiently handling operation and maintenance tasks.

[0028] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0030] Figure 1 A flowchart of a data center infrastructure operation and maintenance management method provided according to an embodiment of the present invention;

[0031] Figure 2 This is an example diagram of a data center infrastructure operation and maintenance management device according to an embodiment of the present invention;

[0032] Figure 3 Schematic diagram of the structure of a digital platform according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0034] In order to effectively support the operation and maintenance management of large-scale data centers, improve operation and maintenance efficiency, and reduce operation and maintenance costs, a new operation and maintenance network computing power support system is urgently needed to improve the operation and maintenance management level of data center infrastructure through intelligent and automated means. To this end, an embodiment of the present invention provides an operation and maintenance network computing power support system, which improves the overall efficiency and response speed of data center infrastructure operation and maintenance by reasonably allocating and scheduling computing resources.

[0035] The following describes the data center infrastructure operation and maintenance management method, device and digital platform of the present invention with reference to the accompanying drawings. Specifically, Figure 1 A flowchart of a data center infrastructure operation and maintenance management method provided by an embodiment of the present invention.

[0036] like Figure 1 As shown, the data center infrastructure operation and maintenance management method includes the following steps:

[0037] In step S101, the operating status of the infrastructure of each node in the data center at the current moment is obtained.

[0038] Among them, the infrastructure of each node in the data center may include electricity, HVAC and other infrastructure, and the operating status may include power load, temperature and humidity, equipment health status, etc.

[0039] It can be understood that the embodiments of the present invention can deploy monitoring tools at each node in the data center, and form a distributed monitoring system by deploying monitoring tools at each node. The distributed monitoring system can be used to obtain the operating status of each node, such as power load, temperature and humidity, and equipment health, in real time. Since the operating status of the infrastructure of each node in the data center affects the stability and efficiency of the entire data center, the embodiments of the present invention can accurately obtain the operating status of each node in real time through the distributed monitoring system, thereby effectively improving the stability and efficiency of the entire data center.

[0040] Specifically, the distributed monitoring system is used to obtain the power load, temperature and humidity, equipment health status and other operating conditions of each node in real time as follows:

[0041] 1. Define monitoring indicators: First, you need to determine which indicators are critical. For example, monitoring indicators include network traffic, temperature, humidity, power status, etc.

[0042] 2. Choose the right monitoring tool: You can choose monitoring tools, such as Nagios, Zabbix, Prometheus, Grafana, etc. Specifically: Choose monitoring tools based on the following factors: (1) Determine what types of data need to be monitored (such as hardware, software, network, application, etc.); (2) Determine the frequency and accuracy of monitoring; (3) Scalability: The monitoring tool should be able to expand with the expansion of the data center, supporting more nodes and more complex monitoring requirements; (4) Integration: Consider whether the monitoring tool can be integrated with other systems (such as configuration management databases, automation tools, alarm systems, etc.); (5) Ease of use: The monitoring tool should be easy to install, configure and use, reducing the burden on operation and maintenance personnel; (6) Cost: Consider the purchase cost, maintenance cost and operation cost of the monitoring tool; (7) Security: The monitoring tool should have sufficient security measures to prevent data leakage and malicious attacks.

[0043] 3. Deploy monitoring agent: Deploy monitoring agent on each node. The monitoring agent is responsible for collecting the node's operating data and sending it to the monitoring server. The agent can be lightweight, such as Prometheus's Node Exporter.

[0044] 4. Configure the monitoring server: The monitoring server receives and processes monitoring data. Configure the monitoring server to receive data from each node and store and analyze it.

[0045] In step S102, the resource utilization of each node in the data center at the current moment is calculated according to the operating status of the infrastructure of each node in the data center at the current moment, and the total resource utilization of the data center at the current moment is calculated according to the resource utilization of each node in the data center at the current moment.

[0046] It can be understood that the embodiments of the present invention can monitor and calculate the resource utilization of each node in the data center in real time, ensure the effective utilization of resources, and achieve timely response.

[0047] Specifically, resource utilization is a key indicator to measure the usage of various resources on data center nodes, which helps to understand the efficiency of resource usage and potential bottlenecks. For example:

[0048] CPU utilization: Indicates the extent to which the CPU is used by applications or system processes over a certain period of time. High CPU utilization may indicate that the CPU is being fully utilized, but if it remains too high, you may need to consider expanding CPU resources or optimizing applications.

[0049] Memory utilization: Indicates the usage of physical memory, including the memory occupied by the operating system, applications, and other processes. Memory utilization reflects the system's memory usage efficiency. High utilization may mean that memory resources are tight and it is necessary to increase memory or optimize memory usage.

[0050] Disk utilization: usually refers to the usage of disk storage space, including data storage and logging. High disk utilization may mean that the storage space is close to full, and you need to consider increasing storage capacity or cleaning up unnecessary data.

[0051] Network utilization: Indicates the usage of network bandwidth, including sending and receiving data. High network utilization may indicate a large amount of data transmission, which may cause network congestion and performance degradation if it exceeds the network bandwidth limit.

[0052] Therefore, the importance of resource utilization can include: performance monitoring: resource utilization is the basis for monitoring system performance and can help us discover performance bottlenecks; capacity planning: by analyzing the trend of resource utilization, we can predict future resource needs and make reasonable capacity planning; cost-benefit analysis: understanding resource usage helps to evaluate whether resource allocation is economical and efficient and avoid waste of resources; system stability: excessive resource utilization may lead to system instability, and timely monitoring and adjustment can ensure the stable operation of the system; fault diagnosis: abnormal changes in resource utilization may be a signal of system failure or configuration error, which helps to quickly locate the problem.

[0053] For example, if CPU utilization is continuously above 90%, you may need to consider adding CPU cores or optimizing the CPU utilization efficiency of the application. If memory utilization reaches 85%, you may need to increase memory capacity or check for memory leaks. If disk utilization is close to 90%, you may need to clean up disk space or expand storage. If network utilization reaches 70% during peak hours, you may need to upgrade network equipment or increase bandwidth.

[0054] Taking resource utilization including CPU utilization, memory utilization, disk utilization and network utilization as an example, the following will explain the calculation method of CPU utilization, memory utilization, disk utilization and network utilization, as follows:

[0055] 1. CPU utilization =

[0056] 100-(avgby(instance)(irate(nodecpusecondstotalmode="idle"[5m]))*100),

[0057] Among them, avg by(instance): This is an aggregation function in PromQL, which is used to calculate the average of different values ​​of the specified tag;

[0058] irate(node_cpu_seconds_total{mode="idle"}[5m]): This is a function in PromQL that calculates the growth rate per second within a given time window; node_cpu_seconds_total: is a metric that indicates the total number of seconds the CPU is in different modes; mode="idle": is a label selector that indicates that the focus is on the idle mode of the CPU; [5m]: indicates that the focus is on the data of the last 5 minutes.

[0059] 2. Memory utilization =

[0060] (1-(node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes))*100,

[0061] Among them, node_memory_MemAvailable_bytes: This is a metric that indicates the amount of memory available in the system (in bytes). node_memory_MemTotal_bytes: This is a metric that indicates the total amount of memory in the system (in bytes).

[0062] 3. Disk utilization =

[0063] 100*(1-(sum(node_filesystem_free_bytes=" / dev / sda") / sum(node_filesystem_size_bytes

[0064] = " / dev / sda"))),

[0065] Among them, node_filesystem_free_bytes: This is a metric that indicates the number of free bytes of the file system on the specified device; node_filesystem_size_bytes: This is a metric that indicates the total number of bytes of the file system on the specified device; sum(): This is an aggregation function in PromQL, which is used to sum all matching sample values; device = " / dev / sda": It is a label selector, indicating that the focus is on the disk utilization of a specific device.

[0066] 4. Network utilization =

[0067] 100*(sum(rate(node_network_transmit_bytes_total[5m]))+sum(rate(node_network_transmit_bytes_total[5m]))) / (node_network_speed_bytes*8),

[0068] Among them, rate(node_network_receive_bytes_total[5m]): This is a function in PromQL that calculates the growth rate of the number of bytes received per second within a given time window. [5m] means that we are concerned about the data in the last 5 minutes; rate(node_network_transmit_bytes_total[5m]): Similarly, this is the growth rate of the number of bytes sent per second; sum(): Sum all matching sample values, here used to calculate the total number of received and sent bytes; node_network_speed_bytes: This is a metric that indicates the total bandwidth of the network interface (in bytes); *8: Because bandwidth is usually measured in bits, and 1 byte = 8 bits, multiply by 8 here to convert bandwidth to bytes.

[0069] Furthermore, through the formula Calculate the current total resource utilization, where R(t) is the total resource utilization at time t, r i (t) is the resource utilization of the ith node, and n is the total number of nodes.

[0070] In step S103, if the total resource utilization is less than or equal to the first preset utilization, and the actual number of nodes in the data center whose resource utilization is greater than the second preset utilization is less than or equal to the preset number, the pre-allocated weights of each node in the data center are obtained, and the infrastructure operation and maintenance of each node in the data center are statically managed based on the pre-allocated weights of each node in the data center.

[0071] Among them, the first preset utilization rate, the second preset utilization rate and the preset quantity can be specifically set according to the actual situation of the data center and are not specifically limited.

[0072] It can be understood that the total resource utilization rate is less than or equal to the first preset utilization rate, indicating that the current computing pressure of the data center is relatively small, and the actual number of nodes in the data center whose resource utilization rate is greater than the second preset utilization rate is less than or equal to the preset number, indicating that there are fewer nodes with higher resource utilization rates, and the computing resources of the data center can still maintain a balance. Therefore, the embodiment of the present invention can statically manage the operation and maintenance of the infrastructure of each node in the data center through pre-allocated weights, and can effectively improve the response speed in the data by setting the pre-allocated weights in advance, and can efficiently handle operation and maintenance tasks, thereby improving user satisfaction with network services.

[0073] In an embodiment of the present invention, obtaining the pre-allocated weight of each node in a data center includes: obtaining historical data in the data and pre-processing the historical data; performing statistical analysis on the pre-processed data to obtain data distribution, visualizing the pre-processed data to obtain data trends, extracting the pre-processed data according to the data distribution and the data trends to generate a data set, and dividing the data set into a training set and a test set; using at least one of a linear regression model, a decision tree, a random forest, a support vector machine, a neural network, and a time series analysis model as a prediction model, iteratively training the prediction model using the training set, and testing the prediction performance of the prediction model using the test set; optimizing the parameters of the prediction model according to the test results, stopping the training until the number of iterative training reaches the target number or the prediction performance of the prediction model meets the target performance, and using the trained prediction model to predict the pre-allocated weight of each node in the data center.

[0074] The target number and target energy can be set according to the actual situation without specific limitation. Data preprocessing can include cleaning data, removing or filling missing values, performing feature engineering, including feature selection, feature extraction and feature conversion, standardizing or normalizing data to eliminate the influence of different dimensions, etc.

[0075] It is understandable that the embodiments of the present invention can analyze the data generated during the operation and maintenance process, provide decision support, and continuously optimize resource scheduling strategies. Data analysis uses big data analysis technology and machine learning models to predict future resource requirements, achieve rapid response and efficient processing of operation and maintenance tasks, and improve user satisfaction with network services.

[0076] Specifically, the prediction model trained based on historical data and using the model to predict the pre-allocated weights of each node in the data center can be divided into the following steps:

[0077] 1. Data Collection:

[0078] Collect historical data, including but not limited to: node CPU usage, memory usage, network bandwidth usage, storage usage, task type and task load, task completion time, and task failure rate.

[0079] 2. Data preprocessing, as follows:

[0080] Clean the data to remove or fill in missing values.

[0081] Perform feature engineering, including feature selection, feature extraction, and feature transformation.

[0082] Standardize or normalize data to eliminate the effects of different dimensions.

[0083] 3. Exploratory Data Analysis:

[0084] Conduct statistical analysis to understand the distribution of the data.

[0085] Visualize data to discover patterns and trends.

[0086] 4. Choose a model

[0087] Choose an appropriate prediction model based on the nature of the problem, which may include: linear regression models, decision trees, random forests, support vector machines, neural networks, and time series analysis models.

[0088] 5. Divide the dataset:

[0089] Divide the dataset into training and testing sets, for example in a ratio of 80:20 or 70:30.

[0090] 6. Model Training

[0091] Train the model using the training set data. Adjust model parameters and perform cross-validation to optimize model performance.

[0092] 7. Model Evaluation

[0093] Use the test set to evaluate the predictive performance of the model. Evaluation metrics can include mean square error, root mean square error, coefficient of determination, etc.

[0094] 8. Model Optimization

[0095] Adjust and optimize the model based on the evaluation results, including further optimization of feature engineering, model parameter adjustment, ensemble learning, etc.

[0096] 9. Weight prediction

[0097] Use the trained model to predict the pre-allocated weights of each node in the data center. The weights can be the node's ability to process tasks, resource utilization, or other related indicators.

[0098] In an embodiment of the present invention, before statically managing the infrastructure operation and maintenance of each node in the data center based on the pre-allocated weights of each node in the data center, it also includes: identifying nodes whose resource utilization is greater than a second preset utilization; optimizing the pre-allocated weights according to the nodes whose resource utilization is greater than the second preset utilization, wherein the weights of the nodes whose resource utilization is greater than the second preset utilization are reduced, and the weights of the nodes whose resource utilization is less than or equal to the second preset utilization are increased.

[0099] It can be understood that for nodes whose resource utilization is greater than the second preset utilization, it means that the resource occupancy rate of the node is relatively high. At this time, in order to maintain load balancing, improve resource utilization and operation and maintenance efficiency, the embodiment of the present invention can, after obtaining the pre-allocated weight, reduce the weight of the node whose resource utilization is greater than the second preset utilization rate, and increase the weight of the node whose resource utilization is less than or equal to the second preset utilization rate, thereby effectively improving the processing efficiency of operation and maintenance tasks, effectively utilizing computing resources, avoiding resource waste, and reducing overall operation and maintenance costs.

[0100] It should be noted that the weight of the node whose resource utilization is greater than the second preset utilization can be reduced by specifically matching the reduced weight according to the resource utilization, or the corresponding node can be directly not assigned a weight, so as to make full use of the nodes with lower resource occupancy rate and effectively improve the utilization of resources.

[0101] In step S104, if the total resource utilization is greater than the first preset utilization, or the actual number of nodes in the data center whose resource utilization is greater than the second preset utilization is greater than the preset number, the resource utilization of each node in the data center when performing the current task is obtained, and the target allocation weight of each node in the data center is calculated based on the resource utilization of each node in the data center when performing the current task. The infrastructure operation and maintenance of each node in the data center is dynamically managed based on the target allocation weight of each node in the data center.

[0102] It is understandable that when it is detected that resource utilization exceeds a certain threshold or the request queue grows too fast, dynamic scheduling will be triggered to automatically allocate additional resources or start spare computing nodes. Therefore, through intelligent scheduling and resource optimization, the processing efficiency of data center infrastructure operation and maintenance tasks is significantly improved. Through dynamic scheduling, the load of each node in the data center is balanced, the computing resources are maximized, and resource waste is avoided; by improving resource utilization and operation and maintenance efficiency, the overall operation and maintenance costs are reduced; through real-time monitoring and dynamic scheduling, the stable operation of the system under high load conditions is guaranteed;

[0103] In an embodiment of the present invention, a target allocation weight of each node in the data center is calculated based on the resource utilization of each node in the data center when executing the current task, including: obtaining the total number of nodes in the data center; inputting the total number of nodes and the resource utilization of each node in the data center into a load balancing algorithm, and the load balancing algorithm outputs the total allocation weight of the data center; and calculating the target allocation weight of each node in the data center based on the resource utilization of each node in the data center and the total allocation weight.

[0104] It is understandable that the embodiments of the present invention can dynamically adjust the allocation of computing resources, and avoid resource waste while ensuring that each node runs efficiently through a load balancing algorithm. The load balancing algorithm uses the following formula:

[0105]

[0106] Among them, f(x) is the total allocation weight, r i (x) is the resource utilization of the i-th node when executing the current task x, and n is the total number of nodes;

[0107] The target allocation weight is calculated as:

[0108] R = f(x)r i (x) -1 ,

[0109] Among them, R is the target allocation weight.

[0110] Therefore, the embodiment of the present invention can adjust resource allocation in real time through the load balancing function to effectively deal with emergencies.

[0111] In an embodiment of the present invention, the operation and maintenance of infrastructure of each node in the data center is dynamically managed based on the target allocation weight of each node in the data center, including: obtaining a task queue and the priority of each target task in the task queue; generating a priority queue according to the task queue and the priority of each target task; allocating nodes to the target tasks in the priority queue based on a greedy algorithm and the target allocation weight, wherein nodes with large target allocation weights are allocated to target tasks with high priorities.

[0112] It can be understood that the dynamic scheduling of the embodiment of the present invention can reallocate tasks to different nodes according to the priority and type of the tasks to optimize the overall performance and response time, so that the priority queue and greedy algorithm can be used to schedule tasks according to the importance and urgency of the tasks, effectively ensuring that critical tasks are given priority and ensuring the stable operation of the system under high load conditions.

[0113] In an embodiment of the present invention, the greedy algorithm includes: traversing the priority queue in sequence until the priority queue is empty, and performing the following operations each time the traversal is performed: selecting the target task with the highest priority from the priority queue; traversing all nodes to find the node that can accommodate the target task with the highest priority, and assigning the target task with the highest priority to the node with the largest target allocation weight; if the node that can accommodate the target task with the highest priority is not found, determining the minimum number of required nodes based on the resources required for the target task with the highest priority and the resource utilization of all nodes when executing the current task, and assigning the target task with the highest priority according to the target allocation weight of the selected node; updating the resource utilization of the selected node when executing the current task, and updating the target allocation weights of all nodes based on the resource utilization obtained after the update.

[0114] It can be understood that a greedy algorithm is an algorithm strategy that takes the best or optimal choice in the current state at each step, hoping to lead to the best or optimal result in the world, as follows:

[0115] 1. Define tasks and nodes:

[0116] Task set T, each task t has a weight w t and demand t .

[0117] A set of nodes N, each node n has a capacity c n .

[0118] 2. Initialization: Create an empty priority queue PQ to store tasks. Tasks are sorted according to weight w. t Sort in descending order.

[0119] 3. Task sorting: sort all tasks according to weight w t Sort from largest to smallest and add the sorted tasks to the priority queue PQ.

[0120] 4. Node evaluation: Each node n is evaluated according to a certain standard (such as the node capacity c n , distance to the task, processing capability, etc.), and use the evaluation result as the priority of the node.

[0121] 5. Task allocation: When PQ is not empty, perform the following operations: a. Take the task t with the highest weight from PQ. b. Traverse all nodes n and find the node that can accommodate task t, i.e., d. t ≤c n c. Select the optimal node based on the node evaluation criteria, such as selecting the node with the largest remaining capacity.

[0122] 6. Node selection: Select nodes according to the order of target allocation weights and assign task t to the corresponding one or more nodes.

[0123] 7. Update status: Update the resource utilization of the selected node under the execution of the current task, and update the target allocation weights of all nodes according to the updated resource utilization.

[0124] 8. Repeat allocation: Repeat steps 5 to 7 until the PQ is empty, that is, all tasks have been allocated.

[0125] 9. The algorithm ends.

[0126] The algorithm ends when all tasks are assigned or there are no nodes to assign tasks.

[0127] It should be noted that the greedy algorithm can provide a sufficiently good approximate solution, the evaluation criteria of tasks and nodes can be adjusted according to actual problems, and the implementation of the priority queue can use a heap data structure to ensure fast dequeuing and enqueuing operations of tasks.

[0128] In an embodiment of the present invention, the target task with the highest priority is allocated according to the target allocation weight of the selected node, including: dividing the target task with the highest priority into multiple subtasks; identifying whether the number of allocated subtasks for each selected node is an integer; if the number of allocated subtasks is not an integer, performing a rounding operation, and optimizing the number of allocated subtasks for each selected node after performing the rounding operation.

[0129] It is understandable that when a node cannot process the target task with the highest priority, multiple nodes can be used to process it in a coordinated manner. In order to efficiently process the target task, the target task can be divided into multiple subtasks to facilitate task allocation. Since the data of the subtask is an integer, the weight of the node is usually not an integer. If the node is allocated according to the target allocation weight, it is easy to cause the allocated subtask to be non-integer, such as 1.4 or 1.5 subtasks, which is not conducive to the processing of the target task.

[0130] Therefore, the embodiments of the present invention can effectively solve the above problems through rounding operations, such as rounding down or rounding up, etc., so as to effectively improve the processing efficiency of the target tasks and the user experience by rounding the assigned tasks, which is more in line with the actual operating conditions.

[0131] The data center infrastructure operation and maintenance management method proposed in the embodiment of the present invention significantly improves the processing efficiency of data center infrastructure operation and maintenance tasks through intelligent scheduling and resource optimization, balances the load of each node in the data center through dynamic scheduling, maximizes the use of computing resources, avoids resource waste, reduces overall operation and maintenance costs by improving resource utilization and operation and maintenance efficiency, and ensures stable operation of the system under high load conditions through real-time monitoring and dynamic scheduling; improves user satisfaction with network services through rapid response and efficient processing of operation and maintenance tasks, thereby significantly improving the efficiency and reliability of data center infrastructure operation and maintenance, reducing operation and maintenance costs, and improving user experience.

[0132] The following will describe the data center infrastructure operation and maintenance management method through a specific embodiment, as follows:

[0133] 1. Resource monitoring module

[0134] The resource monitoring module is the foundation of the entire system and is responsible for collecting and monitoring the operating status of the power, HVAC and other infrastructure of each node in the data center. It obtains key indicators such as power load, temperature and humidity, and equipment health status of each node in real time through a distributed monitoring system. The resource monitoring module uses a distributed database to store monitoring data for quick query and analysis.

[0135] 2. Resource Scheduling Module

[0136] The resource scheduling module dynamically adjusts the allocation of computing resources based on the data provided by the resource monitoring module. Its core is the load balancing algorithm, which ensures that each node runs efficiently while avoiding resource waste. The scheduling strategy includes static scheduling and dynamic scheduling. Static scheduling pre-allocates resources based on historical data and prediction models, while dynamic scheduling adjusts resource allocation in real time to deal with emergencies.

[0137] 3.Task management module

[0138] The task management module is responsible for receiving, allocating and managing operation and maintenance tasks, including inspection and maintenance tasks. It uses priority queues and greedy algorithms to schedule tasks based on the importance and urgency of the tasks. The specific steps are as follows:

[0139] Receiving tasks: The task management module receives new tasks and adds them to the queue to be processed.

[0140] Task classification: Assign priorities based on the importance and urgency of tasks.

[0141] Task scheduling: A greedy algorithm is used to prioritize high-priority tasks and ensure that critical tasks are processed first.

[0142] Task execution: Assign tasks to the most appropriate nodes for execution.

[0143] 4. Data Analysis Module

[0144] The data analysis module analyzes the large amount of data generated during the operation and maintenance process, provides decision support, and continuously optimizes resource scheduling strategies. Data analysis uses big data technology and machine learning models, and mainly includes the following steps:

[0145] Data collection: Collect logs, monitoring data, and task execution records during the operation and maintenance process.

[0146] Data preprocessing: Clean, filter and format the data to ensure data quality.

[0147] Model training: Use machine learning algorithms to train historical data and build a predictive model.

[0148] Data analysis: Use trained models to analyze real-time data, predict future resource requirements, and optimize scheduling strategies.

[0149] Therefore, the embodiments of the present invention can solve the problems of insufficient computing resources and low operation and maintenance efficiency existing in the prior art through reasonable technical solutions and system designs, and provide strong technical support and guarantee for the operation and maintenance of data center infrastructure.

[0150] Next, the data center infrastructure operation and maintenance management device proposed in the embodiment of the present invention is described with reference to the accompanying drawings.

[0151] Figure 2 It is a block diagram of a data center infrastructure operation and maintenance management device according to an embodiment of the present invention.

[0152] like Figure 2 As shown, the data center infrastructure operation and maintenance management device 10 includes: an acquisition module 100, a calculation module 200, a static management module 300 and a dynamic management module 400.

[0153] Among them, the acquisition module 100 is used to obtain the operating status of the infrastructure of each node in the data center at the current moment; the calculation module 200 is used to calculate the resource utilization of each node in the data center at the current moment according to the operating status of the infrastructure of each node in the data center at the current moment, and calculate the total resource utilization of the data center at the current moment according to the resource utilization of each node in the data center at the current moment; the static management module 300 is used to obtain the pre-allocated weight of each node in the data center if the total resource utilization is less than or equal to the first preset utilization, and the actual number of nodes in the data center whose resource utilization is greater than the second preset utilization is less than or equal to the preset number, and statically manage the operation and maintenance of the infrastructure of each node in the data center based on the pre-allocated weight of each node in the data center; the dynamic management module 400 is used to obtain the resource utilization of each node in the data center when executing the current task if the total resource utilization is greater than the first preset utilization, or the actual number of nodes in the data center whose resource utilization is greater than the second preset utilization is greater than the preset number, calculate the target allocation weight of each node in the data center according to the resource utilization of each node in the data center when executing the current task, and dynamically manage the operation and maintenance of the infrastructure of each node in the data center based on the target allocation weight of each node in the data center.

[0154] It should be noted that the aforementioned explanation of the data center infrastructure operation and maintenance management method embodiment is also applicable to the data center infrastructure operation and maintenance management device of this embodiment, and will not be repeated here.

[0155] The data center infrastructure operation and maintenance management device proposed in the embodiment of the present invention significantly improves the processing efficiency of data center infrastructure operation and maintenance tasks through intelligent scheduling and resource optimization, balances the load of each node in the data center through dynamic scheduling, maximizes the use of computing resources, avoids resource waste, reduces overall operation and maintenance costs by improving resource utilization and operation and maintenance efficiency, and ensures stable operation of the system under high load conditions through real-time monitoring and dynamic scheduling; improves user satisfaction with network services through rapid response and efficient processing of operation and maintenance tasks, thereby significantly improving the efficiency and reliability of data center infrastructure operation and maintenance, reducing operation and maintenance costs, and improving user experience.

[0156] Figure 3 A schematic diagram of the structure of a digital platform provided in an embodiment of the present invention. The digital platform may include:

[0157] A memory 301 , a processor 302 , and a computer program stored in the memory 301 and executable on the processor 302 .

[0158] When the processor 302 executes the program, the data center infrastructure operation and maintenance management method provided in the above embodiment is implemented.

[0159] Furthermore, the digital platform also includes:

[0160] The communication interface 303 is used for communication between the memory 301 and the processor 302 .

[0161] The memory 301 is used to store computer programs that can be run on the processor 302 .

[0162] The memory 301 may include a high-speed RAM (Random Access Memory) memory, and may also include a non-volatile memory, such as at least one disk memory.

[0163] If the memory 301, the processor 302 and the communication interface 303 are implemented independently, the communication interface 303, the memory 301 and the processor 302 can be connected to each other through a bus and communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0164] Optionally, in a specific implementation, if the memory 301, the processor 302 and the communication interface 303 are integrated on a chip, the memory 301, the processor 302 and the communication interface 303 can communicate with each other through an internal interface.

[0165] The processor 302 may be a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0166] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0167] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0168] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0169] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, the steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0170] A person of ordinary skill in the art may understand that all or part of the steps carried by the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the above-mentioned program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0171] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A data center infrastructure operation and maintenance management method, characterized in that: The following steps are involved: Get the operating status of each node infrastructure in the data center at the current moment; Calculate the resource utilization rate of each node in the data center at the current moment according to the operating status of the infrastructure of each node in the data center at the current moment, and calculate the total resource utilization rate of the data center at the current moment according to the resource utilization rate of each node in the data center at the current moment; If the total resource utilization is less than or equal to the first preset utilization, and the actual number of nodes in the data center whose resource utilization is greater than the second preset utilization is less than or equal to the preset number, the pre-allocated weight of each node in the data center is obtained, and the infrastructure operation and maintenance of each node in the data center is statically managed based on the pre-allocated weight of each node in the data center; If the total resource utilization is greater than the first preset utilization, or the actual number of nodes in the data center whose resource utilization is greater than the second preset utilization is greater than the preset number, then the resource utilization of each node in the data center when executing the current task is obtained, and the target allocation weight of each node in the data center is calculated according to the resource utilization of each node in the data center when executing the current task, and the infrastructure operation and maintenance of each node in the data center is dynamically managed based on the target allocation weight of each node in the data center; The method of calculating the target allocation weight of each node in the data center according to the resource utilization rate of each node in the data center when executing the current task includes: obtaining the total number of nodes in the data center; inputting the total number of nodes and the resource utilization rate of each node in the data center into a load balancing algorithm, and the load balancing algorithm outputs the total allocation weight of the data center; calculating the target allocation weight of each node in the data center according to the resource utilization rate of each node in the data center and the total allocation weight; the calculation formula of the load balancing algorithm is: Among them, f(x) is the total allocation weight, r i (x) is the resource utilization of the i-th node when executing the current task x, and n is the total number of nodes; The calculation formula of the target allocation weight is: R=f(x)r i (x) -1 , Among them, R is the target allocation weight; The method of dynamically managing the operation and maintenance of infrastructure of each node in the data center based on the target allocation weight of each node in the data center includes: obtaining a task queue and the priority of each target task in the task queue; generating a priority queue according to the task queue and the priority of each target task; allocating nodes to the target tasks in the priority queue based on a greedy algorithm and a target allocation weight, wherein nodes with large target allocation weights are allocated to target tasks with high priorities; the greedy algorithm includes: traversing the priority queue in sequence until the priority queue is empty, and performing the following operations each time the traversal is performed: Selecting the target task with the highest priority from the priority queue; Traverse all nodes, find a node that can accommodate the target task with the highest priority, and assign the target task with the highest priority to the node with the largest target assignment weight; If no node that can accommodate the target task with the highest priority is found, the minimum number of required nodes is determined according to the resources required for the target task with the highest priority and the resource utilization of all nodes when executing the current task, and the target task with the highest priority is allocated according to the target allocation weight of the selected node; The resource utilization of the selected node in executing the current task is updated, and the target allocation weights of all nodes are updated according to the updated resource utilization.

2. The data center infrastructure operation and maintenance management method according to claim 1, characterized in that: The allocating the target task with the highest priority according to the target allocation weight of the selected node includes: Divide the target task with the highest priority into multiple subtasks; Identify whether the number of assigned subtasks of each selected node is an integer; If the number of allocated subtasks is not an integer, a rounding operation is performed, and after performing the rounding operation, the number of allocated subtasks of each selected node is optimized.

3. The data center infrastructure operation and maintenance management method according to claim 1, characterized in that: The obtaining of the pre-allocated weight of each node in the data center includes: Acquire historical data from the data, and preprocess the historical data; Performing statistical analysis on the preprocessed data to obtain data distribution, visualizing the preprocessed data to obtain data trends, extracting the preprocessed data according to the data distribution and the data trends to generate a data set, and dividing the data set into a training set and a test set; At least one of a linear regression model, a decision tree, a random forest, a support vector machine, a neural network, and a time series analysis model is used as a prediction model, the prediction model is iteratively trained using the training set, and the prediction performance of the prediction model is tested using the test set; The parameters of the prediction model are optimized according to the test results, and the training is stopped until the number of iterative training reaches the target number or the prediction performance of the prediction model meets the target performance, and the pre-allocated weights of each node in the data center are predicted using the trained prediction model.

4. The data center infrastructure operation and maintenance management method according to claim 1, characterized in that: Before statically managing the operation and maintenance of infrastructure of each node in the data center based on the pre-allocated weights of each node in the data center, the method further includes: Identifying a node whose resource utilization is greater than a second preset utilization; The pre-allocated weights are optimized according to the nodes whose resource utilization is greater than a second preset utilization, wherein the weights of the nodes whose resource utilization is greater than the second preset utilization are reduced, and the weights of the nodes whose resource utilization is less than or equal to the second preset utilization are increased.

5. A data center infrastructure operation and maintenance management device, characterized in that: include: The acquisition module is used to obtain the operating status of the infrastructure of each node in the data center at the current moment; A calculation module, used to calculate the resource utilization rate of each node in the data center at the current moment according to the operating status of the infrastructure of each node in the data center at the current moment, and calculate the total resource utilization rate of the data center at the current moment according to the resource utilization rate of each node in the data center at the current moment; A static management module, configured to obtain the pre-allocated weights of each node in the data center if the total resource utilization is less than or equal to a first preset utilization and the actual number of resource utilizations in each node of the data center greater than the second preset utilization is less than or equal to a preset number, and statically manage the infrastructure operation and maintenance of each node in the data center based on the pre-allocated weights of each node in the data center; A dynamic management module is used to obtain the resource utilization of each node in the data center when executing the current task if the total resource utilization is greater than the first preset utilization, or the actual number of nodes in the data center whose resource utilization is greater than the second preset utilization is greater than the preset number, calculate the target allocation weight of each node in the data center according to the resource utilization of each node in the data center when executing the current task, and dynamically manage the infrastructure operation and maintenance of each node in the data center based on the target allocation weight of each node in the data center; The method of calculating the target allocation weight of each node in the data center according to the resource utilization rate of each node in the data center when executing the current task includes: obtaining the total number of nodes in the data center; inputting the total number of nodes and the resource utilization rate of each node in the data center into a load balancing algorithm, and the load balancing algorithm outputs the total allocation weight of the data center; calculating the target allocation weight of each node in the data center according to the resource utilization rate of each node in the data center and the total allocation weight; the calculation formula of the load balancing algorithm is: Among them, f(x) is the total allocation weight, r i (x) is the resource utilization of the i-th node when executing the current task x, and n is the total number of nodes; The calculation formula of the target allocation weight is: R=f(x)r i (x) -1 , Among them, R is the target allocation weight; The method of dynamically managing the operation and maintenance of infrastructure of each node in the data center based on the target allocation weight of each node in the data center includes: obtaining a task queue and the priority of each target task in the task queue; generating a priority queue according to the task queue and the priority of each target task; allocating nodes to the target tasks in the priority queue based on a greedy algorithm and a target allocation weight, wherein nodes with large target allocation weights are allocated to target tasks with high priorities; the greedy algorithm includes: traversing the priority queue in sequence until the priority queue is empty, and performing the following operations each time the traversal is performed: Selecting the target task with the highest priority from the priority queue; Traverse all nodes, find a node that can accommodate the target task with the highest priority, and assign the target task with the highest priority to the node with the largest target assignment weight; If no node that can accommodate the target task with the highest priority is found, the minimum number of required nodes is determined according to the resources required for the target task with the highest priority and the resource utilization of all nodes when executing the current task, and the target task with the highest priority is allocated according to the target allocation weight of the selected node; The resource utilization of the selected node in executing the current task is updated, and the target allocation weights of all nodes are updated according to the updated resource utilization.

6. A digital platform, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the data center infrastructure operation and maintenance management method according to any one of claims 1 to 4.

Images