Hardware resource management method based on hyper-converged storage system

Abstract

The invention relates to a hardware resource management method based on a hyper-converged storage system. The method comprises the following steps: acquiring resource data from a hyper-converged storage system, virtualizing the resource data and integrating the resource data into a unified resource pool; when a service request is obtained, querying the current state of the resource pool, analyzing the service request, and predicting parameter information required by the service; on the basis of the parameter information, resources required by the service request are allocated according to a preset resource allocation scheme; by adopting the method, resource allocation can be flexibly adjusted, and the resource use efficiency is optimized; resources are allocated to the service under the condition that the resources are sufficient and the state is good, and stable operation of the service is guaranteed; after the service request is received, resources can be quickly allocated to the service, the waiting time of the service is shortened, and the response speed of the service is improved; the resource allocation process is automated and standardized, the possibility of manual intervention and human errors is reduced, and the management efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and particularly relates to a hardware resource management method based on a hyper-converged storage system. Background Art

[0002] With the development of computer technology, there are many deficiencies in traditional hardware resource management and allocation methods. First, with the wide application of cloud computing and virtualization technologies, the demands for various computing resources have become increasingly diversified and dynamically changing, and a single hardware resource management mode often has difficulty in effectively coping with such rapid changes in demands. Second, existing hardware resource management technologies perform poorly in terms of resource utilization efficiency, flexibility, and scalability. Especially in the fields of large-scale data processing and high-performance applications, traditional methods are difficult to provide sufficient support. Third, in order to meet the different demands of different business applications for resources, a management solution that can flexibly adjust resource allocation, optimize resource usage efficiency, and can respond in real time to system load changes is needed.

[0003] The hyper-converged architecture is a new type of technology that has developed rapidly in recent years. It highly integrates computing, storage, and network resources on a physical platform, and uses virtualization technology to pool and uniformly manage these resources, thus achieving highly flexible and on-demand allocation of IT resources. This architecture not only simplifies management and operation and maintenance work, but also effectively improves resource utilization efficiency and system performance. However, despite the many advantages of the hyper-converged architecture, it still faces some challenges in practical applications, especially in the aspects of intelligent and dynamic optimization of resource allocation, and there is still room for improvement in current solutions. Therefore, how to further improve the intelligent level of resource management and achieve more flexible and efficient resource scheduling has become an urgent problem to be solved. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a hardware resource management method based on a hyper-converged storage system that can improve resource utilization rate.

[0005] In a first aspect, the present application provides a hardware resource management method based on a hyper-converged storage system, including:

[0006] Obtaining resource data in the hyper-converged storage system, virtualizing the resource data and integrating it into a unified resource pool; the resource data includes computing resources, storage resources, and network hardware resources;

[0007] When a service request is obtained, querying the current state of the resource pool, and analyzing the service request to predict the parameter information required by the service; the current state includes the remaining capacity of each resource pool, the usage of the allocated resources, and the load information of each node; the parameter information includes computing power parameters, storage capacity parameters, and network bandwidth parameters;

[0008] Based on the parameter information, allocate the resources required for the service request according to a preset resource allocation scheme.

[0009] In one embodiment, when a service request is obtained, query the current status of the resource pool and analyze the service request to predict the parameter information required for the service, including:

[0010] After receiving the service request, identify the type of the service request; the types include high-computation tasks and low-load tasks;

[0011] Query the usage of the current resource pool in real time; the usage includes resource occupancy and available capacity;

[0012] Parse the content of the service request to obtain the resource requirements of the service request; the requirements include emergency processing, specific resource requirements, and data scale;

[0013] Based on the resource requirements, predict the parameter information required for the service request.

[0014] In one embodiment, based on the resource requirements, predict the parameter information required for the service request, including:

[0015] Use a convolutional neural network model to predict the parameter information required for the service request;

[0016] Among them, the convolutional neural network model includes:

[0017] An input layer for inputting the data of the service request; the feature vectors include the request type, timestamp, and historical resource usage;

[0018] A convolutional layer for performing a convolution operation on the input data using a convolution kernel. The convolution operation is performed by sliding the convolution kernel over the input data for element-wise multiplication and summation; among them, the calculation formula for the convolution operation is: Among them, W ji is the weight of the convolution kernel, b j is the bias term, k is the size of the convolution kernel, C j is the output after the convolution operation, X i is the input data; after the convolution operation, a non-linear activation function is applied to increase the non-linear expression ability of the model; a pooling operation is performed after the activation function, and the pooling operation is used to reduce the dimension and computational amount of the data;

[0019] An output layer for outputting the predicted parameter information.

[0020] In one embodiment, obtain resource data in the hyper-converged storage system, virtualize and integrate the resource data into a unified resource pool, including:

[0021] Aggregate the CPU and memory resources of all computing nodes into a computing resource pool, and aggregate the storage space of storage nodes into a storage resource pool;

[0022] Based on the computing resource pool and the storage resource pool, record the initial capacity and performance parameters of each resource pool.

[0023] In one embodiment, based on a service request, according to a preset resource allocation scheme, allocate the resources required by the service request, including:

[0024] When the CPU and memory of the service request are insufficient, create a new virtual machine or adjust the resource configuration of the existing virtual machine;

[0025] When the storage requirement of the service request is insufficient, allocate a new storage volume or adjust the configuration of the existing storage volume;

[0026] When the network bandwidth of the service request is less than the requirement of the service, configure the network bandwidth limit or optimize the network path.

[0027] In one embodiment, it further includes:

[0028] Obtain the load data of the service;

[0029] When the load data increases and a resource shortage situation is recognized, increase the computing resource pool;

[0030] When the load data recovers or decreases and an excess resource situation is recognized, release the computing resource pool.

[0031] In one embodiment, it further includes:

[0032] Monitor the usage of the allocated resources and obtain the data of the allocated resources;

[0033] When a failure occurs in the usage situation, perform real-time backup and recovery of the data.

[0034] In a second aspect, the present application also provides a hardware resource management device based on a hyper-converged storage system, including:

[0035] A resource data acquisition module, configured to acquire resource data in the hyper-converged storage system, virtualize the resource data, and integrate it into a unified resource pool;

[0036] A parameter prediction module, configured to query the current state of the resource pool and analyze the service request to predict the parameter information required by the service when a service request is obtained;

[0037] A resource allocation module, configured to allocate the resources required by the service request based on the parameter information according to a preset resource allocation scheme.

[0038] In a third aspect, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor executes the computer program to perform the method according to any one of the first aspect.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method according to any one of the first aspect is performed.

[0040] For the above-mentioned hardware resource management method based on a hyper-converged storage system, the hyper-converged storage system virtualizes and integrates computing, storage, and network hardware resources into a unified resource pool, improving resource utilization; during the business request processing, the system queries the current state of the resource pool to ensure that resources are allocated to the business when resources are sufficient and in good condition, guaranteeing the stable operation of the business; since the system can quickly query the resource pool status and predict the parameters required by the business, after receiving a business request, resources can be quickly allocated to the business, reducing the waiting time of the business and improving the response speed of the business. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0042] Figure 1 It is a schematic diagram of the application environment of a hardware resource management method based on a hyper-converged storage system in an embodiment;

[0043] Figure 2 It is a schematic flowchart of a hardware resource management method based on a hyper-converged storage system in an embodiment;

[0044] Figure 3 It is a schematic structural diagram of a hardware resource management device based on a hyper-converged storage system in an embodiment;

[0045] Figure 4 It is a schematic structural diagram of a computer device of a hardware resource management method based on a hyper-converged storage system in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] In order to make the objectives, technical solutions, and advantages of the present application clearer, the following further details the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] A hardware resource management method based on a hyper-converged storage system provided by an embodiment of the present application can be applied to, for example, Figure 1 the application environment shown. Among them, the terminal 101 communicates with the server 102 through the network device 102. Among them, the terminal 101 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 103 can be implemented by an independent server or a server cluster composed of multiple servers.

[0048] In one embodiment, as Figure 2 shown, a hardware resource management method based on a hyper-converged storage system is provided. In this embodiment, an example is given where the method is applied to a terminal. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server.

[0049] In this embodiment, the method includes the following steps:

[0050] Step 201, obtain resource data in the hyper-converged storage system, virtualize the resource data, and integrate it into a unified resource pool.

[0051] Among them, the resource data includes computing resources, storage resources, and network hardware resources;

[0052] The computing resources in the hyper-converged storage system are usually provided by components such as the processor (CPU) and memory of the server. The system will collect data such as the CPU usage rate, the number of idle cores, the used capacity and remaining capacity of the memory of each server in real time through specific monitoring tools or interfaces; the storage resources cover various storage devices in the system, such as local hard disks, solid state drives (SSDs), etc. The system will obtain data such as the total capacity, used capacity, available space, read and write speed, and I / O operation latency of the storage device; the network hardware resources include network devices such as switches and routers, and network interfaces on the server; the system will obtain data such as the bandwidth utilization rate of the network device, the port status (connection status, rate, etc.), and the distribution of network traffic.

[0053] Using virtualization technologies (such as Kernel-based Virtual Machine), the computing resources (CPU, memory) of physical servers are virtualized into multiple virtual computing units (virtual machines or containers); through storage virtualization technologies (such as software-defined storage SDS), scattered physical storage devices are virtualized into a unified storage resource pool; network function virtualization (NFV) technology is adopted to virtualize the functions of network devices (such as routing, switching, firewall, etc.) into software form and run on general servers.

[0054] Step 202, when a service request is obtained, query the current status of the resource pool, and analyze the service request to predict the parameter information required by the service; the current status includes the remaining capacity of each resource pool, the usage of the allocated resources, and the load information of each node; the parameter information includes computing power parameters, storage capacity parameters, and network bandwidth parameters.

[0055] Among them, the computing power parameters include the required number of CPU cores, CPU main frequency, and memory size, the storage capacity parameters include the size of the storage space to be allocated, and the network bandwidth parameters include the required network transmission speed.

[0056] Exemplarily, when a service request is obtained, querying the current status of the resource pool and analyzing the service request to predict the parameter information required by the service may include the following steps:

[0057] Step 1001, when the hyper-converged storage system receives a service request, it will first query the current status of each resource pool in the system. The query includes understanding the remaining capacity of the computing resource pool, storage resource pool, and network hardware resource pool, and clarifying how much available resources can be allocated to new service requests.

[0058] Step 1002, view the usage of the allocated resources, such as the CPU and memory usage ratios of the virtual machines allocated to existing services, the read / write frequencies of the storage volumes, and the occupancy of the network bandwidth, etc., so as to evaluate the overall utilization status of the current system resources.

[0059] Step 1003, also obtain the load information of each node, understand the workloads of each computing node, storage node, and network node, and judge which nodes are in a high-load state and which nodes still have remaining processing capabilities.

[0060] Step 1004, based on an understanding of the current state of the resource pool, the system will conduct an in-depth analysis of the service request. By evaluating factors such as the type, scale, and estimated running time of the service request, the system predicts the computing power parameters, storage capacity parameters, and network bandwidth parameters required by the service during operation; exemplarily, for a big data analysis service, the system will, based on factors such as the amount of data to be processed and the complexity of the analysis algorithm, predict how many CPU cores and how much memory are needed to ensure computing efficiency, how much storage capacity is needed to store the original data and analysis results, and how much network bandwidth is needed to transmit the data.

[0061] Step 203, based on the parameter information and according to the preset resource allocation scheme, allocate the resources required by the service request.

[0062] Exemplarily, based on the parameter information and according to the preset resource allocation scheme, allocating the resources required by the service request may include the following steps:

[0063] Step 2001, through the analysis of the service request, the parameter information required by the service is predicted. Among them, the parameter information may include computing power parameters (such as the required number of CPU cores, CPU main frequency, and memory size), storage capacity parameters (such as the size of the storage space to be allocated), and network bandwidth parameters (such as the required network transmission speed).

[0064] Step 2002, perform resource matching and allocation based on the parameter information and the preset scheme; exemplarily, according to the computing power parameters, the system will select appropriate computing resources from the computing resource pool and allocate them to the service request; based on the storage capacity parameters, the system will allocate a storage volume of the corresponding size from the storage resource pool to the service; according to the network bandwidth parameters, the system will allocate and adjust the network resources.

[0065] Optionally, when a service request is received, query the current state of the resource pool and analyze the service request to predict the parameter information required by the service, including:

[0066] Step 3001, after receiving the service request, identify the type of the service request.

[0067] When the hyper-converged system receives a service request, it judges the type of the request. The system determines its type based on various information such as the characteristics, source, and application scenario of the service request. For tasks that require large-scale data calculation and complex algorithm processing, they can be determined as high-computation tasks, and some simple file reading and basic data query tasks that consume less system resources can be classified as low-load tasks. Identifying the type of the service request helps the system to reasonably allocate resources according to the characteristics of different types of tasks in the follow-up.

[0068] Step 3002, query the usage of the current resource pool in real time; the usage includes resource occupancy and available capacity.

[0069] The resource pool is a unified collection that integrates resources such as computing, storage, and network in a hyper-converged storage system. Querying the usage of the resource pool in real time can obtain the occupancy status of various resources in the current resource pool and the remaining available capacity. Exemplarily, in terms of computing resources, understand the CPU usage rate, the used space and free space of memory; in terms of storage resources, know the amount of data stored on the hard disk and the remaining storage capacity; in terms of network resources, master the occupancy ratio of network bandwidth, etc.

[0070] Step 3003, parse the content of the service request to obtain the resource requirements of the service request; the requirements include emergency processing, specific resource requirements, and data scale.

[0071] The hyper-converged system deeply analyzes the specific content of the service request and extracts the specific resource requirements of the service request from it; among them, the emergency processing requirement refers to whether the service request needs to be processed immediately, with high timeliness requirements; specific resource requirements may involve the need for specific types of computing resources (such as GPU acceleration), storage resources (such as high-speed SSD storage), or network resources (such as high-bandwidth network connections); the data scale refers to the amount of data involved in the service request, which will affect the allocation of storage resources and computing resources.

[0072] Step 3004, based on the resource requirements, predict the parameter information required for the service request.

[0073] After clearly understanding the type and resource requirements of the service request, the system will predict the specific parameters required to complete the service request based on these; Exemplarily, for a service request with high computational tasks and a large data scale, predict parameters such as the number of CPU cores to be allocated, memory capacity, storage capacity, and the estimated processing time; for low-load tasks, appropriate resource parameters will also be predicted accordingly to avoid resource waste; these predicted parameter information will be used for subsequent resource allocation and scheduling to ensure that the service request can run efficiently while meeting its requirements.

[0074] In the above hardware resource management method based on a hyper-converged storage system, the resource allocation can be flexibly adjusted, and the resource usage efficiency can be optimized; ensuring that resources are allocated to services when resources are sufficient and in good condition, which guarantees the stable operation of the services; after receiving a service request, resources can be quickly allocated to the service, reducing the waiting time of the service and improving the response speed of the service; the resource allocation process is automated and standardized, reducing the possibility of manual intervention and human errors, and improving the management efficiency.

[0075] Optionally, based on resource requirements, predict the parameter information required for a service request, including:

[0076] Use a convolutional neural network model to predict the parameter information required for a service request;

[0077] A Convolutional Neural Network (CNN) is a powerful deep learning model, especially suitable for processing data with a grid structure. In a hyper-converged storage system, the CNN model is used to predict the parameter information required for a service request because it can automatically extract features from the input data and learn the complex relationships between service requests and required resource parameters.

[0078] Among them, the convolutional neural network model includes:

[0079] An input layer, used to input the data of the service request; the feature vector includes the request type, timestamp, and historical resource usage;

[0080] The role of the input layer is to receive the data of the service request. The input data is presented in the form of a feature vector, and the feature vector includes the request type (such as high-computation tasks or low-load tasks), timestamp (recording the time when the service request occurs), and historical resource usage (reflecting the past resource usage of this service or similar services); these features can provide the model with multi-faceted information about the service request, helping the model better understand the service requirements and thus make more accurate predictions.

[0081] A convolutional layer, used to perform a convolution operation on the input data using a convolutional kernel. The convolution operation is performed by sliding the convolutional kernel over the input data for element-wise multiplication and summation; among them, the calculation formula for the convolution operation is: Among them, W ji is the weight of the convolutional kernel, b j is the bias term, k is the size of the convolutional kernel, C j is the output after the convolution operation, X i is the input data;

[0082] The convolutional layer is the core part of the CNN. It performs a convolution operation on the input data using a convolutional kernel; the convolutional kernel is a small matrix that slides over the input data for element-wise multiplication and summation operations; the formula describes the calculation process of the convolution operation. Through the convolution operation, the model can extract local features from the input data, such as local association patterns between different types of service requests and resource requirements.

[0083] After the convolution operation, apply a non-linear activation function to increase the non-linear expression ability of the model;

[0084] After the convolution operation, a non-linear activation function (such as the ReLU function) is applied. The role of the non-linear activation function is to introduce non-linearity into the model, enabling the model to learn more complex patterns and relationships. Without a non-linear activation function, the entire CNN model would degenerate into a linear model and be unable to handle the non-linear relationship between complex business requests and resource requirements.

[0085] A pooling operation is performed after the activation function. The pooling operation is used to reduce the dimensionality and computational complexity of the data;

[0086] A pooling operation is performed after the activation function. Common pooling operations include max pooling and average pooling; Exemplarily, using max pooling, the main purpose of the pooling operation is to reduce the dimensionality and computational complexity of the data while retaining the important features of the data; Through the pooling operation, the complexity of the model can be reduced, the training speed can be increased, and the generalization ability of the model can be enhanced.

[0087] The output layer is used to output the predicted parameter information.

[0088] Based on the features and patterns learned by the previous layers, the output layer outputs the predicted parameter information. These parameter information can be the computational resources (such as the number of CPU cores and the amount of memory) required for business requests, storage resources (such as disk space), and network resources (such as bandwidth), etc., providing a specific basis for resource allocation in the hyper-converged storage system.

[0089] Optionally, when the hyper-converged storage system obtains resource data, virtualizes the resource data and integrates it into a unified resource pool, including the following steps:

[0090] Step 4001, aggregate the CPU and memory resources of all computing nodes into a computing resource pool, and aggregate the storage spaces of storage nodes into a storage resource pool.

[0091] In a hyper-converged storage system, there are usually multiple computing nodes, and each computing node is equipped with a certain number of CPU cores and memory; Through specific management software and technical means, the system will uniformly collect and integrate the CPU and memory resources of these computing nodes; Exemplarily, in a hyper-converged cluster composed of multiple servers, each server serves as a computing node, and the management software will obtain information such as the number of CPU cores, CPU main frequency, and memory capacity of each server, and then aggregate these resources together to form a computing resource pool that can be uniformly allocated and scheduled; The entire system can regard these computing resources as a whole and flexibly allocate them to different applications or virtual machines according to business requirements.

[0092] Similarly, the storage nodes are responsible for providing storage space. These storage nodes can be local hard disks, solid-state drives, or external storage devices, etc.; the system will identify and count the storage space of each storage node, aggregate their available capacities, and build a storage resource pool; exemplarily, integrate the hard disk storage capacities of multiple servers and the capacities of external storage arrays to form a unified storage resource pool; users or application programs can obtain the required storage space from this storage resource pool through the system interface without caring about the specific storage device location.

[0093] Step 4002: Based on the computing resource pool and the storage resource pool, record the initial capacities and performance parameters of each resource pool.

[0094] After the integration of the computing resource pool and the storage resource pool is completed, the system will record the initial capacity of each resource pool; for the computing resource pool, it will record the total number of CPU cores, total storage capacity, etc. after aggregation; for the storage resource pool, it will record the total available storage space size; the initial capacity information is the basis for subsequent resource allocation and management, and the system can judge the resource usage and remaining situation based on this information; the system will also record performance parameters; for the computing resource pool, the performance parameters can include the average main frequency of the CPU, the read / write speed of the memory, etc.; for the storage resource pool, the performance parameters can include the read / write bandwidth of the storage device, the latency of I / O operations, etc.; these performance parameters can help the system better understand the characteristics of the resources, and when performing resource allocation, it can make a more reasonable allocation according to the business requirements and the performance characteristics of the resources.

[0095] Optionally, based on the service request, according to the preset resource allocation scheme, allocate the resources required by the service request, including the following steps:

[0096] Step 5001: When the CPU and memory of the service request are insufficient, create a new virtual machine or adjust the resource configuration of the existing virtual machine;

[0097] When the hyper-converged system receives a service request and after analysis finds that the currently available computing resources (CPU and memory) are insufficient to meet the service operation requirements, the system will create a new virtual machine according to the preset rules. During the creation of the new virtual machine, the system will allocate the corresponding number of CPU cores and memory capacity from the computing resource pool to this virtual machine. Exemplarily, if a data mining service requires a large amount of computing resources to process complex algorithms and the existing virtual machines in the current system cannot provide sufficient CPU and memory, the system will create a new virtual machine and allocate an appropriate number of CPU cores and sufficient memory to ensure that the service can run normally.

[0098] In addition to creating new virtual machines, the system can also adjust the resource configurations of existing virtual machines. If the current resource utilization of some virtual machines is low and the business requests urgently need more computing resources, the system can dynamically reclaim some CPU and memory from these virtual machines with more idle resources and allocate them to the virtual machines corresponding to the businesses in need of resources. Exemplarily, if a certain virtual machine is currently using only 20% of the CPU and 30% of the memory, and the CPU and memory of another virtual machine requested by a business are in short supply, the system can allocate some CPU cores and memory of this idle virtual machine to the virtual machine in urgent need of resources.

[0099] Step 5002: When the storage requirement of the business request is insufficient, allocate a new storage volume or adjust the configuration of the existing storage volume.

[0100] A storage volume is a logical storage unit. The system will allocate a storage volume with an appropriate capacity according to the storage requirement size of the business. If the storage space required by the business request exceeds the scope of the current available storage resources, the system will allocate a new storage volume from the storage resource pool to this business. Exemplarily, for a video surveillance business that generates a large amount of video data and the original storage volume cannot meet its continuously growing storage requirement, the system will allocate a new storage volume from the storage resource pool for this business to store the newly generated video data.

[0101] For the storage volume that has been allocated to the business, if its capacity is insufficient, the system can perform an expansion operation on it, which means that the system will obtain additional storage space from the storage resource pool and add it to the existing storage volume to meet the continuously growing storage requirement of the business. Exemplarily, as the data volume of a database business continuously increases and the capacity of the original storage volume gradually becomes insufficient, the system can dynamically increase the capacity of this storage volume so that it can continue to run normally.

[0102] Step 5003: When the network bandwidth requested by the business is less than the business requirement, configure network bandwidth limits or optimize the network path.

[0103] In a hyper-converged system, there may be a situation where multiple businesses compete for network bandwidth simultaneously. When the network bandwidth requested by a certain business cannot meet its requirement, the system can limit the network bandwidth of other businesses to ensure that this business can obtain sufficient network bandwidth. Exemplarily, through a network traffic management tool, limit the network bandwidth of some non-critical businesses and allocate the saved bandwidth to the business with a higher requirement for network bandwidth, such as a real-time video conferencing business.

[0104] The system can also improve the network bandwidth of services by optimizing the network path. It analyzes the current network topology and traffic distribution to find a more efficient network transmission path. Exemplarily, when it is found that the data transmission of a certain service passes through multiple high-load network nodes, resulting in network latency and bandwidth limitation, the system will re-plan the network path to bypass these high-load nodes and select a smoother path for transmission, thereby improving the network bandwidth and transmission efficiency of the service.

[0105] Optionally, a hardware resource management method based on a hyper-converged storage system provided by an embodiment of the present application may further include the following steps:

[0106] Step 6001, obtain the load data of the service.

[0107] Among them, the load data can reflect the degree of use and demand for system resources during the operation of the service. The hyper-converged storage system continuously monitors and collects data on the load situation of the service. Exemplarily, for an online e-commerce platform, the service load data may include the number of order processing per second, page access volume, and data query frequency, etc. For a computing task in a data center, the load data may be the CPU usage rate, memory occupancy rate, and the busy degree of disk I / O operations, etc. The system obtains this load data through various monitoring tools and sensors to provide a basis for subsequent resource management.

[0108] Step 6002, when the load data increases and a resource shortage situation is recognized, increase the computing resource pool.

[0109] When the system monitors that the load data of the service is continuously increasing and, after analysis, determines that the current resource pool cannot meet the service requirements, that is, a resource shortage situation occurs, the system will take corresponding measures to increase the computing resource pool. This may involve multiple methods. One is to add new physical server nodes in the hyper-converged system, and the CPU and memory resources of these new nodes will be incorporated into the computing resource pool, thereby expanding the total amount of computing resources. The other is to create new virtual machine instances on the existing servers through virtualization technology and add their computing resources to the computing resource pool. Exemplarily, during the promotion period of an e-commerce platform, the order volume and access volume increase significantly, and the system detects that the CPU and memory usage rates are close to or reach the upper limit. At this time, the computing resource pool will be increased to ensure the stable operation of the service.

[0110] Step 6003, when the load data recovers or decreases and a redundant resource situation is recognized, release the computing resource pool.

[0111] When the load data of the service returns to the normal level or shows a downward trend, and the system analyzes and finds that there are redundant and underutilized resources in the computing resources, resource release operations will be performed; releasing the computing resource pool can reduce the energy consumption and cost of the system and improve the utilization efficiency of resources; the release methods can include shutting down some virtual machine instances that are no longer needed and returning the CPU and memory resources occupied by these virtual machines to the system; or putting some physical server nodes into a dormant or shutdown state and removing them from the computing resource pool; Exemplarily, after an e-commerce promotion event ends and the order volume and access volume drop significantly, and the system detects that the computing resources are idle, it will release some redundant computing resources.

[0112] Optionally, a hardware resource management method based on a hyper-converged storage system provided by an embodiment of the present application may further include the following steps:

[0113] Step 7001, monitor the usage of the allocated resources and obtain the data of the allocated resources.

[0114] The hyper-converged storage system continuously monitors the usage of various resources (such as computing resources, storage resources, and network resources) allocated to the service; for computing resources, it monitors the CPU usage rate, memory occupancy rate, and thread running status, etc.; for storage resources, it pays attention to the disk read and write rates, I / O operation frequencies, and storage capacity usage, etc.; for network resources, it tracks the network bandwidth occupancy, packet transmission delay, and packet loss rate, etc.; through the real-time monitoring of these metrics, the system can timely understand whether the usage status of the resources is normal; during the monitoring process, the system collects detailed data of the allocated resources, including the configuration information of the resources (such as the number of CPU cores, memory size, and storage volume capacity, etc.), usage records (such as the CPU usage rate change curve in the past period of time, time series data of storage read and write operations, etc.), and current real-time status data, which provide a basis for subsequent fault judgment and handling.

[0115] Step 7002, when a fault occurs in the usage situation, perform real-time backup and recovery of the data.

[0116] The system analyzes and judges the monitored resource usage situation according to preset rules and thresholds. When some metrics exceed the normal range, such as the CPU usage rate being continuously too high, frequent disk read and write errors, and sudden network bandwidth drop, etc., the system will identify that a resource usage fault has occurred.

[0117] Once a fault is detected, the system will immediately activate the real-time backup mechanism. For the data in the storage resources, important data will be quickly copied to other secure storage locations, such as backup storage devices, off-site data centers, etc.; the backup process will be completed as quickly as possible to reduce the risk of data loss; exemplarily, an incremental backup method is adopted, which only backs up the data that has changed since the last backup, improving the backup efficiency.

[0118] After the data backup is completed, the system will attempt to perform a data recovery operation; depending on the type and severity of the fault, the recovery method may vary. For some minor faults, it may only be necessary to simply reset the resources or adjust the configuration to restore normal access to the data; for more serious faults, such as storage device damage, it may be necessary to restore the backup data to a new storage device and reconfigure the relevant applications and services to ensure that the business can resume normal operation as soon as possible.

[0119] In the above-mentioned hardware resource management method based on a hyper-converged storage system, the hyper-converged storage system virtualizes and integrates computing, storage, and network hardware resources into a unified resource pool, improving resource utilization; during the business request processing, the system will query the current status of the resource pool to ensure that resources are allocated to the business when the resources are sufficient and in good condition, guaranteeing the stable operation of the business; since the system can quickly query the resource pool status and predict the parameters required by the business, after receiving a business request, it can quickly allocate resources to the business, reducing the waiting time of the business and enhancing the response speed of the business; resource allocation is carried out based on a preset resource allocation scheme, making the resource allocation process automated and standardized, reducing the possibility of manual intervention and human errors, and improving the management efficiency; when a resource node fails, through the dynamic allocation function of the resource pool, the business can be migrated to other normal nodes, and at the same time, the faulty node can be repaired or replaced without affecting the normal operation of the business, enhancing the maintainability of the hyper-converged storage system.

[0120] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same moment, but can be executed at different moments, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0121] Based on the same inventive concept, an embodiment of the present application further provides a hardware resource management device based on a hyper-converged storage system for implementing the above-mentioned hardware resource management method based on a hyper-converged storage system. The implementation solutions provided by this device to solve problems are similar to those recorded in the above method. Therefore, the specific limitations in one or more embodiments of the hardware resource management device based on a hyper-converged storage system provided below can refer to the limitations on the hardware resource management method based on a hyper-converged storage system in the above text, and will not be elaborated here.

[0122] In an exemplary embodiment, as Figure 3 shown, a hardware resource management device 300 based on a hyper-converged storage system is provided, including:

[0123] A resource data acquisition module 301, configured to acquire resource data in the hyper-converged storage system, virtualize the resource data, and integrate it into a unified resource pool;

[0124] A parameter prediction module 302, configured to query the current status of the resource pool when a service request is acquired, analyze the service request, and predict the parameter information required for the service;

[0125] A resource allocation module 303, configured to allocate the resources required for the service request based on the parameter information according to a preset resource allocation scheme.

[0126] Further, the parameter prediction module 302 is further configured to:

[0127] After receiving a service request, identify the type of the service request; the types include high-computation tasks and low-load tasks;

[0128] Query the usage of the current resource pool in real time; the usage includes resource occupancy and available capacity;

[0129] Parse the content of the service request to obtain the resource requirements of the service request; the requirements include emergency processing, specific resource requirements, and data scale;

[0130] Predict the parameter information required for the service request based on the resource requirements.

[0131] Further, the parameter prediction module 302 is further configured to:

[0132] Use a convolutional neural network model to predict the parameter information required for the service request;

[0133] Among them, the convolutional neural network model includes:

[0134] An input layer, configured to input the data of the service request; the feature vectors include request type, timestamp, and historical resource usage;

[0135] The convolutional layer is used to perform a convolution operation on the input data using a convolutional kernel. The convolution operation is carried out by sliding the convolutional kernel over the input data for element-wise multiplication and summation. The calculation formula for the convolution operation is as follows: Where, W ji is the weight of the convolutional kernel, b j is the bias term, k is the size of the convolutional kernel, C j is the output after the convolution operation, and X i is the input data. After the convolution operation, a non-linear activation function is applied to increase the non-linear expression ability of the model. After the activation function, a pooling operation is performed, and the pooling operation is used to reduce the dimension and computational amount of the data;

[0136] The output layer is used to output the predicted parameter information.

[0137] Furthermore, the resource data acquisition module 301 is also used for:

[0138] Aggregating the CPU and memory resources of all computing nodes into a computing resource pool, and aggregating the storage space of the storage nodes into a storage resource pool;

[0139] Based on the computing resource pool and the storage resource pool, record the initial capacity and performance parameters of each resource pool.

[0140] Furthermore, the resource allocation module 303 is also used for:

[0141] When the CPU and memory of the service request are insufficient, create a new virtual machine or adjust the resource configuration of the existing virtual machine;

[0142] When the storage requirement of the service request is insufficient, allocate a new storage volume or adjust the configuration of the existing storage volume;

[0143] When the network bandwidth of the service request is less than the service requirement, configure the network bandwidth limit or optimize the network path.

[0144] Furthermore, the device further includes:

[0145] The load data acquisition module is used to acquire the load data of the service;

[0146] The computing resource pool increase module is used to increase the computing resource pool when it is recognized that the resources are insufficient when the load data increases;

[0147] The computing resource pool release module is used to release the computing resource pool when it is recognized that there are redundant resources when the load data recovers or decreases.

[0148] Furthermore, the device further includes:

[0149] A usage monitoring module, configured to monitor the usage of allocated resources and obtain data of the allocated resources;

[0150] A backup and recovery module, configured to perform real-time backup and recovery of data when a failure occurs in the usage situation.

[0151] In one embodiment, as Figure 4 There is provided a computer device 400, including:

[0152] At least one processor 401;

[0153] And a memory 402 communicatively connected to at least one of the processors 401;

[0154] The memory stores application program code executable by at least one of the processors, and the application program code is executed by at least one of the processors, so that at least one of the processors can execute the steps of a hardware resource management method based on a hyper-converged storage system as described above.

[0155] The computer device may further include: a sensor 403.

[0156] The processor 401, the memory 402 and the sensor 403 may be connected through a bus 404 or other means. In the figure, taking the connection through the bus 404 as an example, Figure 4 Only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0157] In one embodiment, there is provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0158] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial descriptions of the method embodiments. The device embodiments described above are only illustrative. The components described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present disclosure solution. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0159] The above-described embodiments merely represent several implementation manners of the embodiments of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the embodiments of the application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the embodiments of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the embodiments of the present application.

Claims

1. A hardware resource management method based on a hyper-converged storage system, characterized in that The method includes: Obtaining resource data in a hyper-converged storage system, virtualizing the resource data, and integrating it into a unified resource pool; the resource data includes computing resources, storage resources, and network hardware resources; When a service request is obtained, query the current status of the resource pool, and analyze the service request to predict the parameter information required for the service; the current status includes the remaining capacity of each resource pool, the usage of the allocated resources, and the load information of each node; the parameter information includes computing power parameters, storage capacity parameters, and network bandwidth parameters; Based on the parameter information, allocate the resources required for the service request according to a preset resource allocation scheme.

2. The method according to claim 1, wherein The step of, when a service request is obtained, querying the current status of the resource pool and analyzing the service request to predict the parameter information required for the service includes: After receiving a service request, identify the type of the service request; the types include high-computation tasks and low-load tasks; Query the usage of the current resource pool in real time; the usage includes resource occupancy and available capacity; Parse the content of the service request to obtain the resource requirements of the service request; the requirements include emergency processing, specific resource requirements, and data scale; Based on the resource requirements, predict the parameter information required for the service request.

3. The method according to claim 2, wherein The step of, based on the resource requirements, predicting the parameter information required for the service request includes: Using a convolutional neural network model to predict the parameter information required for the service request; Wherein, the convolutional neural network model includes: An input layer for inputting the data of the service request; the feature vector includes request type, timestamp, and historical resource usage; Convolution layer, used to perform a convolution operation on the input data using a convolution kernel. The convolution operation is performed by sliding the convolution kernel over the input data for element-wise multiplication and summation. The calculation formula for the convolution operation is as follows: where, W ji is the weight of the convolution kernel, b j is the bias term, k is the size of the convolution kernel, C j is the output after the convolution operation, X i is the input data. After the convolution operation, a non-linear activation function is applied to increase the non-linear expression ability of the model. After the activation function, a pooling operation is performed. The pooling operation is used to reduce the dimension and computational amount of the data; An output layer for outputting the predicted parameter information.

4. The method according to claim 1, wherein The step of, in a hyper-converged storage system, obtaining resource data, virtualizing the resource data, and integrating it into a unified resource pool includes: Aggregating the CPU and memory resources of all computing nodes into a computing resource pool, and aggregating the storage spaces of storage nodes into a storage resource pool; Based on the computing resource pool and the storage resource pool, record the initial capacity and performance parameters of each resource pool.

5. The method according to claim 1, characterized in that, The step of, based on the service request, allocating the resources required for the service request according to a preset resource allocation scheme includes: When the CPU and memory of the service request are insufficient, create a new virtual machine or adjust the resource configuration of an existing virtual machine; When the storage requirement of the service request is insufficient, allocate a new storage volume or adjust the configuration of an existing storage volume; When the network bandwidth of the service request is less than the requirement of the service, configure network bandwidth limits or optimize the network path.

6. The method according to claim 1, wherein It further includes: Obtaining the load data of the service; When the load data increases and a resource shortage situation is recognized, increase the computing resource pool; When the load data recovers or decreases and a situation of redundant resources is recognized, release the computing resource pool.

7. The method according to claim 1, characterized in that It further includes: Monitoring the usage of the allocated resources and obtaining the data of the allocated resources; When a failure occurs in the usage situation, perform real-time backup and recovery of the data.

8. A hardware resource management device based on a hyper-converged storage system, characterized in that, The device includes: A resource data acquisition module, which is used to acquire resource data in a hyper-converged storage system, virtualize the resource data and integrate it into a unified resource pool; A parameter prediction module, which is used to query the current state of the resource pool and analyze the service request to predict the parameter information required by the service when a service request is acquired; A resource allocation module, which is used to allocate the resources required by the service request based on the parameter information according to a preset resource allocation scheme.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images