Dynamic adjustment method and system for storage resource pool

Abstract

The invention provides a dynamic adjustment method and system for a storage resource pool, and belongs to the technical field of resource management.The method comprises the steps that available computing nodes in a network are scanned, network verification is carried out, a hardware ready instruction is obtained, an initial resource pool is created based on the ready instruction, network configuration and storage configuration are carried out on the initial resource pool, and the initial resource pool is obtained. Performing key index monitoring of an initial resource pool based on the network configuration and the storage configuration, predicting a storage demand and a resource utilization rate according to the monitored key index, and making a storage optimization plan and a resource expansion plan according to the storage demand and the resource utilization rate, and a storage resource pool is dynamically adjusted based on the storage optimization plan and the resource expansion plan, so that intelligent management and dynamic optimization of resources are realized, the storage demand and the resource utilization rate are accurately predicted, the resource management efficiency is improved, human errors are reduced, the operation and maintenance efficiency is improved, and the overall performance is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of resource management, and in particular to a method and system for dynamically adjusting a storage resource pool. Background Art

[0002] With the widespread application of cloud computing and virtualization technologies, enterprises need to efficiently manage and optimize computing, storage, and network resources to reduce costs and improve performance. Traditional resource management methods are usually static and cannot dynamically adjust resource configuration according to actual needs. They rely on manual operation and maintenance, which is inefficient and prone to errors.

[0003] Therefore, the present invention provides a method and system for dynamically adjusting a storage resource pool. Summary of the Invention

[0004] The present invention provides a method and system for dynamically adjusting a storage resource pool. By scanning and verifying computing nodes, a resource pool is created and network and storage configuration is performed. Key indicators are monitored and storage demand and resource utilization are predicted. Storage optimization and expansion plans are dynamically adjusted to achieve intelligent management and dynamic optimization of resources, accurately predict storage demand and resource utilization, improve resource management efficiency, reduce human errors, improve operation and maintenance efficiency, and enhance overall performance.

[0005] The present invention provides a method for dynamically adjusting a storage resource pool, comprising: Step 1: Scan the available computing nodes in the network, perform network verification, and obtain hardware readiness instructions; Step 2: Create an initial resource pool based on the ready instruction, and perform network configuration and storage configuration on the initial resource pool; Step 3: Monitor key indicators of the initial resource pool based on the network configuration and storage configuration, and predict storage demand and resource utilization based on the monitored key indicators; Step 4: Formulate a storage optimization plan and a resource expansion plan according to the storage demand and resource utilization, and dynamically adjust the storage resource pool based on the storage optimization plan and the resource expansion plan.

[0006] The present invention provides a method for dynamically adjusting a storage resource pool, which scans available computing nodes in a network, performs network verification, and obtains a hardware readiness instruction, including: Defining the IP address range where the expected computing nodes are located, obtaining scanning requirements based on the IP address range, and selecting a scanning method based on the scanning requirements; The scanning method is used to determine computing nodes, and network verification is performed on the computing nodes. According to preset thresholds of latency and packet loss, the network-verified available computing nodes are determined to form a cluster and output a hardware ready instruction.

[0007] The present invention provides a method for dynamically adjusting a storage resource pool, which creates an initial resource pool based on a ready instruction and performs network configuration and storage configuration on the initial resource pool, including: Based on the cluster in the ready instruction, select a host for creating an initial resource pool, determine the type of resource pool to be created based on actual needs, and create the initial resource pool based on the cluster, host, and resource pool type; Obtaining business requirements, performing a scale analysis on the cluster, and performing a first configuration on the initial resource pool based on the business requirements and the cluster scale; Determining a network topology according to the distribution of the initial resource pool, and performing a second configuration on the initial resource pool in combination with the network topology and hardware resources; Determine a network configuration by combining the first configuration and the second configuration; Based on the cluster, divide the initial resource pool into LUNs, determine the heartbeat disk, data disk, and storage redundancy, and perform storage configuration.

[0008] The present invention provides a method for dynamically adjusting a storage resource pool, which divides the initial resource pool into LUNs based on a cluster, determines heartbeat disks, data disks, and storage redundancy, and performs storage configuration, including: ,in, represents the combined predicted storage value of the cluster at time t; Represents a time series model consisting of clusters; represents the weight of the time series model composed of clusters; c represents the constant term; p represents the order of the autoregressive model; i1 represents the index variable of the order; Indicates the impact of past i1 observations on the current value; represents the storage requirement at time point t-i1; represents white noise; q represents the moving average order; Represents the influence coefficient of the past i1 error terms on the current value; represents the white noise at time point t-i1; represents a regression model consisting of clusters; represents the weight of the regression model composed of clusters; Represents the intercept term of the regression model; i2 represents the index variable of the external variable; Represents a machine learning model composed of clusters; i3 represents the index variable of the decision tree; represents the influence coefficient of the i2th external variable on storage demand; Indicates time External variables; represents the weight of the machine learning model composed of clusters, where ; N represents the total number of decision trees; Represents the feature vector of the i3th decision tree The prediction results; LUNs are divided based on the comprehensive predicted storage value, heartbeat disks, data disks, and storage redundancy are determined, and the divided LUNs are mapped to nodes in the cluster to determine the storage configuration.

[0009] The present invention provides a method for dynamically adjusting a storage resource pool, which monitors key indicators of an initial resource pool based on the network configuration and storage configuration, and predicts storage demand and resource utilization based on the monitored key indicators, including: Determine network configuration related indicators based on network configuration, determine storage configuration related indicators based on storage configuration, and simultaneously obtain hardware configuration related indicators corresponding to hardware configuration, and derive key indicators by combining the network configuration related indicators, storage configuration related indicators, and hardware configuration related indicators; Monitoring the key indicators and determining storage requirements based on the monitored key indicators and the comprehensive predicted storage value; Perform load change analysis on the business demand, and determine resource utilization by combining the load change analysis results with key monitoring indicators.

[0010] The present invention provides a method for dynamically adjusting a storage resource pool, which formulates a storage optimization plan and a resource expansion plan according to the storage demand and resource utilization, and dynamically adjusts the storage resource pool based on the storage optimization plan and the resource expansion plan, including: Based on storage needs, we clean up useless data in the storage resource pool, adjust LUN division, optimize storage performance, and use data compression and deduplication technologies to reduce storage space usage, thereby developing a storage optimization plan. Decomposing and analyzing the storage requirements and resource utilization to derive storage capacity requirements, and determining to add storage devices and expand storage capacity based on the storage capacity requirements. Determining CPU and memory utilization based on the decomposition and analysis results, adding computing nodes, optimizing the cluster, and ultimately deriving a resource expansion plan. The storage resource pool is dynamically adjusted according to the storage optimization plan and resource expansion plan.

[0011] The present invention provides a method for dynamically adjusting a storage resource pool, which dynamically adjusts the storage resource pool according to the storage optimization plan and the resource expansion plan, comprising: Dynamically adjust storage optimization plans and resource expansion plans based on storage demand and resource utilization. If the predicted storage demand increases, expand storage capacity; if the predicted resource utilization decreases, reduce the scale of resource expansion. Dynamically adjust storage optimization plans and resource expansion plans based on actual usage. If the actual storage demand is lower than the predicted storage demand, reduce storage optimization efforts. If the actual resource utilization is higher than the predicted resource utilization, increase the scale of resource expansion.

[0012] The present invention provides a storage resource pool dynamic adjustment system, comprising: Verification module: scans the available computing nodes in the network, performs network verification, and obtains hardware readiness instructions; Configuration module: creates an initial resource pool based on the ready instruction, and performs network configuration and storage configuration on the initial resource pool; Prediction module: monitors key indicators of the initial resource pool based on the network configuration and storage configuration, and predicts storage demand and resource utilization based on the monitored key indicators; Adjustment module: formulates a storage optimization plan and a resource expansion plan according to the storage demand and resource utilization, and dynamically adjusts the storage resource pool based on the storage optimization plan and the resource expansion plan.

[0013] Compared with the existing technology, the beneficial effects of this application are as follows: by scanning and verifying computing nodes, creating resource pools and performing network and storage configuration, monitoring key indicators and predicting storage demand and resource utilization, dynamically adjusting storage optimization and expansion plans, realizing intelligent management and dynamic optimization of resources, accurately predicting storage demand and resource utilization, improving resource management efficiency, reducing human errors, improving operation and maintenance efficiency, and improving overall performance.

[0014] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0015] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 This is a structural diagram of a method for dynamically adjusting a storage resource pool provided by an embodiment of the present invention; Figure 2 This is a flow chart of a storage resource pool dynamic adjustment system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0018] Example 1: The embodiment of the present invention provides a method for dynamically adjusting a storage resource pool. Figure 1 Shown, including: Step 1: Scan the available computing nodes in the network, perform network verification, and obtain hardware readiness instructions; Step 2: Create an initial resource pool based on the ready instruction, and perform network configuration and storage configuration on the initial resource pool; Step 3: Monitor key indicators of the initial resource pool based on the network configuration and storage configuration, and predict storage demand and resource utilization based on the monitored key indicators; Step 4: Formulate a storage optimization plan and a resource expansion plan according to the storage demand and resource utilization, and dynamically adjust the storage resource pool based on the storage optimization plan and the resource expansion plan.

[0019] In this embodiment, the available computing nodes are computing nodes whose latency and packet loss conditions meet preset thresholds in network verification. For example, the node with IP address 192.168.1.10 has a latency of 15ms and a packet loss rate of 0%, and has passed verification.

[0020] In this embodiment, network verification is to test the network connection of the computing node to ensure that it meets the cluster requirements, for example, using the ping command to test the network connectivity of the computing node.

[0021] In this embodiment, the ready instruction is an instruction indicating that the hardware infrastructure is ready and can be used to create a resource pool. For example, outputting the instruction HardwareReady indicates that the cluster is ready.

[0022] In this embodiment, the hardware is a physical device, including computing nodes, storage devices, and network devices, such as servers, hard disks, switches, and the like.

[0023] In this embodiment, the network configuration is the network setting of the resource pool, including IP address, subnet mask, gateway, etc. For example, the IP address range of the resource pool is configured to be 192.168.1.10 to 192.168.1.19, the subnet mask is 255.255.255.0, and the gateway is 192.168.1.1.

[0024] In this embodiment, the initial resource pool is a resource collection composed of selected hosts, which is used to provide computing, storage and network resources. For example, a resource pool composed of 10 servers provides CPU, memory and storage resources.

[0025] In this embodiment, the storage requirements of the cluster are comprehensively predicted through time series models, regression models and machine learning models, LUN division is performed based on the predicted values, heartbeat disks, data disks and storage redundancy are determined, and LUNs are mapped to cluster nodes to complete storage configuration.

[0026] In this embodiment, storage configuration is the configuration of storage resources, including LUN division, heartbeat disk, data disk and storage redundancy. For example, 10 LUNs are configured, each LUN size is 10TB, and RAID 1 configuration is used to ensure data redundancy.

[0027] In this embodiment, the key indicators are comprehensive indicators extracted from network configuration, storage configuration and hardware configuration related indicators, which are used to comprehensively evaluate the performance and health of the system, such as system availability: the proportion of system normal operation time, such as 99.9%; resource utilization: the comprehensive utilization rate of resources such as CPU, memory, storage, etc., such as 80%; performance bottleneck: the key point in the system that limits performance, such as insufficient network bandwidth.

[0028] In this embodiment, storage demand refers to the storage resource demand determined based on business needs, predicted storage values, and system monitoring data, including requirements in terms of capacity, performance, and reliability. For example, capacity demand: 50TB of storage space is required in the next 6 months; performance demand: storage performance needs to support 20,000 IOPS; reliability demand: 99.99% availability is required.

[0029] In this embodiment, resource utilization refers to the utilization ratio of various resources (such as CPU, memory, storage, network, etc.) in the system, which is used to evaluate the resource utilization efficiency and whether there are bottlenecks. For example, CPU utilization: the current CPU utilization is 70%, memory utilization: the current memory utilization is 60%, storage utilization: the current storage utilization is 80%, and network utilization: the current network bandwidth utilization is 50%.

[0030] In this embodiment, the storage optimization plan refers to a plan to improve storage resource utilization by cleaning up useless data, adjusting LUN partitioning, optimizing storage performance, using data compression and deduplication technology, etc. For example, cleaning up useless data: deleting log files from 3 years ago to free up 10 TB of space; adjusting LUN partitioning: reallocating unused 5 TB of LUN space to high-load business systems; using data compression and deduplication: reducing storage space usage by 30% through compression and deduplication technology.

[0031] In this embodiment, the resource expansion plan refers to a plan to determine the addition of storage devices, computing nodes and other resources based on storage capacity requirements and resource utilization analysis. For example, to add storage devices: deploy a new storage array to provide 200TB of storage capacity; to add computing nodes: add 5 servers to improve the computing power of the cluster; to optimize the cluster: use load balancing technology to distribute high-load services to the newly added computing nodes.

[0032] In this embodiment, dynamic adjustment refers to adjusting the storage resource pool in real time or periodically according to the storage optimization plan and resource expansion plan to cope with changes in business needs. For example, according to changes in business load, the allocation of LUNs is dynamically adjusted, and during high load periods, the resource allocation of computing nodes is automatically increased.

[0033] In this embodiment, by cleaning up useless data, adjusting LUN division, using data compression and deduplication to optimize storage performance, combining storage demand and resource utilization analysis, a storage optimization and resource expansion plan is formulated, and the storage resource pool configuration is dynamically adjusted.

[0034] In this embodiment, the storage optimization plan and the resource expansion plan are dynamically adjusted. Dynamic adjustment refers to flexibly adjusting the storage optimization plan and the resource expansion plan based on real-time or predicted storage demand and resource utilization to ensure efficient resource utilization and stable operation of the system. The corresponding process is to monitor storage demand and resource utilization in real time, predict future storage demand and resource utilization based on historical data and trends, dynamically adjust the storage optimization plan and the resource expansion plan based on the prediction results, implement the adjusted plan, and continuously monitor the effect.

[0035] The working principle and beneficial effects of the above technical solution are: by scanning and verifying computing nodes, creating resource pools and performing network and storage configuration, monitoring key indicators and predicting storage demand and resource utilization, dynamically adjusting storage optimization and expansion plans, realizing intelligent management and dynamic optimization of resources, accurately predicting storage demand and resource utilization, improving resource management efficiency, reducing human errors, improving operation and maintenance efficiency, and enhancing overall performance.

[0036] Example 2: An embodiment of the present invention provides a method for dynamically adjusting a storage resource pool, which scans available computing nodes in a network, performs network verification, and obtains a hardware readiness instruction, including: Defining the IP address range where the expected computing nodes are located, obtaining scanning requirements based on the IP address range, and selecting a scanning method based on the scanning requirements; The scanning method is used to determine computing nodes, and network verification is performed on the computing nodes. According to preset thresholds of latency and packet loss, the network-verified available computing nodes are determined to form a cluster and output a hardware ready instruction.

[0037] In this embodiment, the IP address range is the IP address interval of the expected computing nodes, for example, 192.168.1.1 to 192.168.1.100; the expected computing nodes are the computing nodes planned to form a cluster, for example, 10 servers, with IP addresses of 192.168.1.10 to 192.168.1.19.

[0038] In this embodiment, the scanning requirement is based on the IP address range to determine the computing nodes that need to be scanned and their network status. For example, all IP addresses in the range of 192.168.1.1 to 192.168.1.100 are scanned to find available computing nodes.

[0039] In this embodiment, the scanning method is a technology or tool for scanning and identifying computing nodes, for example, using the nmap tool to scan an IP address range and identify online nodes.

[0040] In this embodiment, latency is the transmission time of a data packet from a source node to a destination node, and packet loss is the proportion of data packets lost during the transmission process, for example, latency: 10ms, packet loss: 0%.

[0041] In this embodiment, the preset threshold is the acceptable upper limit of latency and packet loss in network verification, for example, the latency threshold is 20ms, and the packet loss threshold is 1%.

[0042] In this embodiment, a cluster is a collection of computing nodes that have passed network authentication and is used to provide computing and storage resources, for example, a cluster consisting of 10 servers from 192.168.1.10 to 192.168.1.19.

[0043] The working principle and beneficial effects of the above technical solution are: by defining the IP address range of computing nodes, selecting the scanning method and performing network verification, screening qualified nodes according to preset thresholds of delay and packet loss, forming a cluster and outputting ready instructions, ensuring hardware availability, reducing manual operations, improving efficiency, and enhancing the efficiency and reliability of cluster deployment.

[0044] Example 3: An embodiment of the present invention provides a method for dynamically adjusting a storage resource pool, which creates an initial resource pool based on a ready instruction and performs network configuration and storage configuration on the initial resource pool, including: Based on the cluster in the ready instruction, select a host for creating an initial resource pool, determine the type of resource pool to be created based on actual needs, and create the initial resource pool based on the cluster, host, and resource pool type; Obtaining business requirements, performing a scale analysis on the cluster, and performing a first configuration on the initial resource pool based on the business requirements and the cluster scale; Determining a network topology according to the distribution of the initial resource pool, and performing a second configuration on the initial resource pool in combination with the network topology and hardware resources; Determine a network configuration by combining the first configuration and the second configuration; Based on the cluster, divide the initial resource pool into LUNs, determine the heartbeat disk, data disk, and storage redundancy, and perform storage configuration.

[0045] In this embodiment, the host is a computing node in the cluster, used to run virtual machines and provide computing resources, for example, a server with an IP address of 192.168.1.10.

[0046] In this embodiment, the actual demand is the resource demand determined according to business needs and technical requirements, for example, 100TB of storage capacity and 10,000 IOPS of storage performance are required.

[0047] In this embodiment, the resource pool type is a classification of the resource pool, which is determined according to usage and performance requirements, such as a high-performance resource pool (using SSD) and a large-capacity resource pool (using HDD).

[0048] In this embodiment, the business demand is the business demand for resources, including storage capacity, computing performance, and network bandwidth. For example, an online service that supports 1,000 concurrent users requires high IOPS and low latency.

[0049] In this embodiment, the scale analysis is to analyze the scale of the cluster to determine the size and configuration of the resource pool, for example, analyzing the number of hosts, the number of CPU cores, the memory capacity, and the storage capacity in the cluster.

[0050] In this embodiment, the first configuration is a preliminary configuration of the initial resource pool based on business requirements and cluster size, for example, configuring the CPU and memory resources of the resource pool to ensure that business requirements are met.

[0051] In this embodiment, the second configuration is a further configuration of the initial resource pool based on the network topology and hardware resources, for example, configuring the network bandwidth and storage performance of the resource pool to ensure that business needs are met.

[0052] In this embodiment, the distribution status refers to the distribution status of hosts and resources in the initial resource pool. For example, the hosts in the resource pool are distributed on different racks to ensure high availability.

[0053] In this embodiment, the network topology refers to the connection mode between hosts and network devices in the resource pool, for example, a star topology where all hosts are connected via switches.

[0054] In this embodiment, hardware resources are physical devices in a resource pool, including computing nodes, storage devices, and network devices, such as servers, hard disks, switches, and the like.

[0055] In this embodiment, LUN division is to divide the storage device into logical units for storing data. For example, the storage device is divided into 10 LUNs, and the size of each LUN is 10 TB.

[0056] In this embodiment, the heartbeat disk is a storage area used to monitor the status of cluster nodes in real time. For example, each node is allocated a 5GB heartbeat disk for monitoring the node status; the data disk is a storage area used to store business data. For example, each node is allocated a 100TB data disk for storing business data; storage redundancy is to improve data reliability and availability through redundant configuration, for example, using RAID 1 configuration to ensure high data availability.

[0057] The working principle and beneficial effects of the above technical solution are: by selecting the hosts in the cluster to create an initial resource pool, the first configuration is performed in combination with business needs and cluster size, the second configuration is performed according to the network topology and hardware resources, the network configuration is comprehensively obtained, and the storage configuration is completed through LUN division and storage redundancy, which improves the flexibility of resource management and the high availability of the system to meet diverse needs.

[0058] Example 4: An embodiment of the present invention provides a method for dynamically adjusting a storage resource pool. The method divides an initial resource pool into LUNs based on a cluster, determines heartbeat disks, data disks, and storage redundancy, and performs storage configuration, including: ,in, represents the combined predicted storage value of the cluster at time t; Represents a time series model consisting of clusters; represents the weight of the time series model composed of clusters; c represents the constant term; p represents the order of the autoregressive model; i1 represents the index variable of the order; Indicates the impact of past i1 observations on the current value; represents the storage requirement at time point t-i1; represents white noise; q represents the moving average order; Represents the influence coefficient of the past i1 error terms on the current value; represents the white noise at time point t-i1; represents a regression model consisting of clusters; represents the weight of the regression model composed of clusters; Represents the intercept term of the regression model; i2 represents the index variable of the external variable; Represents a machine learning model composed of clusters; i3 represents the index variable of the decision tree; represents the influence coefficient of the i2th external variable on storage demand; Indicates time External variables; represents the weight of the machine learning model composed of clusters, where ; N represents the total number of decision trees; Represents the feature vector of the i3th decision tree The prediction results; LUNs are divided based on the comprehensive predicted storage value, heartbeat disks, data disks, and storage redundancy are determined, and the divided LUNs are mapped to nodes in the cluster to determine the storage configuration.

[0059] In this embodiment, a time series model is a statistical model used to analyze and predict time series data. Time series data consists of data points arranged in chronological order and typically exhibits temporal dependence. For example, ARIMA (Autoregressive Integrated Moving Average) is a common time series model used to predict future data points. For example, predicting stock prices, temperature, or electricity demand for a specific day in the future.

[0060] In this embodiment, the autoregressive model is a type of time series model, which assumes that the current value can be expressed as a linear combination of values ​​at several past time points.

[0061] In this embodiment, a regression model is a statistical model used to analyze the relationship between a dependent variable and one or more independent variables. It can be used to predict or explain the relationship between variables. For example, a linear regression model can be used to predict housing prices, where housing price is the dependent variable and house size, location, age, etc. are independent variables.

[0062] In this embodiment, a machine learning model uses an algorithm to learn patterns from data and use them for prediction or classification. Common machine learning models include decision trees, random forests, and support vector machines. For example, a random forest model can be used to predict customer churn based on characteristics such as customer spending behavior and demographics.

[0063] In this embodiment, external variables refer to factors outside the model that may affect the forecast results. These variables are typically not part of the time series, but can be used to enhance the model's forecasting capabilities. For example, when forecasting electricity demand, weather temperature, holiday information, etc. can be considered external variables because they affect electricity consumption.

[0064] In this embodiment, the prediction result is the future value or classification result calculated by the model based on the input data and model parameters. For example, in sales forecasting, the prediction result may be the sales volume for a certain month in the future. In classification problems, the prediction result may be whether a customer will purchase a product.

[0065] In this embodiment, a decision tree is a machine learning model that makes decisions using a tree-like structure. Each internal node represents a feature test, each branch represents the test result, and each leaf node represents a category or regression value. For example, in a customer classification problem, a decision tree can predict whether a customer will purchase a product based on characteristics such as age, income, and spending habits.

[0066] In this embodiment, white noise refers to a random error term with zero mean, finite variance, and no correlation. In time series models, white noise typically represents random fluctuations that cannot be explained by the model. For example, in stock price forecasting, white noise may represent random fluctuations in the market that cannot be predicted by the model.

[0067] In this embodiment, a LUN is a logical storage unit in a storage area network. LUN partitioning refers to dividing physical storage resources into multiple logical units for allocation to different servers or applications. For example, in a storage cluster, a physical disk can be divided into multiple LUNs, one for heartbeat disks, one for data disks, and one for storage redundancy.

[0068] In this embodiment, the heartbeat disk is a disk used for communication between cluster nodes and is used to detect node liveness. If a node cannot access the heartbeat disk, the cluster will consider the node failed and perform a failover. For example, in a high-availability cluster, the heartbeat disk is used to ensure communication between the primary node and the backup node, so that a rapid failover to the backup node is possible in the event of a primary node failure.

[0069] In this embodiment, the data disk is a disk used to store actual data. It is usually used to store application data, databases, files, etc. For example, in a database cluster, the data disk is used to store database tables, indexes, and other data.

[0070] In this embodiment, storage redundancy refers to improving data reliability and availability by duplicating data or using redundant storage technologies (such as RAID). Redundant storage can prevent data loss in the event of a disk failure. For example, using RAID 1 (mirroring) technology, data is written to two disks simultaneously to ensure data availability even if one disk fails.

[0071] The working principle and beneficial effects of the above technical solution are: comprehensively predicting the storage needs of the cluster through time series models, regression models and machine learning models, dividing LUNs based on the predicted values, determining heartbeat disks, data disks and storage redundancy, and mapping LUNs to cluster nodes to complete storage configuration, thereby improving resource utilization and system reliability and enhancing overall system performance.

[0072] Example 5: An embodiment of the present invention provides a method for dynamically adjusting a storage resource pool, which monitors key indicators of an initial resource pool based on the network configuration and storage configuration, and predicts storage demand and resource utilization based on the monitored key indicators, including: Determine network configuration related indicators based on network configuration, determine storage configuration related indicators based on storage configuration, and simultaneously obtain hardware configuration related indicators corresponding to hardware configuration, and derive key indicators by combining the network configuration related indicators, storage configuration related indicators, and hardware configuration related indicators; Monitoring the key indicators and determining storage requirements based on the monitored key indicators and the comprehensive predicted storage value; Perform load change analysis on the business demand, and determine resource utilization by combining the load change analysis results with key monitoring indicators.

[0073] In this embodiment, network configuration-related indicators refer to performance or status parameters related to network configuration, which are used to evaluate the operating status and performance of the network, such as bandwidth: the ability of the network to transmit data, such as 1 Gbps, latency: the time required for data to travel from the source to the destination, such as 10 ms, packet loss rate: the proportion of data packets lost during network transmission, such as 0.1%, throughput: the amount of data successfully transmitted per unit time, such as 500 Mbps, and network topology: the structure of the network, such as star topology, ring topology, etc.

[0074] In this embodiment, the storage configuration-related indicators refer to performance or status parameters related to the storage system configuration, which are used to evaluate the capacity, performance and reliability of the storage system, such as the total capacity of the storage system, such as 100 TB, and IOPS: the number of input / output operations per second, such as 10,000 IOPS.

[0075] In this embodiment, hardware configuration-related indicators refer to performance or status parameters related to hardware such as servers and storage devices, which are used to evaluate the performance and health of the hardware. For example, the CPU usage ratio is 70%, the memory capacity is the memory size of the server, such as 64 GB, and the disk health status is the SMART status of the disk, such as healthy, warning, or faulty.

[0076] In this embodiment, the load change analysis result refers to the conclusion drawn after analyzing the change trend, peak value and valley value of the business load, which is used to predict future resource requirements. For example, the load trend is: the business load will increase by 30% during holidays, the peak time is: 14:00-16:00 every day is the peak load period, and the valley time is: 02:00-04:00 every day is the valley load period.

[0077] The working principle and beneficial effects of the above technical solution are: by integrating key indicators of network, storage and hardware configuration, monitoring and analyzing storage demand and resource utilization, dynamically adjusting resource configuration based on load changes and predicted values, achieving efficient resource management and optimization, improving overall system performance, and enhancing system flexibility and performance.

[0078] Example 6: An embodiment of the present invention provides a method for dynamically adjusting a storage resource pool, which formulates a storage optimization plan and a resource expansion plan according to storage demand and resource utilization, and dynamically adjusts the storage resource pool based on the storage optimization plan and the resource expansion plan, including: Based on storage needs, we clean up useless data in the storage resource pool, adjust LUN division, optimize storage performance, and use data compression and deduplication technologies to reduce storage space usage, thereby developing a storage optimization plan. Decomposing and analyzing the storage requirements and resource utilization to derive storage capacity requirements, and determining to add storage devices and expand storage capacity based on the storage capacity requirements. Determining CPU and memory utilization based on the decomposition and analysis results, adding computing nodes, optimizing the cluster, and ultimately deriving a resource expansion plan. The storage resource pool is dynamically adjusted according to the storage optimization plan and resource expansion plan.

[0079] In this embodiment, useless data refers to data in the storage resource pool that is no longer accessed, expired, or invalid. This data takes up storage space but has no business value. Examples include expired log files (e.g., access logs from three years ago), completed temporary backup files, and unused test data or development environment data.

[0080] In this embodiment, a LUN is a logical storage unit in a storage system. Adjusting LUN partitioning refers to reallocating or partitioning LUNs based on storage requirements to optimize storage resource utilization. For example, a large-capacity LUN can be partitioned into multiple smaller LUNs, each for different business systems, and unused LUN space can be reallocated to a high-load business system.

[0081] In this embodiment, storage performance refers to the efficiency of the storage system in processing data read and write requests, and is usually measured by indicators such as IOPS, throughput, and response time. For example, improving IOPS: By replacing HDDs with SSDs, IOPS can be increased from 5,000 to 50,000. Reducing response time: By optimizing the storage array's cache strategy, the response time can be reduced from 10 ms to 2 ms.

[0082] In this embodiment, data compression: reduces the storage space occupied by data through algorithms, data deduplication: reduces the storage space occupied by data by eliminating duplicate data, for example, data compression: compresses 1 TB of text data into 200 GB, data deduplication: in the backup system, removes duplicate file blocks and reduces 10 TB of backup data to 2 TB.

[0083] In this embodiment, decomposition analysis refers to breaking down storage requirements and resource utilization into more fine-grained components to more accurately determine resource requirements. For example, the total storage requirement is broken down into the requirements of different business systems: the database system requires 50 TB, and the file storage system requires 30 TB. The CPU utilization is broken down into the utilization of different applications: application A occupies 30% and application B occupies 20%.

[0084] In this embodiment, storage capacity demand refers to future storage space demand derived from business demand and resource utilization analysis. For example, 100 TB of storage capacity needs to be added in the next 6 months, with the database system requiring 50 TB, the file storage system requiring 30 TB, and the backup system requiring 20 TB.

[0085] In this embodiment, storage devices refer to hardware devices used to store data, including hard disks, SSDs, storage arrays, etc. For example, adding 10 10 TB hard disks and deploying a new storage array can provide 200 TB of storage capacity.

[0086] The working principle and beneficial effects of the above technical solution are: by cleaning up useless data, adjusting LUN division, using data compression and deduplication to optimize storage performance, combining storage demand and resource utilization analysis, formulating storage optimization and resource expansion plans, dynamically adjusting storage resource pool configuration, improving storage performance, optimizing cluster performance, and improving system performance and resource utilization.

[0087] Example 7: An embodiment of the present invention provides a method for dynamically adjusting a storage resource pool, which dynamically adjusts the storage resource pool according to the storage optimization plan and the resource expansion plan, including: Dynamically adjust storage optimization plans and resource expansion plans based on storage demand and resource utilization. If the predicted storage demand increases, expand storage capacity; if the predicted resource utilization decreases, reduce the scale of resource expansion. Dynamically adjust storage optimization plans and resource expansion plans based on actual usage. If the actual storage demand is lower than the predicted storage demand, reduce storage optimization efforts. If the actual resource utilization is higher than the predicted resource utilization, increase the scale of resource expansion.

[0088] In this embodiment, the storage capacity is increased and expanded according to the predicted storage demand. If the predicted storage demand increases, it means that more storage space will be needed in the future, so the storage capacity needs to be expanded to meet the demand. Forecasting storage demand: predicting future storage demand through a time series model or a machine learning model. Evaluating current capacity: evaluating whether the capacity of the current storage resource pool can meet the predicted demand. Expanding storage capacity: if the current capacity is insufficient, adding storage devices (such as hard disks, storage arrays) or optimizing existing storage resources (such as adjusting LUN division, using data compression and deduplication technology). For example, it is predicted that the storage demand will increase by 50 TB in the next 6 months, so a new storage array is deployed to provide 100 TB of storage capacity.

[0089] In this embodiment, the scale of resource expansion is reduced according to the predicted decrease in resource utilization. If the predicted resource utilization decreases, it means that the demand for computing resources in the future will decrease. Therefore, the scale of resource expansion can be reduced to avoid resource waste. Forecast resource utilization: predict future resource utilization through historical data and trends. Evaluate current resources: evaluate the usage of current computing resources. Reduce the scale of resource expansion: if the predicted resource utilization is low, reduce the planned increase in computing nodes or storage devices. For example, it is predicted that the CPU utilization will decrease by 20% in the next 6 months, so the number of planned computing nodes is reduced from 10 to 5.

[0090] In this embodiment, storage optimization efforts are reduced based on the fact that actual storage demand is lower than predicted storage demand. If actual storage demand is lower than predicted storage demand, it means that the pressure on storage resources is small. Therefore, the storage optimization efforts can be reduced to avoid over-optimization. Actual storage demand is monitored: real-time monitoring of storage resource usage and comparison of actual and predicted demand: actual storage demand is compared with predicted demand. Storage optimization efforts are reduced: if actual demand is lower than predicted demand, the frequency or scale of optimization measures such as cleaning up useless data and adjusting LUN division is reduced. For example, the actual storage demand is 50 TB, which is lower than the predicted 80 TB. Therefore, the frequency of data compression and deduplication is reduced from once a day to once a week.

[0091] In this embodiment, the scale of resource expansion is increased based on the fact that the actual resource utilization is higher than the predicted resource utilization. If the actual resource utilization is higher than the predicted resource utilization, it means that the pressure on computing resources is large, so the scale of resource expansion needs to be increased to meet business needs. Monitor the actual resource utilization: monitor the usage of resources such as CPU, memory, and storage in real time, compare the actual and predicted utilization: compare the actual resource utilization with the predicted utilization, increase the scale of resource expansion: if the actual utilization is higher than the predicted utilization, increase the number of computing nodes or storage devices. For example, the actual CPU utilization is 80%, which is higher than the predicted 60%, so 5 computing nodes are added to improve the computing power of the cluster.

[0092] The working principle and beneficial effects of the above technical solution are: by dynamically adjusting the storage optimization plan and resource expansion plan, according to the latest storage demand and resource utilization forecast results and actual usage, the storage capacity can be flexibly expanded or the expansion scale can be reduced to ensure efficient use of resources, avoid resource waste and performance bottlenecks, and improve the flexibility of resource management and system performance.

[0093] Example 8: The embodiment of the present invention provides a storage resource pool dynamic adjustment system, such as Figure 2 As shown, including: Verification module: scans the available computing nodes in the network, performs network verification, and obtains hardware readiness instructions; Configuration module: creates an initial resource pool based on the ready instruction, and performs network configuration and storage configuration on the initial resource pool; Prediction module: monitors key indicators of the initial resource pool based on the network configuration and storage configuration, and predicts storage demand and resource utilization based on the monitored key indicators; Adjustment module: formulates a storage optimization plan and a resource expansion plan according to the storage demand and resource utilization, and dynamically adjusts the storage resource pool based on the storage optimization plan and the resource expansion plan.

[0094] The working principle and beneficial effects of the above technical solution are: by scanning and verifying computing nodes, creating resource pools and performing network and storage configuration, monitoring key indicators and predicting storage demand and resource utilization, dynamically adjusting storage optimization and expansion plans, realizing intelligent management and dynamic optimization of resources, accurately predicting storage demand and resource utilization, improving resource management efficiency, reducing human errors, improving operation and maintenance efficiency, and enhancing overall performance.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for dynamically adjusting a storage resource pool, characterized in that: include: Step 1: Scan the available computing nodes in the network, perform network verification, and obtain hardware readiness instructions; Step 2: Create an initial resource pool based on the ready instruction, and perform network configuration and storage configuration on the initial resource pool; Step 3: Based on the network configuration and storage configuration, key indicators of the initial resource pool are monitored, and storage demand and resource utilization are predicted according to the monitored key indicators; Step 4: Formulate a storage optimization plan and a resource expansion plan according to the storage demand and resource utilization, and dynamically adjust the storage resource pool based on the storage optimization plan and the resource expansion plan.

2. A method for dynamically adjusting a storage resource pool according to claim 1, characterized in that: Scan the available computing nodes in the network, perform network verification, and obtain hardware readiness instructions, including: Define an IP address range where the expected computing nodes are located, obtain a scanning requirement based on the IP address range, and select a scanning method according to the scanning requirement; The scanning method is used to determine the computing nodes, and network verification is performed on the computing nodes. According to preset thresholds of latency and packet loss, the network verification is determined to be an available computing node, a cluster is formed, and a hardware readiness instruction is output.

3. The method for dynamically adjusting a storage resource pool according to claim 1, characterized in that: Creating an initial resource pool based on the ready instruction, and performing network configuration and storage configuration on the initial resource pool, including: Based on the cluster in the ready instruction, a host for creating an initial resource pool is selected, and according to actual needs, the type of resource pool to be created is determined, and the initial resource pool is created by combining the cluster, the host, and the resource pool type; Obtaining business requirements, performing scale analysis on the cluster, and performing a first configuration on the initial resource pool in combination with the business requirements and the cluster scale; Determine a network topology according to the distribution of the initial resource pool, and perform a second configuration on the initial resource pool in combination with the network topology and hardware resources; Combining the first configuration and the second configuration to obtain a network configuration; Based on the cluster, divide the initial resource pool into LUNs, determine the heartbeat disk, data disk, and storage redundancy, and perform storage configuration.

4. A method for dynamically adjusting a storage resource pool according to claim 3, characterized in that: Based on the cluster, divide the initial resource pool into LUNs, determine the heartbeat disk, data disk, and storage redundancy, and perform storage configuration, including: ,in, represents the comprehensive predicted storage value of the cluster at time t; Represents a time series model consisting of clusters; represents the weight of the time series model composed of clusters; c represents the constant term; p represents the order of the autoregressive model; i1 represents the index variable of the order; Indicates the impact of past i1 observations on the current value; represents the storage requirement at time point t-i1; represents white noise; q represents the moving average order; Represents the influence coefficient of the past i1 error terms on the current value; represents the white noise at time point t-i1; represents a regression model consisting of clusters; represents the weight of the regression model composed of clusters; represents the intercept term of the regression model; i2 represents the index variable of the external variable; Represents a machine learning model composed of clusters; i3 represents the index variable of the decision tree; represents the influence coefficient of the i2th external variable on storage demand; Indicates time External variables; represents the weight of the machine learning model composed of clusters, where ; N represents the total number of decision trees; Represents the feature vector of the i3th decision tree The prediction results; LUNs are divided according to the comprehensive predicted storage values, heartbeat disks, data disks, and storage redundancy are determined, and the divided LUNs are mapped to nodes in the cluster to determine the storage configuration.

5. A method for dynamically adjusting a storage resource pool according to claim 4, characterized in that: Based on the network configuration and storage configuration, the key indicators of the initial resource pool are monitored, and storage demand and resource utilization are predicted according to the monitored key indicators, including: Determine network configuration related indicators according to the network configuration, determine storage configuration related indicators according to the storage configuration, and simultaneously obtain hardware configuration related indicators corresponding to the hardware configuration, and obtain key indicators by combining the network configuration related indicators, storage configuration related indicators and hardware configuration related indicators; Monitoring the key indicators, and determining storage requirements by combining the monitored key indicators with the comprehensive predicted storage value; Perform load change analysis on the business demand, and determine resource utilization by combining the load change analysis result with key monitored indicators.

6. A method for dynamically adjusting a storage resource pool according to claim 1, characterized in that: Formulate a storage optimization plan and a resource expansion plan according to the storage demand and resource utilization, and dynamically adjust the storage resource pool based on the storage optimization plan and the resource expansion plan, including: Clean up useless data in the storage resource pool based on storage needs, adjust LUN division, optimize storage performance, use data compression and deduplication technology to reduce storage space usage, and then come up with a storage optimization plan; Decomposing and analyzing the storage demand and the resource utilization to obtain the storage capacity demand, and determining to increase storage devices and expand storage capacity based on the storage capacity demand, and determining the CPU and memory utilization based on the decomposition and analysis results, increasing computing nodes, optimizing the cluster, and then obtaining a resource expansion plan; The storage resource pool is dynamically adjusted according to the storage optimization plan and resource expansion plan.

7. A method for dynamically adjusting a storage resource pool according to claim 1, characterized in that: Dynamically adjusting the storage resource pool according to the storage optimization plan and the resource expansion plan includes: Dynamically adjust storage optimization plans and resource expansion plans based on storage demand and resource utilization. If the predicted storage demand increases, expand storage capacity; if the predicted resource utilization decreases, reduce the scale of resource expansion. Dynamically adjust storage optimization plans and resource expansion plans based on actual usage. If the actual storage demand is lower than the predicted storage demand, reduce storage optimization efforts. If the actual resource utilization is higher than the predicted resource utilization, increase the scale of resource expansion.

8. A storage resource pool dynamic adjustment system, characterized in that: include: Verification module: scans available computing nodes in the network, performs network verification, and obtains hardware readiness instructions; Configuration module: creates an initial resource pool based on the ready instruction, and performs network configuration and storage configuration on the initial resource pool; Prediction module: monitors the key indicators of the initial resource pool based on the network configuration and storage configuration, and predicts storage demand and resource utilization according to the monitored key indicators; Adjustment module: formulates a storage optimization plan and a resource expansion plan according to the storage demand and resource utilization, and dynamically adjusts the storage resource pool based on the storage optimization plan and the resource expansion plan.

Images