Storage resource allocation method, storage medium and electronic equipment

Abstract

The invention discloses a storage resource allocation method, a storage medium and electronic equipment, and relates to the technical field of resource allocation, and the method comprises the steps: determining a group of candidate nodes from a second type node set under the condition that a first type node corresponding to a storage request does not exist in a first type node set, the first type of nodes corresponding to the storage request are bound service types corresponding to the service data, and the first type of nodes are in an available state, according to resource utilization parameters of the candidate nodes in the group of candidate nodes, the allocation probability of the candidate nodes in the group of candidate nodes is calculated, and the allocation probability of the candidate nodes in the group of candidate nodes is calculated; and storing the service data into the candidate node with the maximum distribution probability in the group of candidate nodes, thereby solving the technical problem of load imbalance, and achieving the technical effect of improving the performance of the storage system.

Description

Technical Field

[0001] The present application relates to the technical field of resource allocation, and in particular to a storage resource allocation method, a storage medium, and an electronic device. Background Art

[0002] Distributed storage systems play a vital role in modern data centers and cloud environments, providing high-availability and high-performance storage solutions for massive amounts of data. In such systems, efficient storage and fast access to data are crucial to improving overall performance.

[0003] Related technologies employ static resource allocation strategies, such as round-robin allocation or hash mapping. While simple to implement, these strategies lack flexibility and cannot adapt to the dynamic changes in storage node load. In situations such as surging data volumes, inaccurate resource allocation decisions can easily lead to overloading of some nodes while underutilizing resources in other nodes, impacting overall system performance. Consequently, related technologies suffer from the technical problem of load imbalance caused by localized resource overload. Summary of the Invention

[0004] The present application provides a storage resource allocation method, a storage medium, and an electronic device to at least solve the problem of load imbalance caused by local resource overload in the related art.

[0005] The present application provides a storage resource allocation method, which is applied to a storage system including a storage node cluster, wherein the storage node cluster includes a first-type node set and a second-type node set, wherein a first-type node in the first-type node set is bound to at least one service type; the method includes:

[0006] In response to the obtained storage request, if there is no first-type node corresponding to the storage request in the first-type node set, determining a group of candidate nodes from the second-type node set, wherein the storage request is for requesting storage of business data, and the first-type node corresponding to the storage request is a first-type node that is bound to a business type corresponding to the business data and is in an available state;

[0007] calculating, according to resource utilization parameters of the candidate nodes in the set of candidate nodes, an allocation probability of the candidate nodes in the set of candidate nodes, wherein the resource utilization parameters of the candidate nodes in the set of candidate nodes are used to describe resource utilization conditions of the candidate nodes in the set of candidate nodes;

[0008] The service data is stored in a candidate node with the highest allocation probability among the group of candidate nodes.

[0009] The present application further provides a storage resource allocation device, which is applied to a storage system including a storage node cluster, wherein the storage node cluster includes a first-type node set and a second-type node set, wherein a first-type node in the first-type node set is bound to at least one service type; the device includes:

[0010] a determination module configured to, in response to an acquired storage request, determine a group of candidate nodes from the second-type node set if no first-type node corresponding to the storage request exists in the first-type node set, wherein the storage request is for requesting storage of business data, and the first-type node corresponding to the storage request is a first-type node that is bound to a business type corresponding to the business data and is in an available state;

[0011] a calculation module, configured to calculate an allocation probability of a candidate node in the set of candidate nodes according to a resource utilization parameter of the candidate node in the set of candidate nodes, wherein the resource utilization parameter of the candidate node in the set of candidate nodes is used to describe a resource utilization situation of the candidate node in the set of candidate nodes;

[0012] The first storage module is configured to store the service data in a candidate node with the highest allocation probability among the group of candidate nodes.

[0013] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned storage resource allocation methods when executing the computer program.

[0014] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned storage resource allocation methods are implemented.

[0015] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above storage resource allocation methods when executed by a processor.

[0016] Through the present application, in response to an acquired storage request, when there is no first-class node corresponding to the storage request in the first-class node set, a group of candidate nodes is determined from the second-class node set, wherein the first-class node corresponding to the storage request is a first-class node whose bound business type is the business type corresponding to the business data and is in an available state. According to the resource utilization parameters of the candidate nodes in a group of candidate nodes, the allocation probability of the candidate nodes in a group of candidate nodes is calculated, and the business data is stored in the candidate node with the largest allocation probability in a group of candidate nodes, ensuring that when the first-class node of the storage request is unavailable, the business data can be stored on the candidate node with the most optimized resource utilization and the most balanced load. To a certain extent, it solves the technical problem of load imbalance caused by local resource overload in the related technology, achieves more balanced resource allocation, improves overall storage efficiency and data access speed, and thus significantly improves the performance of the storage system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A schematic diagram of an application of a storage resource allocation method provided in an embodiment of the present application;

[0019] Figure 2 A flowchart of an optional storage resource allocation method provided according to an embodiment of the present application;

[0020] Figure 3 A schematic diagram of an optional storage resource allocation method provided according to an embodiment of the present application;

[0021] Figure 4 A schematic diagram of another optional storage resource allocation method provided according to an embodiment of the present application;

[0022] Figure 5 A schematic diagram of an optional storage system provided in an embodiment of the present application;

[0023] Figure 6 A flowchart of another optional storage resource allocation method provided according to an embodiment of the present application;

[0024] Figure 7 This is a structural block diagram of an optional storage resource allocation device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0027] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0028] Related technologies employ static resource allocation strategies, such as round-robin allocation or hash mapping. While simple to implement, these strategies lack flexibility and cannot adapt to the dynamic changes in storage node load. In situations such as surging data volumes, inaccurate resource allocation decisions can easily lead to overloading some nodes and underutilizing resources on other nodes, impacting overall system performance. Consequently, related technologies suffer from the technical problem of unbalanced resource allocation, resulting in poor storage system performance.

[0029] To solve the above problems, an embodiment of the present application provides a storage resource allocation method. When there is no first-class node corresponding to the storage request in the first-class node set, a group of candidate nodes is determined from the second-class node set, wherein the first-class node corresponding to the storage request is a first-class node whose bound business type is the business type corresponding to the business data and is in an available state. According to the resource utilization parameters of the candidate nodes in a group of candidate nodes, the allocation probability of the candidate nodes in a group of candidate nodes is calculated, and the business data is stored in the candidate node with the highest allocation probability in a group of candidate nodes. This ensures that when the first-class node of the storage request is unavailable, the business data can be stored on the candidate node with the most optimized resource utilization and the most balanced load. To a certain extent, it solves the technical problem of load imbalance caused by local resource overload in the related technology, achieves more balanced resource allocation, improves overall storage efficiency and data access speed, and thus significantly improves the performance of the storage system.

[0030] According to one aspect of the embodiment of the present application, a storage resource allocation method is provided. Optionally, in this embodiment, the above storage resource allocation method can be applied to, but is not limited to, Figure 1 The hardware environment shown includes a terminal device 102 and a server 104. The server 104 can be connected to the terminal device 102 via a network and can be used to provide services (e.g., application services, etc.) for the terminal device 102 or a client installed on the terminal device 102. A database can be set on the server 104 or independently of the server 104 to provide data storage services for the server 104.

[0031] The aforementioned network may include, but is not limited to, at least one of the following: a wired network and a wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, or a local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity) and Bluetooth. The terminal device 102 may be, but is not limited to, a PC (Personal Computer), a mobile phone, a tablet computer, etc. The server 104 may be, but is not limited to, a cloud server, a server cluster, or other server types.

[0032] The storage resource allocation method of the embodiment of the present application can be executed by the server 104, or by the terminal device 102, or by both the server 104 and the terminal device 102. The storage resource allocation method of the embodiment of the present application can also be executed by a client installed on the terminal device 102.

[0033] Taking the storage resource allocation method in this embodiment executed by the terminal device 102 as an example, Figure 2 A flowchart of an optional storage resource allocation method provided according to an embodiment of the present application is shown in FIG. Figure 2 As shown, the process of the method may include the following steps:

[0034] Step S202: In response to the obtained storage request, if there is no first-class node corresponding to the storage request in the first-class node set, determine a group of candidate nodes from the second-class node set, wherein the storage request is used to request the storage of business data, and the first-class node corresponding to the storage request is a first-class node whose bound business type is the business type corresponding to the business data and is in an available state.

[0035] It should be noted that the terminal device can be deployed with a storage system including a storage node cluster. The storage system can be a distributed storage system. The storage node cluster can be a set of multiple storage nodes. The storage nodes in the storage node cluster jointly undertake data storage tasks, which can mainly include a first-class node set and a second-class node set. Among them, the first-class node can refer to a storage node set bound to a specific business, also known as an attribution node set. The first-class node set includes at least one first-class node, and a first-class node in the first-class node set is bound to at least one business type. The first-class node can be an attribution node. In the storage node cluster, different types of first-class nodes can be set according to different business needs.

[0036] Optionally, the most suitable storage node can be dynamically bound based on business-aware strategies. Specifically, the data generation location (prioritizing edge nodes to reduce latency), user access patterns (high-frequency access to IP corresponding nodes to improve access speed) and data feature labels (such as medical images are preferentially stored locally to ensure data security and access efficiency) can be comprehensively considered to ensure that data is allocated to nodes that can best realize their business value, achieving efficient resource utilization and balanced load distribution. This mechanism shortens the data processing path through intelligent matching, while reducing network transmission pressure and improving the overall performance of the system.

[0037] The second type of node set may be a set of storage nodes that do not belong to the first type of nodes, and may also be referred to as a non-belonging node set, and is used as backup storage resources when the resources of the first type of nodes are insufficient.

[0038] Optionally, business data of a characteristic type is associated with a set of exclusive storage nodes, ie, a first type node set, so that such business data is preferentially stored on the corresponding home node, so as to improve data access efficiency and reduce latency.

[0039] A storage request is an application's request to write data to a storage system. This request can include the data to be stored, also known as business data. Business data refers to data that users or applications need to store and access when executing specific business processes, such as medical images, financial transaction records, video files, or documents.

[0040] Candidate nodes are a group of storage nodes selected from the second set of nodes (non-home nodes) that meet the resource requirements for a storage request when the first set of nodes (home nodes) are insufficient or unavailable. These nodes serve as backup storage resources and are further evaluated to determine which is best suited to store specific business data.

[0041] The first type of nodes corresponding to storage requests are those that are bound to the requested business data type and are in an available state. These nodes are prioritized for data storage because they are closely associated with specific business types and can provide the best performance and efficiency.

[0042] Optionally, upon receiving a storage request, the storage request is parsed to obtain parsed data. Based on the parsed data and pre-set conditions, it is determined whether a first-category node corresponding to the storage request exists. If no first-category node corresponding to the storage request exists, a set of second-category nodes may be obtained and selected from the set of second-category nodes as candidate nodes. The selection process may be based on factors such as the remaining space and performance level of the second-category nodes to ensure that the candidate nodes have the capacity to store the business data.

[0043] Optionally, when it is determined that there is a first type node corresponding to the storage request, the business data corresponding to the storage request can be stored in the first type node corresponding to the storage request, and the metadata corresponding to the business data can be stored in a designated metadata database.

[0044] Step S204 , calculating the allocation probability of a candidate node in a group of candidate nodes according to the resource utilization parameters of the candidate nodes in the group of candidate nodes, wherein the resource utilization parameters of the candidate nodes in the group of candidate nodes are used to describe the resource utilization of the candidate nodes in the group of candidate nodes.

[0045] It should be noted that resource utilization parameters can be used to reflect the current resource usage status of each candidate node in a group of candidate nodes. Resource utilization parameters may include, but are not limited to, remaining storage space, CPU utilization, network bandwidth, and storage node performance. Resource utilization parameters can be used to assess whether a candidate node has sufficient capacity and stability to handle new storage requests.

[0046] Allocation probability measures the likelihood of each candidate node being selected to store specific business data. This probability is calculated based on the candidate node's resource utilization parameters. Nodes with better resource utilization have higher allocation probabilities, meaning they are more likely to be selected for data storage. This calculation ensures fair and efficient resource allocation, avoids node overload and resource idleness, and promotes balanced and optimized resource utilization across the storage node cluster.

[0047] Optionally, after screening out a group of candidate nodes, the resource utilization parameters of the candidate nodes in the group of candidate nodes are analyzed and the resource utilization parameters are standardized and converted, so as to determine the allocation probability corresponding to the candidate nodes based on the standardized resource utilization parameters, wherein the sum of the allocation probabilities corresponding to a group of candidate nodes is 1.

[0048] Step S206: storing the service data in a candidate node with the highest allocation probability among a group of candidate nodes.

[0049] It should be noted that the size of the allocation probability reflects the comprehensive resource utilization of the node and its suitability for data storage, ensuring that data can be stored efficiently and stably.

[0050] The allocation probability of each candidate node in a set of candidate nodes can be directly correlated with the resource utilization parameters of the corresponding candidate node. The candidate node with the highest allocation probability indicates that its resource conditions are most suitable for storing the current business data, with the lowest latency, highest storage efficiency, and best performance.

[0051] For example, when there are three candidate nodes (such as nodes A, B, and C), the allocation probability of node A is 0.4, while the allocation probabilities of nodes B and C are 0.3 and 0.3 respectively, node A will be automatically selected to store data, making full use of its resource advantages while maintaining the load balance of the entire storage node cluster.

[0052] Through the embodiments of the present application, in response to an acquired storage request, when there is no first-class node corresponding to the storage request in the first-class node set, a group of candidate nodes is determined from the second-class node set, wherein the first-class node corresponding to the storage request is a first-class node whose bound business type is the business type corresponding to the business data and is in an available state. According to the resource utilization parameters of the candidate nodes in a group of candidate nodes, the allocation probability of the candidate nodes in a group of candidate nodes is calculated, and the business data is stored in the candidate node with the largest allocation probability in a group of candidate nodes, ensuring that when the first-class node of the storage request is unavailable, the business data can be stored on the candidate node with the most optimized resource utilization and the most balanced load. To a certain extent, it solves the technical problem of load imbalance caused by local resource overload in the related technology, achieves more balanced resource allocation, improves overall storage efficiency and data access speed, and thus significantly improves the performance of the storage system.

[0053] In an exemplary embodiment, a storage request includes a requested business type and a requested storage capacity, and the above method also includes: when a target node corresponding to the storage request is determined in a first-category node set, obtaining a first specified capacity threshold, wherein the first specified capacity threshold is determined based on the requested storage capacity and a first preset coefficient, and the target node is determined based on the requested business type; obtaining a load score of the target node, wherein the load score is obtained based on a preset scoring model; when the remaining resource capacity of the target node is greater than or equal to the first specified capacity threshold and the load score of the target node is greater than the first preset scoring threshold, determining that the target node is the first-category node corresponding to the storage request, and storing the business data in the first-category node corresponding to the storage request.

[0054] It should be noted that a storage request may include, but is not limited to, a requested service type and a requested storage capacity. The requested service type may refer to the type of data or service scenario involved in the storage request, such as streaming, database backup, or medical imaging, and can be used to determine data storage priorities and rules. The requested storage capacity may be the required storage space for the data to be stored, as indicated in the storage request. The target node may be a storage node in the first-category node set, determined based on the requested service type, and is typically bound to that service type to leverage data locality.

[0055] The first specified capacity threshold can be calculated based on the requested storage capacity and a first preset coefficient. The first specified capacity threshold can be used to assess whether the target node can meet the minimum capacity requirement of the storage request. The first preset coefficient reflects additional considerations of the resource requirements of the service type. For example, a higher coefficient may be set for latency-sensitive services to reserve more buffer space.

[0056] Optionally, the first specified capacity threshold may be obtained by multiplying the requested storage capacity by a first preset coefficient. For example, when the first preset coefficient is 1.2 and the requested storage capacity is 100 MB, the corresponding first specified capacity threshold is 120 MB.

[0057] The load score reflects the overall score of a storage node's current resource usage requests and can be calculated based on a preset scoring model. Alternatively, the scoring model can be a mathematical model that comprehensively evaluates the target node's load status. The scoring model can be constructed using linear models, neural networks, or other statistical learning methods. Its core function is to convert multi-dimensional node status into a single scoring metric to facilitate decision-making.

[0058] The first preset scoring threshold may be a preset load scoring standard, which may be set based on empirical values, or may be updated at preset time intervals based on the performance of the storage node cluster in historical periods. Optionally, the first preset scoring threshold may be calculated using a machine learning model.

[0059] Optionally, a second preset scoring threshold may also be set in the storage system. When the load score corresponding to the target node is less than the second preset scoring threshold, the target node is marked to indicate that the target node is in an overloaded state, wherein the second preset scoring threshold is less than the first preset scoring threshold.

[0060] Optionally, in this embodiment, the load score of each storage node in the storage node cluster can be recorded in real time and compared with a second preset score threshold to dynamically monitor the node status. When the load score of a storage node falls below the second preset score threshold, the node is automatically marked as overloaded to prevent it from receiving new storage requests.

[0061] Optionally, for each storage node, a corresponding first preset scoring threshold and a second preset scoring threshold can also be set. The first preset scoring threshold and the second preset scoring threshold corresponding to each storage node can be generated by a machine learning model. Specifically, for each storage node, a threshold generation model is used to dynamically generate the corresponding first preset scoring threshold and the second preset scoring threshold to achieve personalized resource management. The threshold generation model is based on historical data, including the node's resource utilization, performance, business type load pattern, etc., by predicting the optimal threshold of the storage node in different states. For overload warning (second preset scoring threshold), the indicator characteristics of the storage node before it is about to overload are learned, and a lower threshold is set to avoid resource bottlenecks in advance. The health status threshold (first preset scoring threshold) is determined based on the indicator distribution of the storage node during long-term stable operation to ensure reasonable resource allocation. This mechanism achieves precise control of node status by continuously optimizing the scoring threshold, thereby improving the adaptability and overall performance of the storage system.

[0062] Through this embodiment, by introducing the first specified capacity threshold and load score, it is ensured that the target nodes in the first type of node set are used for data storage first when the resource and load status meet the conditions, thereby improving the data access speed and effectively avoiding the home node from being used for data storage when the resources are insufficient or overloaded, preventing the occurrence of performance bottlenecks and maintaining the high availability of the system.

[0063] In an exemplary embodiment, obtaining a load score of a target node includes: collecting indicator data of the target node, wherein the indicator data includes resource utilization data and performance data; obtaining a set of weight factors, wherein the set of weight factors is determined based on specified status data of a storage node cluster within a specified time period; and inputting the indicator data of the target node and the set of weight factors into a scoring model to obtain a load score of the target node.

[0064] It should be noted that indicator data can include resource utilization data and performance data, specifically covering the target node's hardware resource usage (such as CPU, memory, I / O utilization), network performance indicators (such as bandwidth, latency), and hardware performance data. For example, resource utilization data may include remaining storage space and CPU utilization, while performance data may include network bandwidth and the performance of the storage medium corresponding to the storage node. Indicator data can be used to provide a quantitative description of the target node's current operating status and can be used to calculate a load score.

[0065] Resource utilization data refers to resource usage request data, used to assess whether a node has sufficient resource capacity to meet new storage requirements. Performance data can include metrics such as network bandwidth, I / O response time, and storage media performance. It reflects the node's efficiency in processing data read and writes, network transmission, and other aspects, and is a key parameter for measuring node performance.

[0066] A set of weighting factors can be a set of pre-set or dynamically adjusted values ​​used to express the importance of different metrics in the scoring model. Weighting factors can be trained using a machine learning algorithm based on historical data from the storage node cluster, or they can be manually set based on specific business needs. For example, for latency-sensitive applications, network bandwidth and I / O response time might receive higher weighting factors.

[0067] Scoring models can be used to receive indicator data and a set of weight factors as input and output a load score for the target node. Scoring models can be constructed using linear models, neural networks, or other statistical learning methods.

[0068] Specifically, the scoring model is expressed using formula (1).

[0069] Score i =A×S free +B×(1-U cpu )+C×B net +D×P medium ;(1)

[0070] Among them, S free is the remaining storage space after normalization of node i; U cpu is the CPU utilization of node i; B net is the network bandwidth of node i; P medium is the storage medium performance coefficient of node i; A, B, C, and D are weight factors, and the sum of A, B, C, and D is 1.

[0071] It should be noted that, before inputting the indicator data of the target node into the scoring model, the indicator data may be preprocessed so that the range of the scoring data output by the scoring model is within a preset range, for example, between 0 and 1.

[0072] The preprocessing process may include normalizing the remaining storage space and performing coefficient conversion based on the storage medium type. For example, if the storage medium type is NVMe SSD (Non-Volatile Memory Express Solid State Drive) = 1.0; if the storage medium type is HDD (Hard Disk Drive) = 0.6.

[0073] In this example, assume the target node is a high-performance storage node that needs to store a batch of critical business data. The following are the specific steps to obtain the target node load score:

[0074] The deployed monitoring agent collects the target node's resource utilization data (such as the remaining hard disk space is 2TB, the CPU utilization is 30%) and performance data (the network bandwidth is 900Mbps, the storage medium performance is the storage medium coefficient, and the storage medium coefficient is determined based on the type of storage medium) in real time.

[0075] Assume that based on the average status data of the past week, the set of weight factors optimized by the machine learning algorithm is: A is 0.4, B is 0.3, C is 0.2, and D is 0.1.

[0076] The above indicator data and weight factor set are input into the scoring model corresponding to formula (1), and the load score of the target node is obtained by linear weighted summation.

[0077] This embodiment collects the target node's index data and combines it with a set of weight factors to input into the scoring model, achieving dynamic calculation of the load score and ensuring the accuracy and real-time nature of the score. Dynamically adjusting the weight factors based on historical data enables the scoring model to adapt to different business scenarios and resource states, improving the intelligence and flexibility of resource allocation decisions. By evaluating the target node's load score, it is possible to intelligently determine whether the node is overloaded, avoiding storing data on nodes with higher loads, and promoting overall load balancing of the storage node cluster.

[0078] In an exemplary embodiment, obtaining a set of weight factors includes: obtaining specified status data of a storage node cluster within a specified time period, wherein the specified status data includes health status data and performance status data; inputting the specified status data into a pre-trained weight allocation model, and outputting a set of weight factors.

[0079] It should be noted that the specified status data within a specified time period may be the health status data and performance status data of the storage node within a preset historical time period. Among them, the specified status data may include hardware health indicators (such as SSD wear, hard disk SMART attributes), resource utilization efficiency (such as CPU utilization, memory utilization, I / O operation rate), network performance (such as bandwidth utilization, network latency), etc. Health status data may include health indicators of storage media, such as the P / E cycle of SSD, the number of reallocated sectors of hard disk, etc. Performance status data: covers the resource utilization and network performance data of the node, such as CPU occupancy, memory remaining, network bandwidth utilization, etc., which are used to evaluate the processing power and efficiency of the node. The sum of the weight factors in the weight factor set is 1.

[0080] The weight distribution model can be a pre-trained machine learning model that can be used to automatically adjust the weight factors of multiple different preset indicators based on the input specified status data. By optimizing the distribution of weight factors, the scoring model can ensure that it can more accurately reflect the load status of the node.

[0081] Optionally, the training process of the weight distribution model may include:

[0082] 1. Collect a large amount of historical status data, including health status data and performance status data in the storage node cluster, as well as the actual load of the nodes in these states.

[0083] 2. Preprocess the collected raw data, including data cleaning, feature extraction (such as calculating average CPU utilization, network latency distribution, etc.), and feature selection (selecting the indicators most relevant to the load).

[0084] 3. Select an appropriate machine learning model, such as a random forest, support vector machine, or deep learning model, to learn the weight factors of the impact of different indicators on node load.

[0085] 4. Use historical status data and actual node load as training samples to train the weight allocation model to minimize the gap between the predicted weight factor set and the actual load situation.

[0086] 5. Evaluate the model's prediction accuracy and generalization ability through cross-validation or reserved test sets to ensure the model's performance on unknown data.

[0087] 6. Adjust model parameters based on the verification results and retrain if necessary.

[0088] In one example, if Figure 3As shown, multi-dimensional indicator data is collected, such as CPU utilization, which reflects the computing load of the node; memory utilization, which shows the node's memory resource usage; I / O operation rate, which shows the frequency and efficiency of the node's input and output operations; network bandwidth, which shows the node's network transmission capacity; and storage media health, which assesses the health of storage devices, such as SSD wear and HDD SMART attributes.

[0089] Obtain a set of weight factors. Specifically, obtain the set of weight factors through a weight distribution model and specified state data.

[0090] The indicator data and weight factor set are input into the scoring model to obtain the node load score. Through normalization, the multi-dimensional indicators are converted into a unified score range (such as 0-1), reflecting the overall health and performance status of the node.

[0091] This embodiment automatically generates a set of weight factors by inputting specified storage node cluster status data into a pre-trained weight allocation model. This enables automated and personalized adjustment of the weight factors, improving the accuracy of the scoring model. Furthermore, the dynamic generation of the weight factor set enables the scoring model to reflect the health and performance status of the cluster in real time, effectively avoiding scoring bias caused by static weights and ensuring the dynamic adaptability of load scoring and the reliability of intelligent selection.

[0092] In an exemplary embodiment, a group of candidate nodes is determined from a set of second-class nodes, including: obtaining the remaining resource capacity and the performance level of the second-class nodes, wherein the performance level is determined based on the type of storage medium of the second-class nodes; obtaining a second specified capacity threshold, wherein the second specified capacity threshold is determined based on the requested storage capacity and a second preset coefficient, and the second specified capacity threshold is greater than the first specified capacity threshold; and selecting the second-class nodes in the set of second-class nodes whose remaining resource capacity is greater than the second specified capacity threshold and whose performance level is greater than the preset level as candidate nodes.

[0093] It should be noted that the second type node set may include multiple second type nodes, and the second type nodes may be nodes that are not bound to a service type, also called non-attributed nodes.

[0094] The remaining resource capacity of the second type of node may be the size of the storage space currently available to the second type of node.

[0095] Performance levels can be divided into node performance levels based on storage media types (such as NVMe SSD, HDD). Different types of storage media have different performance characteristics such as read and write speed, latency, and lifespan.

[0096] The second specified capacity threshold can be a minimum remaining resource capacity criterion for screening candidate nodes. It is calculated based on the requested storage capacity and a second preset coefficient, ensuring the adequacy and security of allocated resources. The second preset coefficient is greater than the first preset coefficient. The second specified capacity threshold is greater than the first specified capacity threshold.

[0097] The preset level may be a minimum performance level standard set when determining a candidate node, to ensure that the selected node meets certain requirements in terms of performance.

[0098] In one example, assume a storage request requires 500GB of space and the system has a preset amplification factor of 1.5. This means that when screening candidate nodes, the second specified capacity threshold is 750GB. The preset performance level is Tier 2 to ensure data security and processing speed. The remaining resource capacity and performance level of each second-class node are compared, and second-class nodes with a capacity greater than 750GB and a performance level higher than or equal to Tier 2 are selected as candidate nodes. From the selected candidate nodes, the optimal storage node is further selected for data storage operations.

[0099] Optionally, the best storage node can be selected from the selected candidate nodes based on the scoring model for data storage. Alternatively, the allocation probability of the candidate nodes can be calculated based on their resource utilization parameters, thereby selecting the candidate node with the highest allocation probability as the best storage node.

[0100] Of course, when selecting non-attributed nodes for data storage, network topology information can also be used to identify all available nodes and select a set of nodes with latency below 5ms to ensure efficient data transmission. At the same time, 10% of bandwidth is reserved for cross-node data transmission. By dynamically adjusting network resource allocation, network congestion caused by sudden large traffic requests is prevented, ensuring the stability of data transmission. This strategy combines latency sensitivity with bandwidth reservation to effectively optimize network resource utilization in distributed storage systems and improve overall data processing performance. Through continuous monitoring and intelligent decision-making, the system can flexibly adjust resource allocation based on network conditions, avoiding the impact of network bottlenecks on storage efficiency.

[0101] This embodiment sets a second specified capacity threshold to ensure sufficient capacity to handle unexpected situations during data writing or subsequent capacity expansion, reducing the risk of data loss. The screening process not only considers storage space availability but also incorporates the performance level of the storage medium, prioritizing high-performance nodes and reducing data processing latency. By selecting only nodes that meet both high-capacity and high-performance requirements as candidates, the inappropriate allocation of low-performance nodes is avoided, helping to maintain load balancing across the storage cluster.

[0102] In an exemplary embodiment, the resource utilization parameters include utilization rates of different types of resources in the candidate nodes; the allocation probability of the candidate nodes in a group of candidate nodes is calculated based on the resource utilization parameters of the candidate nodes in a group of candidate nodes, including: standardizing the utilization rates of different types of resources in the candidate nodes in a group of candidate nodes to obtain the occupancy ratios of different types of resources in the candidate nodes in a group of candidate nodes; and calculating the allocation probability of the candidate nodes in a group of candidate nodes based on the occupancy ratios of different types of resources in the candidate nodes in a group of candidate nodes.

[0103] It should be noted that resource utilization parameters can be used to reflect the usage of various types of resources (such as CPU, memory, storage I / O, etc.) on the candidate node, and are usually expressed in percentage form, such as 85% CPU utilization and 70% memory utilization.

[0104] Normalization involves converting resource utilization rates of different types into a uniform range (e.g., a value between 0 and 1) to facilitate comparison and calculation. This typically involves data preprocessing to ensure that resource utilization rates of different resource types are comparable when calculating allocation probabilities.

[0105] The occupancy ratio may be a standardized resource utilization rate, representing the relative degree to which each resource on the candidate node is occupied, and is used to evaluate the comprehensive load of the candidate node.

[0106] The allocation probability represents the likelihood that each candidate node will be selected to allocate a new resource request. When calculating the allocation probability, the occupancy rate of each node is taken into account to achieve load balancing and resource optimization.

[0107] Through this embodiment, by converting resource utilization parameters into allocation probabilities, a node with more balanced resource utilization is selected for storage allocation, thereby reducing the risk of overloading a single resource and achieving load balancing for the entire cluster.

[0108] In an exemplary embodiment, the allocation probability of a candidate node in a group of candidate nodes is calculated based on the occupancy ratio of different types of resources in the candidate nodes in a group of candidate nodes, including: calculating the resource utilization entropy value of the candidate node based on the occupancy ratio of different types of resources in the candidate nodes in a group of candidate nodes; performing negative exponential conversion processing on the resource utilization entropy value of the candidate node in a group of candidate nodes to obtain the negative entropy weight of the candidate node in the group of candidate nodes; and normalizing the negative entropy weight of the candidate node in a group of candidate nodes to obtain the allocation probability of the candidate node in the group of candidate nodes.

[0109] It should be noted that the resource utilization entropy value can be used to measure the degree of balance of resource utilization of candidate nodes. The lower the resource utilization entropy value, the more concentrated and unbalanced the resource utilization is, and the higher the resource utilization entropy value, the more balanced the resource utilization is.

[0110] The negative entropy weight can be obtained by converting the negative exponential of the resource utilization entropy value. It can be used as a weight value to reflect the load balancing status of the candidate node. The larger the weight value, the more balanced the load resources of the candidate node and the higher the possibility of allocating new tasks, that is, the higher the possibility of storing business data.

[0111] Alternatively, as Figure 4 As shown, by converting the resource utilization corresponding to each resource in the candidate node into an occupancy ratio, the resource utilization entropy value of each candidate node is calculated using a preset information entropy formula, so as to obtain the allocation probability of the candidate node based on the resource utilization entropy value. Optionally, the resource utilization entropy value can be subjected to a negative exponential conversion process to obtain a negative entropy weight, and then normalized according to the negative entropy weight of the candidate node to obtain the allocation probability of the candidate node, wherein the sum of the allocation probabilities of the candidate nodes in a group of candidate nodes is 1.

[0112] Specifically, the calculation process of the candidate node allocation probability can be as follows:

[0113] The calculation process of resource utilization entropy value can be shown as formula (2) and (3):

[0114] Normalize the resource utilization. Assume that candidate node i has m resources (such as CPU, memory, I / O, etc.), and their original utilizations are u1, u2, ..., u m . It needs to be converted into a probability distribution p k ,satisfy

[0115]

[0116] Where k represents the kth resource in candidate node i, H i is the resource utilization entropy of candidate node i, H i The larger the value, the more balanced the resource utilization (the utilization rates of each resource are close). i The smaller it is, the more unbalanced the resource utilization is. k is the probability distribution of the kth resource of candidate node i, that is, the occupancy ratio.

[0117] The process of performing negative exponential conversion and normalization on the resource utilization entropy value to obtain the allocation probability of a candidate node in a group of candidate nodes can be shown as formula (4):

[0118]

[0119] Among them, H i The resource utilization entropy value of candidate node i; represents the negative entropy weight of candidate node i; P i is the probability of assigning candidate node i; n is the number of candidate nodes, where ∑P i =1.

[0120] Among them, the resource utilization entropy value H i The smaller (load imbalance), The larger the value, the greater the probability of assignment P. i The higher.

[0121] Resource utilization entropy H i The larger (load balancing), The smaller the distribution probability P i The lower.

[0122] In an example, assuming that there are three candidate nodes, namely node A, node B, and node C, the resource utilization entropy value corresponding to each node can be:

[0123] Node A: H1 = 0.8;

[0124] Node B: H2 = 0.5;

[0125] Node C: H3=0.2.

[0126] The corresponding negative entropy weight is:

[0127]

[0128] The corresponding distribution probability can be:

[0129] P2≈0.32, P3≈0.44;

[0130] Therefore, node C is selected as the optimal storage node to store the business data in node C.

[0131] Through this embodiment, by calculating the resource utilization entropy value of the candidate node and converting it into allocation probability, new tasks tend to be allocated to candidate nodes with more balanced loads, avoiding overloading of some nodes while idle resources of other nodes, and improving resource utilization efficiency and the overall performance of the storage system.

[0132] In an exemplary embodiment, a storage system includes a global metadata database and multiple metadata sub-pools. A storage node in a storage node cluster is configured with at least one metadata sub-pool, and the at least one metadata sub-pool is used to record metadata corresponding to business data stored by the storage node. The method also includes: monitoring the remaining storage space of metadata sub-pools in the multiple metadata sub-pools and the remaining storage space of the global metadata database; and adjusting the storage space of the metadata sub-pool or the storage space of the global metadata database based on the remaining storage space of the metadata sub-pool and the remaining storage space of the global metadata database.

[0133] It should be noted that the global metadata repository can be the central database in the storage system, used to uniformly manage and store information for all metadata sub-pools, including but not limited to metadata classification, storage location, and access permissions. The global metadata block can serve as a metadata information aggregation and coordination center, ensuring the accuracy and consistency of metadata in a distributed storage environment.

[0134] A metadata subpool is one or more database components configured on each storage node in a storage node cluster. It stores and manages metadata corresponding to the business data stored on that node. Metadata subpools operate independently, dynamically adjusting their storage space based on node load and business needs to optimize local metadata access efficiency and resource utilization.

[0135] Optionally, after storing data, metadata is stored in dedicated metadata sub-pools that are independently managed to ensure fast retrieval and efficient updates. This separation of data and metadata is achieved through global metadata database tracking, enhancing system stability and resource management flexibility.

[0136] Optionally, in this embodiment, the process of adjusting the storage space of the metadata sub-pool or the storage space of the global metadata database based on the remaining storage space of the metadata sub-pool and the remaining storage space of the global metadata database may include: when it is detected that the remaining storage space of a metadata sub-pool is less than a first preset storage threshold, performing space expansion processing on the metadata sub-pool whose remaining storage space is less than the first preset storage threshold;

[0137] When it is monitored that the remaining storage space of the global metadata database is less than the second preset storage threshold, or the remaining storage space of a metadata sub-pool is greater than the third preset storage threshold, the space of the metadata sub-pool is triggered to be recalled to increase the storage space of the global metadata database.

[0138] The metadata sub-pool includes multiple data storage blocks. When there is a first target data block, the target data blocks are merged. The first target data block is an adjacent data storage block in a space state with a free space size greater than a first preset value.

[0139] In the presence of a second target data block, defragmentation processing is performed to merge the second target data blocks in the metadata sub-pool, wherein the second target data block is an adjacent data storage block in a space state and has a free space size less than a second preset value; the first preset value is greater than the second preset value.

[0140] Through the above embodiment, when it is monitored that the remaining storage space of the metadata sub-pool is lower than the first preset threshold, the storage space of the metadata sub-pool is automatically expanded, thereby avoiding metadata operation blockage caused by insufficient space and ensuring the continuity and efficiency of metadata management. At the same time, when the remaining storage space of the global metadata database is lower than the second preset threshold, or the remaining storage space of the metadata sub-pool is abnormally large, the space recall mechanism is triggered to call the excess space in the metadata sub-pool back to the global metadata database, balancing the resource allocation between the global metadata database and each metadata sub-pool, and avoiding the waste and uneven use of storage resources. For adjacent data storage blocks that are in an idle state and whose size exceeds the first preset value, a merge process is performed to form a larger continuous free block, which is convenient for subsequent large-block data storage and improves the utilization of storage space. Secondly, for adjacent free blocks that are smaller than the second preset value (the first preset value is greater than the second preset value), defragmentation is performed to merge these small blocks into large continuous spaces, further reducing storage fragmentation, improving storage layout, and improving data read and write performance and storage compactness.

[0141] In one example, each storage node A, B, C, etc. is configured with its own metadata sub-pool for storing and managing the metadata of the business data stored by that node. Meanwhile, a global metadata repository is used to record and manage the status information of all metadata sub-pools, as well as metadata records for the entire system.

[0142] First, the metadata database is initialized and each metadata sub-pool is initially reserved. For example, 100MB of space is reserved for each metadata sub-pool to prevent sudden metadata operation blockage.

[0143] The monitoring system regularly checks the remaining storage space in each metadata subpool. For example, if the remaining space in the metadata subpool on Node A is 100MB, the remaining space in the metadata subpool on Node B is 256MB. The monitoring system also monitors the remaining space in the global metadata database. Assume that the remaining space in the global metadata database is 1GB.

[0144] When the remaining space in a metadata subpool (such as Node A's metadata subpool) falls below a preset threshold (e.g., below 256MB), the system proactively requests additional storage space from the global metadata repository to expand the subpool's capacity. For example, Node A's metadata subpool requests 16MB of space from the global metadata repository, restoring its remaining space to 116MB.

[0145] Conversely, if the global metadatabase is running low on storage space, but a metadata subpool (such as Node B's) has ample free space, the system can initiate a space recall mechanism to reclaim some storage space from the metadata subpool to alleviate the pressure on the global metadatabase. For example, the system might recall 64MB of space from Node B's metadata subpool and add it to the global metadatabase.

[0146] The free space in the metadata subpool on the storage node is regularly checked. When the total free space of adjacent small blocks reaches or exceeds the first preset value, these small blocks are merged into a large, continuous space to facilitate subsequent resource allocation. This merged space is then proactively returned to the global metadata repository for unified management. This mechanism effectively utilizes scattered small blocks of space and improves storage efficiency. When any continuous free space is detected to be less than the second preset value, making it difficult to store large files, the defragmentation process is automatically initiated to reorganize and merge multiple small space blocks to form a larger, continuous space and optimize the storage layout. In this dynamic adjustment strategy, the thresholds for the first and second preset values ​​are not fixed, but are intelligently adjusted based on the current system load to achieve optimal storage resource management and performance.

[0147] This embodiment monitors the storage space usage of metadata sub-pools and the global metadata database in real time, and can intelligently adjust storage allocation to avoid insufficient space in local metadata sub-pools or waste of global metadata database resources, thereby ensuring efficient resource utilization.

[0148] In order to better understand the storage resource allocation process in the embodiment of the present application, an example is used for illustration. Figure 5 As shown, Figure 5 A schematic diagram of an optional storage system provided in an embodiment of the present application.

[0149] The storage system may include a monitoring and feedback layer, a resource allocation layer, a storage node cluster, and a metadata management layer.

[0150] Specifically, the monitoring and feedback layer can be responsible for collecting and monitoring various real-time indicators of storage nodes, including but not limited to CPU utilization, memory usage, I / O operation rate, network bandwidth, and the health status of storage media. By updating these indicators in real time and feeding them back to the resource allocation layer, the system can dynamically adjust resource allocation strategies to cope with changing workloads and environmental factors. The monitoring and feedback layer can also include an entropy value calculation and update module, which determines the entropy value of candidate nodes by collecting and monitoring various real-time indicators of storage nodes, thereby determining candidate nodes that require data storage.

[0151] The resource allocation layer uses scoring models and load balancing algorithms based on data provided by the monitoring and feedback layer to determine data storage locations. It first analyzes the storage request to determine the data type and capacity requirements. It then evaluates the resource availability of the home node and decides whether to allocate resources directly to the home node or use a load balancing algorithm to select the most suitable alternative node among non-home nodes for data write.

[0152] A storage node cluster, comprised of both home and non-home nodes, is where data is actually stored. Home nodes are the default storage locations for data, determined based on business policies or geographic factors, while non-home nodes serve as a supplementary resource when home nodes are insufficient. A backup resource pool can also be included within the cluster to provide additional storage or computing resources in emergencies.

[0153] The metadata management layer is responsible for storing and managing metadata information related to data objects, such as the data's physical location, permissions, and format. Metadata subpools are subsets of metadata management and are managed independently to improve efficiency and reduce resource contention. The global metadata repository aggregates information from all subpools to ensure metadata consistency and integrity. Spatial recall and synchronization mechanisms are used to reclaim resources from inactive metadata subpools when resources are limited, ensuring the efficient operation of core metadata services.

[0154] In order to better understand the storage resource allocation process in the embodiment of the present application, an example is used for illustration. Figure 6 As shown, Figure 6 A flowchart of another optional storage resource allocation method provided in an embodiment of the present application.

[0155] Step S602: receiving a storage request;

[0156] Specifically, the storage request may be a request for business data that needs to be stored, received from a client or other system.

[0157] Step S604, parsing request parameters;

[0158] Specifically, the storage request is parsed to obtain the data service type (such as cold / hot data) and the requested storage capacity.

[0159] Step S606, checking home node resources;

[0160] Specifically, the remaining space and load score of the home node are evaluated.

[0161] Step S608: Check whether the remaining space is sufficient and whether the load score meets the requirements;

[0162] Specifically, it is determined whether the remaining space and load score of the home node are sufficient to meet the current storage request. If the current storage requirement is met, the process jumps to step S610; if not, the process jumps to step S612.

[0163] Step S610, allocating to a home node;

[0164] Step S612, screening candidate nodes;

[0165] Specifically, candidate nodes are screened according to the remaining resource capacity and performance level of the non-attribute nodes.

[0166] Step S614, calculating the allocation probability;

[0167] Specifically, by converting the resource utilization corresponding to each resource in the candidate node into an occupancy ratio, the resource utilization entropy value of each candidate node is calculated using a preset information entropy formula, so as to obtain the allocation probability of the candidate node based on the resource utilization entropy value.

[0168] Step S616: Select the node with the highest allocation probability and store the data;

[0169] Specifically, the optimal non-home node is selected based on the allocation probability to write data and update the metadata record.

[0170] Step S618: monitor resource status.

[0171] Specifically, it continuously monitors the status of nodes, including resource consumption and health, and triggers space recall or defragmentation processes when necessary to optimize resource utilization and system performance.

[0172] The above examples demonstrate not only how a distributed storage system handles storage requests but also how it intelligently makes decisions in the face of resource constraints and changing demand, ensuring efficient data storage and fast access while maintaining storage system stability and high utilization. By combining the principle of priority attribution, dynamic scoring, and sophisticated metadata management, a highly intelligent and adaptive resource allocation mechanism is achieved.

[0173] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0174] The embodiment of the present application further provides a storage resource allocation device, which is applied to a storage system including a storage node cluster, wherein the storage node cluster includes a first type node set and a second type node set, wherein a first type node in the first type node set is bound to at least one service type; Figure 7 As shown, the device includes:

[0175] Determining module 702 is configured to, in response to an acquired storage request, determine a group of candidate nodes from the second-category node set if no first-category node corresponding to the storage request exists in the first-category node set, wherein the storage request is for requesting storage of business data, and the first-category node corresponding to the storage request is a first-category node that is bound to a business type corresponding to the business data and is in an available state;

[0176] a calculation module 704 for calculating an allocation probability of a candidate node in a set of candidate nodes based on a resource utilization parameter of the candidate node in the set of candidate nodes, wherein the resource utilization parameter of the candidate node in the set of candidate nodes is used to describe a resource utilization condition of the candidate node in the set of candidate nodes;

[0177] The first storage module 706 is configured to store the service data in a candidate node with the highest allocation probability among a group of candidate nodes.

[0178] In an exemplary embodiment, a storage request includes a requested business type and a requested storage capacity, and the above-mentioned device also includes: a second storage module, which is used to obtain a first specified capacity threshold when a target node corresponding to the storage request is determined in a first type of node set, wherein the first specified capacity threshold is determined based on the requested storage capacity and a first preset coefficient, and the target node is determined based on the requested business type; obtain a load score of the target node, wherein the load score is obtained based on a preset scoring model; when the remaining resource capacity of the target node is greater than or equal to the first specified capacity threshold and the load score of the target node is greater than the first preset scoring threshold, determine that the target node is the first type of node corresponding to the storage request, and store the business data in the first type of node corresponding to the storage request.

[0179] In an exemplary embodiment, the second storage module is also used to: collect indicator data of the target node, wherein the indicator data includes resource utilization data and performance data; obtain a set of weight factors, wherein the set of weight factors is determined based on specified status data of the storage node cluster within a specified time period; input the indicator data of the target node and the set of weight factors into the scoring model to obtain the load score of the target node.

[0180] In an exemplary embodiment, the second storage module is also used to: obtain specified status data of the storage node cluster within a specified time period, wherein the specified status data includes health status data and performance status data; input the specified status data into a pre-trained weight allocation model, and output a set of weight factors.

[0181] In an exemplary embodiment, the determination module 702 is also used to obtain the remaining resource capacity of the second type of node and the performance level of the second type of node, wherein the performance level is determined based on the type of storage medium of the second type of node; obtain a second specified capacity threshold, wherein the second specified capacity threshold is determined based on the requested storage capacity and a second preset coefficient, and the second specified capacity threshold is greater than the first specified capacity threshold; and select the second type of node in the second type of node set whose remaining resource capacity is greater than the second specified capacity threshold and whose performance level is greater than the preset level as a candidate node.

[0182] In one exemplary embodiment, the resource utilization parameters include utilization rates of different types of resources in the candidate node;

[0183] The calculation module 704 is also used to: standardize the utilization rates of different types of resources in a group of candidate nodes to obtain the occupancy ratios of different types of resources in a group of candidate nodes; and calculate the allocation probability of a group of candidate nodes based on the occupancy ratios of different types of resources in a group of candidate nodes.

[0184] In an exemplary embodiment, the calculation module 704 is also used to: calculate the resource utilization entropy value of the candidate node based on the occupancy ratio of different types of resources in the candidate nodes in a group of candidate nodes; perform negative exponential conversion on the resource utilization entropy value of the candidate node in a group of candidate nodes to obtain the negative entropy weight of the candidate node in a group of candidate nodes; normalize the negative entropy weight of the candidate node in a group of candidate nodes to obtain the allocation probability of the candidate node in a group of candidate nodes.

[0185] In an exemplary embodiment, a storage system includes a global metadata repository and multiple metadata sub-pools. A node in a storage node cluster is configured with at least one metadata sub-pool, and the at least one metadata sub-pool is used to record metadata corresponding to business data stored by the node. The above-mentioned device also includes: an adjustment module for monitoring the remaining storage space of metadata sub-pools in the multiple metadata sub-pools and the remaining storage space of the global metadata repository; and adjusting the storage space of the metadata sub-pools or the storage space of the global metadata repository based on the remaining storage space of the metadata sub-pools and the remaining storage space of the global metadata repository.

[0186] For the description of the features in the embodiment corresponding to the storage resource allocation device, reference can be made to the relevant description of the embodiment corresponding to the storage resource allocation method, which will not be repeated here.

[0187] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned storage resource allocation method embodiments.

[0188] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned storage resource allocation method embodiments when running.

[0189] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0190] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above-mentioned storage resource allocation method embodiments are implemented.

[0191] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned storage resource allocation method embodiments are implemented.

[0192] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0193] The above is a detailed introduction to a storage resource allocation method, storage medium, and electronic device provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only intended to help understand the method and core ideas of the present application. It should be pointed out that, for those skilled in the art, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A storage resource allocation method, characterized in that: Applied to a storage system including a storage node cluster, the storage node cluster including a first-type node set and a second-type node set, wherein a first-type node in the first-type node set is bound to at least one service type; The method comprises: In response to the obtained storage request, if there is no first-type node corresponding to the storage request in the first-type node set, determining a group of candidate nodes from the second-type node set, wherein the storage request is for requesting storage of business data, and the first-type node corresponding to the storage request is a first-type node that is bound to a business type corresponding to the business data and is in an available state; calculating, according to resource utilization parameters of the candidate nodes in the set of candidate nodes, an allocation probability of the candidate nodes in the set of candidate nodes, wherein the resource utilization parameters of the candidate nodes in the set of candidate nodes are used to describe resource utilization conditions of the candidate nodes in the set of candidate nodes; The service data is stored in a candidate node with the highest allocation probability among the group of candidate nodes.

2. The method according to claim 1, characterized in that The storage request includes a requested service type and a requested storage capacity, and the method further includes: When a target node corresponding to the storage request is determined in the first type of node set, obtaining a first specified capacity threshold, wherein the first specified capacity threshold is determined based on the requested storage capacity and a first preset coefficient, and the target node is determined based on the requested service type; Obtaining a load score of the target node, wherein the load score is obtained based on a preset scoring model; When the remaining resource capacity of the target node is greater than or equal to the first specified capacity threshold and the load score of the target node is greater than a first preset score threshold, the target node is determined to be the first type of node corresponding to the storage request, and the business data is stored in the first type of node corresponding to the storage request.

3. The method according to claim 2, characterized in that The obtaining of the load score of the target node includes: Collecting indicator data of the target node, wherein the indicator data includes resource utilization data and performance data; Obtaining a set of weight factors, wherein the set of weight factors is determined based on specified status data of the storage node cluster within a specified time period; The indicator data of the target node and the weight factor set are input into the scoring model to obtain the load score of the target node.

4. The method according to claim 3, characterized in that The obtaining of the weight factor set includes: Acquire the specified status data of the storage node cluster within the specified time period, wherein the specified status data includes health status data and performance status data; The specified state data is input into a pre-trained weight allocation model, and the weight factor set is output.

5. The method according to claim 1, wherein The step of determining a group of candidate nodes from the second type of node set includes: Acquire the remaining resource capacity of the second type of node and the performance level of the second type of node, wherein the performance level is determined based on the type of storage medium of the second type of node; Obtaining a second specified capacity threshold, wherein the second specified capacity threshold is determined based on the requested storage capacity and a second preset coefficient, and the second specified capacity threshold is greater than the first specified capacity threshold; The second-type nodes in the second-type node set whose remaining resource capacity is greater than the second specified capacity threshold and whose performance level is greater than a preset level are selected as candidate nodes.

6. The method according to claim 1, characterized in that The resource utilization parameters include utilization rates of different types of resources in the candidate nodes; The calculating, based on the resource utilization parameters of the candidate nodes in the group of candidate nodes, the allocation probability of the candidate nodes in the group of candidate nodes comprises: Normalizing utilization rates of different types of resources in the candidate nodes in the set of candidate nodes to obtain occupancy ratios of different types of resources in the candidate nodes in the set of candidate nodes; The allocation probability of the candidate nodes in the group of candidate nodes is calculated according to the occupation ratios of different types of resources in the candidate nodes in the group of candidate nodes.

7. The method according to claim 6, characterized in that The calculating, according to the occupation ratios of different types of resources in the candidate nodes in the group of candidate nodes, the allocation probability of the candidate nodes in the group of candidate nodes includes: Calculating a resource utilization entropy value of the candidate node according to an occupation ratio of different types of resources in the candidate node in the group of candidate nodes; Performing negative exponential conversion processing on the resources of the candidate nodes in the set of candidate nodes using entropy values ​​to obtain negative entropy weights of the candidate nodes in the set of candidate nodes; The negative entropy weights of the candidate nodes in the set of candidate nodes are normalized to obtain the allocation probabilities of the candidate nodes in the set of candidate nodes.

8. The method according to any one of claims 1 to 7, characterized in that The storage system includes a global metadata database and multiple metadata sub-pools, a storage node in the storage node cluster is configured with at least one metadata sub-pool, and the at least one metadata sub-pool is used to record metadata corresponding to business data stored by the storage node. The method further includes: monitoring remaining storage spaces of metadata sub-pools in the plurality of metadata sub-pools and remaining storage space of the global metadata database; The storage space of the metadata sub-pool or the storage space of the global metadata database is adjusted according to the remaining storage space of the metadata sub-pool and the remaining storage space of the global metadata database.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the storage resource allocation method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the storage resource allocation method according to any one of claims 1 to 8 are implemented.

Images