SDN-based Ceph heterogeneous distributed storage system and its read / write optimization method

Abstract

The application discloses a Ceph heterogeneous distributed storage system based on SDN and a read-write optimization method thereof, and first designs a cloud storage system model based on SDN technology; then classifies OSDs based on a node heterogeneous resource classification strategy; subsequently, network and load conditions are acquired in real time by using the SDN technology, and an OSD performance prediction model based on a random forest is established, so that the influence weight of a performance influence factor is obtained; finally, the read-write fitting degree of OSD is calculated by using a multi-attribute decision TOPSIS model in combination with OSD states and the influence weight, and the read-write performance of the cluster is adaptively optimized. In this way, the network state of the cluster node and the load balancing of the OSD on the node are considered, the read-write load can be adaptively concentrated on OSDs of different performance types, the load balancing of the heterogeneous OSD device is ensured, and therefore the reliability of the cluster is increased.

Description

Technical Field

[0001] This invention relates to the field of distributed storage technology, specifically to a Ceph heterogeneous distributed storage system based on SDN and its read / write optimization method. Background Technology

[0002] Ceph is an object-based distributed storage system. While it is widely used, the question of how to efficiently store and manage massive amounts of data has attracted attention from both industry and academia. Classic Ceph cloud storage systems only use the storage capacity of nodes as the sole criterion for selecting storage nodes, without considering network status, node heterogeneity, and load conditions. This node selection method affects the system's read and write performance under conditions of poor network performance and uneven load on heterogeneous nodes. Obtaining network status and node load information in traditional network architectures requires cumbersome configuration and significant measurement overhead. Summary of the Invention

[0003] The present invention aims to solve the read and write performance problem of Ceph distributed cloud storage system under conditions of poor network performance and uneven load of heterogeneous nodes, and provides an SDN-based Ceph heterogeneous distributed storage system and its read and write optimization method.

[0004] To solve the above problems, the present invention is achieved through the following technical solution:

[0005] A read / write optimization method for a Ceph heterogeneous distributed storage system based on SDN includes the following steps:

[0006] Step 1: Ceph storage nodes collect their own heterogeneous node information and upload it to the SDN controller for integration; the SDN controller sends the integrated system heterogeneous node information to the Ceph monitoring nodes; the Ceph monitoring nodes classify the object storage devices in the system based on the node heterogeneous resource classification strategy to obtain the final minimal set of object storage devices.

[0007] Step 2: The Ceph monitoring node sequentially changes the CRUSH weight values ​​of each object storage device and its class of object storage devices in the final object storage device minimal classification set according to the set weight change step size. During this process, each Ceph storage node in the system periodically collects its own CPU utilization and memory utilization, as well as the PG occupancy ratio of each object storage device, the amount of read I / O request data, and the amount of write I / O request data, and uploads them to the SDN controller. The SDN controller periodically collects the network bandwidth consumption of each Ceph storage node in the system.

[0008] Step 3: The SDN controller uses the network bandwidth consumption, CPU utilization, memory utilization, PG occupancy ratio, read I / O request data volume, and write I / O request data volume obtained from periodic sampling to construct read performance training data sample sets and write performance training data sample sets, and distributes them to the Ceph monitoring nodes.

[0009] For a single read performance training data sample in the read performance training data sample set, its input parameter matrix is ​​a matrix composed of the network bandwidth consumption, CPU utilization, memory utilization, PG occupancy ratio, and write I / O request data volume of all object storage devices obtained from one periodic sampling, and its output parameter vector is a vector composed of the read I / O request data volume of all object storage devices obtained from one periodic sampling.

[0010] For a single write performance training data sample in the write performance training data sample set, its input parameter matrix is ​​a matrix composed of the network bandwidth consumption, CPU utilization, memory utilization, PG occupancy ratio, and read I / O request data volume of all object storage devices obtained from one periodic sampling, and its output parameter vector is a vector composed of the write I / O request data volume of all object storage devices obtained from one periodic sampling.

[0011] Step 4: The Ceph monitoring node trains the random forest model using the read performance training data sample set and the write performance training data sample set respectively, and analyzes the importance of each parameter in the input parameter matrix of the read performance training data sample and the write performance training data sample based on the trained random forest model to obtain the read influence weight vector and the write influence weight vector.

[0012] Step 5: When the user needs to optimize read and / or write operations:

[0013] During read optimization, Ceph storage nodes collect the CPU utilization, memory utilization, PG occupancy ratio, and write I / O request data volume of each object storage device in the system at the current moment and upload them to the SDN controller. The SDN controller collects the remaining network bandwidth of each object storage device in the system at the current moment and integrates the remaining network bandwidth, CPU utilization, memory utilization, PG occupancy ratio, and write I / O request data volume into the read load information of the current moment and sends it to the Ceph monitoring node. The Ceph monitoring node first normalizes the read load information of the current moment and constructs a read decision matrix. Then, it feeds the read decision matrix and read influence weight vector into the multi-attribute decision TOPSIS model to obtain the read fit vector. Based on the read fit vector, the read data volume in the object storage device with low read fit is transferred to the object storage device with high read fit for execution.

[0014] During the write optimization process, Ceph storage nodes collect the CPU utilization, memory utilization, PG occupancy ratio, and read I / O request data volume of each object storage device in the system at the current moment, and upload them to the SDN controller. The SDN controller collects the remaining network bandwidth of each object storage device in the system at the current moment, and integrates the remaining network bandwidth, CPU utilization, memory utilization, PG occupancy ratio, and read I / O request data volume into the write load information of the current moment and sends it to the Ceph monitoring node. The Ceph monitoring node first normalizes the write load information of the current moment and constructs a read decision matrix. Then, it feeds the read decision matrix and read influence weight vector into the multi-attribute decision TOPSIS model to obtain the read fit vector. Based on the read fit vector, the read data volume in the object storage device with low read fit is transferred to the object storage device with high read fit for execution.

[0015] The specific process of the node heterogeneous resource classification strategy in step 1 above is as follows:

[0016] Step 1.1: Based on the heterogeneous performance of each object storage device in the system, construct the object storage device OSD. j heterogeneous performance set α j ={e1,e2,...,e I}; where e i Let i represent the i-th heterogeneous performance influence factor, 1≤i≤I, where I represents the number of heterogeneous performance influence factors;

[0017] Step 1.2: First, let the initial minimal performance set of object storage devices β = {α1}, and let the initial minimal classification set of object storage devices δ = {}; then, starting from the second object storage device, traverse the remaining object storage devices α. j′ heterogeneous performance set α j′ Where 2≤j′≤J: If the minimum performance set of the current object storage device is equal to that of the object storage device OSD j′ heterogeneous performance set α j′ If the union of the sets is not equal to the minimum performance set of the current object storage device, then the object storage device OSD will be... j′ heterogeneous performance set α j′ Merge into the current object storage device's minimal performance set, and combine the object storage device OSD j′ Merge into the current object storage device's minimal category set; otherwise, the current object storage device's minimal performance set and the current object storage device's minimal category set remain unchanged; after traversal, the final object storage device's minimal category set is obtained.

[0018] The aforementioned heterogeneous performance influencing factors include node bandwidth limit, CPU size, memory size, and object storage device type.

[0019] The specific process of step 2 above is as follows:

[0020] Step 2.1: Set the CRUSH weight value of each object storage device in the system to 1;

[0021] Each Ceph storage node in the system periodically collects its own CPU utilization and memory utilization, as well as the PG occupancy ratio of each object storage device, the amount of read I / O request data, and the amount of write I / O request data at a set time interval t and the number of collections N, and uploads them to the SDN controller; at the same time, the SDN controller periodically collects the network bandwidth consumption of each Ceph storage node in the system at a set time interval t and the number of collections N.

[0022] After N samplings, the SDN controller obtains the average cluster read performance across the N sampling processes. and average cluster write performance

[0023] Step 2.2: Loop through and select each object storage device in the final minimal object storage device category set;

[0024] Step 2.3: Set the CRUSH weight of the currently selected object storage device and its similar object storage devices to 1+is, and the CRUSH weight of the remaining object storage devices to 1; where i is the number of increments, i = 1, 2, ...; s is the set weight change step size;

[0025] Each Ceph storage node in the system periodically collects its own CPU utilization and memory utilization, as well as the PG occupancy ratio of each object storage device, the amount of read I / O request data, and the amount of write I / O request data at a set time interval t and the number of collections N, and uploads them to the SDN controller; at the same time, the SDN controller periodically collects the network bandwidth consumption of each Ceph storage node in the system at a set time interval t and the number of collections N.

[0026] After N samplings, the SDN controller obtains the average cluster read performance across the N sampling processes. and average cluster write performance

[0027] Step 2.4: The SDN controller determines the average read performance of the current cluster. Is it greater than the average cluster read performance of the last time? And the average write performance of this cluster Is it greater than the average cluster write performance of the last time?

[0028] If both conditions are met, that is and Proceed to step 2.2;

[0029] Otherwise, let i = i + 1 and return to step 2.3.

[0030] In step 5 above, in the read decision matrix, the remaining network bandwidth and the proportion of occupied PGs are positive, while the CPU utilization, memory utilization, and write I / O request data volume are negative; in the write decision matrix, the remaining network bandwidth and the proportion of occupied PGs are positive, while the CPU utilization, memory utilization, and read I / O request data volume are negative.

[0031] The SDN-based Ceph heterogeneous distributed storage system that implements the above read / write optimization methods consists of Ceph nodes, OpenFlow switches, and an SDN controller. Ceph nodes are located at the bottom layer of the system architecture; each Ceph node includes at least one Ceph storage node and at least one Ceph monitoring node, with topological connections between them. Each Ceph storage node contains at least one object storage device. The Ceph monitoring node maintains global configuration information for all Ceph nodes in the cluster. OpenFlow switches are located in the middle layer of the system architecture; they connect the SDN controller and Ceph nodes and are responsible for data transmission. The SDN controller is located at the top layer of the system architecture; it monitors the required object storage device information, and the Ceph monitoring node remotely invokes the information collected by the SDN controller to make decision-making references for selecting object storage devices on the Ceph storage nodes.

[0032] Compared with existing technologies, this invention first designs a cloud storage system model based on SDN (Software Defined Network) technology; then, it classifies OSDs (Object Storage Devices) based on a node heterogeneous resource classification strategy; subsequently, it uses SDN technology to acquire network and load information in real time and establishes the influence weights of performance impact factors based on random forest; finally, it combines OSD status and influence weights, and uses a multi-attribute decision TOPSIS model to calculate the read-write fit of OSDs, thereby adaptively optimizing cluster read-write performance. This approach considers both the network status of cluster nodes and the load balancing of OSDs on nodes, adaptively concentrating read-write loads on OSDs of different performance types and ensuring load balancing of heterogeneous OSD devices, thus increasing cluster reliability. Attached Figure Description

[0033] Figure 1 This is a Ceph heterogeneous distributed storage system architecture based on SDN. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific examples.

[0035] Software-defined networking (SDN), as a novel network model, primarily separates the control plane from the data plane, breaking down closed vertical architectures and introducing network programming capabilities. This enhances overall system management from a global perspective, significantly improving the efficiency of data transmission, control, and management in large-scale clustered storage. In traditional network architectures, obtaining network status and load information for monitoring and storage nodes in a distributed storage system requires cumbersome configuration and significant measurement overhead. Therefore, applying SDN-based network measurement technology to the network status information measurement of distributed storage systems allows for the acquisition of storage node bandwidth consumption and remaining bandwidth with less hardware configuration and measurement overhead. Furthermore, it facilitates convenient information exchange with underlying storage nodes via UDP packets. This network model enables us to monitor the overall network status and load information of the system, thereby optimizing the read and write performance of the Ceph cloud storage system using this information. To this end, this invention designs... Figure 1 The Ceph heterogeneous distributed storage system based on SDN shown consists of Ceph nodes, OpenFlow switches, and an SDN controller.

[0036] The system architecture is based on at least two interconnected Ceph nodes. Each Ceph node includes at least one Ceph storage node and at least one Ceph monitoring node, with the storage and monitoring nodes interconnected topologically. Each Ceph storage node can contain at least one OSD (Object Storage Device). The Ceph monitoring node maintains global configuration information for all nodes in the cluster. Figure 1 In the example shown, Ceph nodes 1, 3, and 4 are both Ceph storage nodes and Ceph monitoring nodes, while Ceph nodes 2, 5, and 6 are only Ceph storage nodes.

[0037] The middle layer of the system architecture consists of at least one OpenFlow switch. When there are two or more OpenFlow switches, they are interconnected in a topology. The OpenFlow switches connect upwards to the SDN controller and downwards to the Ceph nodes, responsible for data transmission.

[0038] The top layer of the system architecture consists of at least one SDN controller. When there are two or more SDN controllers, they are interconnected in a topology. The SDN controllers utilize a load balancing monitoring module to monitor necessary OSD information. Ceph monitoring nodes remotely access the information collected by the SDN controllers to make decisions regarding the selection of OSDs on Ceph storage nodes. This system architecture, by separating the control plane and data plane, facilitates monitoring of the underlying network and load conditions.

[0039] Based on the above system, the present invention proposes a read / write optimization method for a Ceph heterogeneous distributed storage system based on SDN, which includes the following steps:

[0040] (1) Reading and writing optimization preparation stage:

[0041] Step 1: First, the underlying Ceph storage nodes collect their own heterogeneous node information and upload it to the SDN controller for integration. Then, the SDN controller distributes the integrated heterogeneous node information to the Ceph monitoring nodes. Finally, the Ceph monitoring nodes classify the object storage devices in the system based on the node heterogeneous resource classification strategy, obtaining the final minimal set of object storage devices.

[0042] The specific process of the above-mentioned heterogeneous resource classification strategy for nodes is as follows:

[0043] Step 1.1: Based on the heterogeneous performance of each object storage device in the system, construct the object storage device OSD. j heterogeneous performance set α j ={e1,e2,...,e I}; where e i Let represent the i-th heterogeneous performance impact factor, 1 ≤ i ≤ I, where I represents the number of heterogeneous performance impact factors. The number of heterogeneous performance impact factors is selected according to performance requirements. In this embodiment, the heterogeneous performance impact factors include bandwidth limit (GB / s), CPU and memory size, and OSD type (HDD or SSD). This yields the heterogeneous performance set α = {α1, α2, ..., α...} of all object storage devices in the system. J}, 1≤j≤J, where J represents the total number of OSDs in the SDN-based Ceph heterogeneous distributed storage system.

[0044] Step 1.2: First, let the initial minimal performance set of object storage devices β = {α1}, and let the initial minimal classification set of object storage devices δ = {}; then, starting from the second object storage device, traverse the remaining object storage devices α. j′ heterogeneous performance set α j′ Where 2≤j′≤J: If the minimum performance set of the current object storage device is equal to that of the object storage device OSDj′ heterogeneous performance set α j′ The union of these sets is not equal to the minimum performance set of the current object storage device, i.e., β∪α j′ If ≠β, then the object storage device OSD will be used. j′ heterogeneous performance set α j′ The minimum performance set of the current object storage device is merged, i.e., β = β∪α. j′ and the object storage device OSD j′ Merge into the current object storage device's minimal category set, i.e., δ = δ∪osd j′ Otherwise, the current object storage device's minimal performance set β and the current object storage device's minimal classification set δ remain unchanged. After traversal, the final object storage device's minimal performance set β = {α1, α2, ..., α...} is obtained. l} and the final minimal set of object storage devices δ={osd1,osd2,...,osd l}, where l≤J, osd l It is an OSD heterogeneous performance set α l The corresponding OSD number.

[0045] The Ceph community, in its initial design, did not consider how to fully leverage the performance of heterogeneous storage nodes in a hybrid storage environment. This required domain experts to manually categorize heterogeneous resources to adapt to different storage performance needs, a time-consuming and labor-intensive process. However, the heterogeneous resource classification strategy proposed in this invention can automatically classify OSDs in the system, obtaining OSDs of different performance categories—that is, the simplest performance set and the minimum classification set—reducing the cost of manual editing. Furthermore, subsequent network status and load monitoring only needs to be performed on the OSDs in the minimum classification set, rather than on all OSDs in the system, reducing the overhead of OSD load monitoring. Finally, by finely dividing the cloud storage system's storage resource pools, the read and write performance of OSDs in each resource pool is ensured to be almost identical. This allows for the allocation of specific storage pools to corresponding clients based on performance requirements, thereby reducing resource waste in heterogeneous systems.

[0046] Step 2: The Ceph monitoring node sequentially changes the CRUSH weight values ​​of each object storage device and its class of object storage devices in the final object storage device minimal classification set according to the set weight change step size. During this process, each Ceph storage node in the system periodically collects its own CPU utilization and memory utilization, as well as the PG occupancy ratio of each object storage device, the amount of read I / O request data, and the amount of write I / O request data, and uploads them to the SDN controller. The SDN controller periodically collects the network bandwidth consumption of each Ceph storage node in the system.

[0047] Step 2.1: Set the CRUSH weight value of each object storage device in the system to 1.

[0048] Each Ceph storage node in the system periodically collects its own CPU utilization and memory utilization, as well as the PG occupancy ratio of each object storage device, the amount of read I / O request data, and the amount of write I / O request data at a set time interval t and a set number of collections N, and uploads this data to the SDN controller. Simultaneously, the SDN controller periodically collects the network bandwidth consumption of each Ceph storage node in the system at a set time interval t and a set number of collections N.

[0049] After N samplings, the SDN controller obtains the average cluster read performance across the N sampling processes. and average cluster write performance

[0050] Step 2.2: Loop through and select each object storage device in the final minimal object storage device category set;

[0051] Step 2.3: Set the CRUSH weight of the currently selected object storage device and its similar object storage devices to 1+is, and the CRUSH weight of the remaining object storage devices to 1. Where i is the number of times the step size increases, i = 1, 2, ...; s is the set weight change step size. In this invention, the value of N is in the range of [1, 10].

[0052] Each Ceph storage node in the system periodically collects its own CPU utilization and memory utilization, as well as the PG occupancy ratio of each object storage device, the amount of read I / O request data, and the amount of write I / O request data at a set time interval t and a set number of collections N, and uploads this data to the SDN controller. Simultaneously, the SDN controller periodically collects the network bandwidth consumption of each Ceph storage node in the system at a set time interval t and a set number of collections N.

[0053] After N samplings, the SDN controller obtains the average cluster read performance across the N sampling processes. and average cluster write performance

[0054] Step 2.4: The SDN controller determines the average read performance of the current cluster. Is it greater than the average cluster read performance of the last time? And the average write performance of this cluster Is it greater than the average cluster write performance of the last time?

[0055] If both conditions are met, that is and This indicates that the cluster's read and write performance will no longer increase when the CRUSH weight value of the currently selected object storage device and its similar object storage devices is further increased. Proceed to step 2.2, select the next object storage device in the final object storage device minimal category set, and continue until every object storage device in the final object storage device minimal category set has been traversed.

[0056] Otherwise, let i = i + 1 and return to step 2.3 until the read and write performance of the currently selected object storage device and its cluster of similar object storage devices no longer increases.

[0057] The Ceph cloud storage system uses the CRUSH algorithm (Controlled Replication Under Scalable Hashing, a pseudo-random data distribution algorithm based on hashing) to calculate the location of data storage. The original CRUSH weight value of an OSD is simply converted to its disk capacity. PGs (Placement Groups, a collection of data objects) are distributed on the OSD according to its CRUSH weight value; that is, the larger the CRUSH weight value of an OSD, the more PGs are distributed on it.

[0058] The average read and write performance of the cluster were obtained using Fio (version 3.7), a mainstream benchmarking tool. The Ceph monitoring node first uses built-in commands to map a disk from the cloud storage space and mount it on the Ceph storage node. Then, within a set test period, the Ceph storage node uses Fio to continuously test the read and write performance of the mapped disk and obtains the average value, which is the average read and write performance of the cluster. Finally, the Ceph storage node uploads the cluster read and write performance information to the SDN controller, which then uses the changes in cluster read and write performance as the termination condition for collecting selected OSD information.

[0059] Step 3: The SDN controller uses the network bandwidth consumption, CPU utilization, memory utilization, PG occupancy ratio, read I / O request data volume, and write I / O request data volume obtained from periodic sampling to construct read performance training data sample sets and write performance training data sample sets, and distributes them to the Ceph monitoring nodes.

[0060] Network bandwidth consumption, CPU utilization, and memory utilization are load information for Ceph storage nodes. The network bandwidth consumption, CPU utilization, and memory utilization of each object storage device are the network bandwidth consumption, CPU utilization, and memory utilization of the Ceph storage node to which that object storage device belongs. PG occupancy, read I / O request data volume, and write I / O request data volume are load information for object storage devices. The SDN controller uses the matrix composed of the bandwidth consumption (bw), CPU utilization (cpu), memory utilization (mem), PG occupancy (pgs), and write I / O request data volume (w_io) of all object storage devices obtained from each periodic sampling as the input parameter matrix for one read performance training data sample in the read performance training data sample set. Correspondingly, it uses the vector composed of the read I / O request data volume (r_io) of all object storage devices obtained from each periodic sampling as the output parameter vector for one read performance training data sample in the read performance training data sample set. The read performance training data sample set S... r for:

[0061] S r ={{consume r1 ,r_io r1};{consume r2 ,r_io r2};...;{consume rP ,r_io rP}}

[0062] consume rp ={bw rp CPU rp mem rp pgs rp ,w_io rp}, p∈[1,P r ]

[0063] The SDN controller uses a matrix composed of the bandwidth consumption (bw), CPU utilization (cpu), memory utilization (mem), PG usage ratio (pgs), and read I / O request data volume (r_io) of all object storage devices obtained from each periodic sampling as the input parameter matrix for one write performance training data sample in the write performance training data sample set. Correspondingly, it uses a vector composed of the write I / O request data volume (w_io) of all object storage devices obtained from each periodic sampling as the output parameter vector for one write performance training data sample in the write performance training data sample set. The write performance training data sample set S... w for:

[0064] S w ={{consume w1 ,w_iow1};{consume w2 ,w_io w2};...;{consume wP ,w_io wP}}

[0065] consume wp ={bw wp CPU wp mem wp pgs wp ,r_io wp}, p∈[1,P w ]

[0066] Since the purpose of this invention in collecting OSD load information is to establish an OSD performance prediction model and obtain the influence weights of performance factors, the system uses the OSD resource consumption set... rp and consume wp Establishing a performance prediction model and using the OSD resource surplus set to optimize decision-making has two advantages: 1. It normalizes the resource consumption of multiple clients on the same OSD into a total resource consumption, such as normalizing read / write request data volume into total read request data volume and total write request data volume, in kb / s. This method provides performance factor influence weights with better adaptability, masking the differences in performance prediction models caused by varying client resource consumption; 2. Using the OSD resource surplus set for decision-making is more conducive to cluster load balancing.

[0067] Step 4: The Ceph monitoring node trains the random forest model using the read performance training data sample set and the write performance training data sample set respectively. Based on the trained random forest model, it analyzes the importance of each parameter in the input parameter matrix of the read performance training data sample and the write performance training data sample respectively, and obtains the read influence weight vector and the write influence weight vector.

[0068] The random forest model is trained using the read performance training data sample set. The trained random forest model (read-based OSD prediction model) is then used to analyze the importance of each parameter in the input parameter matrix of the read performance training data sample, namely bandwidth consumption, CPU utilization, memory utilization, PG occupation ratio, and write I / O request data volume. This yields a read influence weight vector composed of the influence weights of bandwidth consumption, CPU utilization, memory utilization, PG occupation ratio, and write I / O request data volume.

[0069] The random forest model is trained using the write performance training data sample set. The trained random forest model (the OSD prediction model based on write performance) is then used to analyze the importance of each parameter in the input parameter matrix of the write performance training data sample, namely bandwidth consumption, CPU utilization, memory utilization, PG occupation ratio, and read I / O request data volume. This results in a write influence weight vector composed of the influence weights of bandwidth consumption, CPU utilization, memory utilization, PG occupation ratio, and read I / O request data volume.

[0070] Based on the OSD prediction model rf_reg trained using the random forest prediction model, the built-in function rf_reg.feature_importances_ of the random forest model can be used to perform feature importance analysis on the parameters of each OSD performance influencing factor in the model's input data, thereby obtaining the influence weights of each performance factor. For reading the influence weight vector W... r The influence weight W of the remaining bandwidth Br The impact weight of CPU utilization W Cr The impact weight of memory utilization W Mr The influence weight of the proportion of PG possession W Pr The weight W of the impact of the amount of data written I / O requests Lr Composed of, namely W r =[W Br W Cr W Mr W Pr W Lr For writing the influence weight vector W w The influence weight W of the remaining bandwidth Bw The impact weight of CPU utilization W Cw The impact weight of memory utilization W Mw The influence weight of the proportion of PG possession W Pw The weight W of the impact of the amount of data written I / O requests Lw Composed of, namely W w =[W Bw W Cw W Mw W Pw W Lw ].

[0071] This invention first divides the resource consumption set of OSDs into read performance training sets and write performance training sets. This facilitates the analysis of the non-linear influence weight relationship between system read performance factors and write performance factors, thereby enabling appropriate system read and write optimization decisions. When optimizing system read performance, the influence weight of different resources consumed by OSDs varies. For example, HDD-type OSDs consume significantly more bandwidth resources when performing read operations, approximately three times that of SSD-type OSDs. Similarly, when optimizing system write performance, the influence weight of different resources consumed by OSDs also varies. For instance, SSD-type OSDs consume significantly more bandwidth resources when performing write operations, approximately twice that of HDD-type OSDs. This weighting relationship is beneficial for targeted and quantitative increases in resources for specific types of object storage devices in actual production environments, thereby increasing cluster reliability.

[0072] (2) Actual read and / or write optimization phase:

[0073] Step 5: When a user needs to optimize read and / or write operations, they send read and / or write optimization commands to the Ceph monitoring node. The Ceph monitoring node then triggers the Ceph storage node and SDN controller to collect load information based on these commands. For example, if a user is sensitive to system read performance latency, read optimization is needed; if a user is sensitive to system write performance latency, write optimization is needed. Read and write optimizations can be performed separately or simultaneously, which can be achieved using a pointer flag: for example, flag 0 optimizes reads; flag 1 optimizes writes; and flag 2 optimizes both reads and writes simultaneously.

[0074] During read optimization, Ceph storage nodes collect the CPU utilization, memory utilization, PG usage ratio, and write I / O request data volume of each object storage device in the system at the current moment, and upload them to the SDN controller; the SDN controller collects the remaining network bandwidth B of each object storage device in the system at the current moment. r And will include the network remaining bandwidth information B r CPU utilization C r Memory utilization M r , Percentage of PG r and write I / O request data volume L r The current read load information is integrated and sent to the Ceph monitoring node; the Ceph monitoring node first normalizes the current read load information and then constructs the read decision matrix X. r Then, the read decision matrix and read influence weight vector are fed into the multi-attribute decision TOPSIS model to obtain the read fit vector. Based on this read fit vector, the amount of read data in the object storage device with low read fit is transferred to the object storage device with high read fit for execution.

[0075] Reading decision matrix Xr for:

[0076]

[0077] Read the influence weight vector W r for:

[0078] W r =[W Br W Cr W Mr W Pr W Lr ]

[0079] In step 2, this invention collects the bandwidth consumed by the object storage device as a training parameter affecting its performance, while during read optimization decision-making, it uses the remaining bandwidth of the object storage device as the read decision matrix X. r The input parameters are as follows. This is because when obtaining the influence weights of the bandwidth performance factor for read requests on object storage devices, the random forest model needs to input the actual bandwidth resources consumed as the training set to output the corresponding performance factor influence weights, and then use the remaining bandwidth resources of the object storage device and the bandwidth performance factor influence weights to make read optimization decisions.

[0080] Read the weighted decision matrix Z r for:

[0081] Z r =W r ×X r

[0082] Read the positive ideal solution Z of the weighted decision matrix r + for:

[0083] Z r + =max{Z r_ij |i=1,2,...,n;j=1,2,...,5}

[0084] Read the positive ideal solution Z of the weighted decision matrix r - for:

[0085] Z r + =min{Z r_ij |i=1,2,...,n; j=1,2,...,5}

[0086] Distance from each object storage device to the ideal solution for reading for:

[0087]

[0088] Distance from each object storage device to the read negative ideal solution for:

[0089]

[0090] Read proximity of each object storage device for:

[0091]

[0092] Where n is the number of object storage devices in the system.

[0093] During the optimization process, Ceph storage nodes collect the CPU utilization and memory utilization of each object storage device in the system at the current moment. r The system calculates the percentage of PGs occupied and the amount of read I / O request data, and uploads this data to the SDN controller. The SDN controller then collects the remaining network bandwidth (B) of each object storage device in the system at the current moment. r and the remaining network bandwidth B r CPU utilization C r Memory utilization M r , Percentage of PG r and the amount of data requested for read I / O w The write load information at the current moment is integrated and sent to the Ceph monitoring node; the Ceph monitoring node first normalizes the write load information at the current moment and then constructs the read decision matrix X. w Then, the read decision matrix and read influence weight vector are fed into the multi-attribute decision TOPSIS model to obtain the read fit vector. Based on this read fit vector, the amount of read data in the object storage device with low read fit is transferred to the object storage device with high read fit for execution.

[0094] Write the decision matrix X w for:

[0095]

[0096] Write the influence weight vector W w for:

[0097] W w =[W Bw W Cw W Mw W Pw W Lw ]

[0098] In step 2, this invention collects the bandwidth consumed by the object storage device as a training parameter affecting its performance, while during write optimization decision-making, it uses the remaining bandwidth of the object storage device as the write decision matrix X. wThe input parameters are as follows. This is because when obtaining the influence weights of the bandwidth performance factor for write requests to object storage devices, the random forest model needs to input the actual bandwidth resources consumed as the training set to output the corresponding performance factor influence weights, and then use the remaining bandwidth resources of the object storage device and the bandwidth performance factor influence weights to make write optimization decisions.

[0099] Write the weighted decision matrix Z w for:

[0100] Z w =W w ×X w

[0101] Write the positive ideal solution Z of the weighted decision matrix. w + for:

[0102] Z w + =max{Z w_ij |i=1,2,...,n; j=1,2,...,5}

[0103] Write the positive ideal solution Z of the weighted decision matrix. r - for:

[0104] Z w + =min{Z w_ij |i=1,2,...,n; j=1,2,...,5}

[0105] Distance from each object storage device to the ideal solution for:

[0106]

[0107] Distance from each object storage device to the write negative ideal solution for:

[0108]

[0109] Write proximity C per object storage device w_i + for:

[0110]

[0111] Where n is the number of object storage devices in the system.

[0112] The TOPSIS model incorporates predicted impact weights for performance-influenced factors, which score the OSDs currently providing services and then select one or a group of OSDs to optimize the read and write performance of the Ceph cluster. This approach better avoids localized optimization "hotspot" bottlenecks, considering global load balancing across the cluster, including load balancing between object storage devices of the same and different performance types, thus increasing system reliability. This builds upon the original OSD selection algorithm by considering five indicators of object storage device network performance and load as constraints for OSD weighting factors. By establishing and solving a multi-attribute decision model, the selection of one or a group of OSDs can be determined. Furthermore, after the system scores object storage devices based on real-time network and load conditions, users can periodically and quantitatively select the range of object storage devices to optimize based on actual needs, and decide which performance optimization to perform, making the entire optimization decision-making process more controllable.

[0113] The proximity (read proximity and write proximity) values ​​themselves express the proportion of load transfer (read load and write load), with both values ​​ranging from [0,1]. A larger proximity value indicates a smaller OSD load, allowing the system to concentrate more load on OSDs with higher proximity values ​​to optimize read and write performance. Conversely, a smaller proximity value indicates a larger OSD load, allowing the system to reduce load concentration on OSDs with lower proximity values ​​to optimize read and write performance. Based on proximity, the amount of read and write data transferred depends on the OSD's real-time network and load status; that is, the worse the OSD's performance, the less read and write data it should handle; conversely, the better the OSD's performance, the more read and write data it can handle.

[0114] It should be noted that although the embodiments described above are illustrative, they are not intended to limit the invention. Therefore, the invention is not limited to the specific embodiments described above. Any other embodiments obtained by those skilled in the art under the guidance of this invention without departing from its principles are considered to be within the protection scope of this invention.

Claims

1. A read-write optimization method for an SDN-based Ceph heterogeneous distributed storage system, characterized in that, The steps include the following: Step 1, the Ceph storage node collects the heterogeneous node information of itself and uploads it to the SDN controller for integration; The SDN controller distributes the integrated system heterogeneous node information to the Ceph monitoring node; the Ceph monitoring node classifies the object storage devices in the system based on the node heterogeneous resource classification strategy to obtain the final object storage device minimum classification set; Step 2, the Ceph monitoring node changes the CRUSH weight value of each object storage device in the final object storage device minimum classification set and its same type object storage device in turn with a set weight change step; in this process, each Ceph storage node in the system periodically collects its CPU utilization and memory utilization, and the PG proportion, read I / O request data volume and write I / O request data volume of each object storage device contained, and uploads them to the SDN controller; the SDN controller periodically collects the network consumption bandwidth of each Ceph storage node in the system; Step 3, the SDN controller uses the network consumption bandwidth, CPU utilization, memory utilization, PG proportion, read I / O request data volume and write I / O request data volume obtained by periodic sampling to construct a read performance training data sample set and a write performance training data sample set, and distributes them to the Ceph monitoring node; For a read performance training data sample of the read performance training data sample set, the input parameter matrix is a matrix composed of the network consumption bandwidth, CPU utilization, memory utilization, PG proportion and write I / O request data volume of all object storage devices obtained by one periodic sampling, and the output parameter vector is a vector composed of the read I / O request data volume of all object storage devices obtained by one periodic sampling; For a write performance training data sample of the write performance training data sample set, the input parameter matrix is a matrix composed of the network consumption bandwidth, CPU utilization, memory utilization, PG proportion and read I / O request data volume of all object storage devices obtained by one periodic sampling, and the output parameter vector is a vector composed of the write I / O request data volume of all object storage devices obtained by one periodic sampling; Step 4, the Ceph monitoring node trains the random forest model respectively using the read performance training data sample set and the write performance training data sample set, and analyzes the importance of each parameter in the input parameter matrix of the read performance training data sample and the write performance training data sample based on the trained random forest model respectively to obtain the read influence weight vector and the write influence weight vector; Step 5, when the user needs to perform read and / or write optimization: In the read optimization process, the Ceph storage node collects the CPU utilization, memory utilization, PG proportion and write I / O request data volume of each object storage device in the system at the current time, and uploads them to the SDN controller; the SDN controller collects the network residual bandwidth of each object storage device in the system at the current time, and integrates the network residual bandwidth, CPU utilization, memory utilization, PG proportion and write I / O request data volume into the read load information at the current time, and sends it to the Ceph monitoring node; the Ceph monitoring node first normalizes the read load information at the current time to construct a read decision matrix, and then sends the read decision matrix and the read influence weight vector into the multi-attribute decision TOPSIS model to obtain a read fitting degree vector, and then based on the read fitting degree vector, the read data volume in the object storage device with low read fitting degree is transferred to the object storage device with high read fitting degree for execution; In the write optimization process, the Ceph storage node collects the CPU utilization, memory utilization, PG proportion and read I / O request data volume of each object storage device in the system at the current time, and uploads them to the SDN controller; the SDN controller collects the network residual bandwidth of each object storage device in the system at the current time, and integrates the network residual bandwidth, CPU utilization, memory utilization, PG proportion and read I / O request data volume into the write load information at the current time, and sends it to the Ceph monitoring node; the Ceph monitoring node first normalizes the write load information at the current time to construct a read decision matrix, and then sends the read decision matrix and the read influence weight vector into the multi-attribute decision TOPSIS model to obtain a read fitting degree vector, and then based on the read fitting degree vector, the read data volume in the object storage device with low read fitting degree is transferred to the object storage device with high read fitting degree for execution.

2. The read-write optimization method for the SDN-based Ceph heterogeneous distributed storage system according to claim 1, characterized in that, The specific process of the node heterogeneous resource classification strategy in step 1 is as follows: Step 1.1: Based on the heterogeneous performance of each object storage device in the system, construct the object storage device OSD. j heterogeneous performance set α j ={e1,e2,...,e I }; where e i Let i represent the i-th heterogeneous performance influence factor, 1≤i≤I, where I represents the number of heterogeneous performance influence factors; Step 1.2: First, let the initial minimal performance set of object storage devices β = {α1}, and let the initial minimal classification set of object storage devices δ = {}; then, starting from the second object storage device, traverse the remaining object storage devices α. j′ heterogeneous performance set α j′ Where 2≤j′≤J: If the minimum performance set of the current object storage device is equal to that of the object storage device OSD j′ heterogeneous performance set α j′ If the union of the sets is not equal to the minimum performance set of the current object storage device, then the object storage device OSD will be... j′ heterogeneous performance set α j′ Merge into the current object storage device's minimal performance set, and combine the object storage device OSD j′ Merge into the current object storage device's minimum category set; Otherwise, the current object storage device minimum performance set and the current object storage device minimum classification set remain unchanged; After the traversal is completed, the final object storage device minimum classification set is obtained.

3. The read-write optimization method for the SDN-based Ceph heterogeneous distributed storage system according to claim 2, characterized in that, The heterogeneous performance influence factors include node bandwidth upper limit, CPU size, memory size and object storage device type.

4. The read-write optimization method for the SDN-based Ceph heterogeneous distributed storage system according to claim 1, characterized in that, The specific process of step 2 is as follows: Step 2.1, let the CRUSH weight value of each object storage device in the system be 1; The Ceph storage nodes in the system periodically collect their own CPU utilization and memory utilization, as well as the PG proportion, read I / O request data volume and write I / O request data volume of each object storage device contained therein at a set time interval t and collection times N, and upload them to the SDN controller; at the same time, the SDN controller periodically collects the network consumption bandwidth of each Ceph storage node in the system at a set time interval t and collection times N; After N times of sampling, the SDN controller obtains the average of the cluster read performance in the N times of collection processes and the average of the cluster write performance Step 2.2, loop through each object storage device in the final object storage device minimum classification set; Step 2.3, let the CRUSH weight value of the current selected object storage device and its similar object storage devices be 1+is, and the CRUSH weight value of the remaining object storage devices be 1; Where i is the step increase number, i = 1, 2, …; s is the set weight change step; The Ceph storage nodes in the system periodically collect their own CPU utilization and memory utilization, and the PG proportion, read I / O request data volume and write I / O request data volume of each object storage device contained therein at a set time interval t and collection number N, and upload them to the SDN controller; at the same time, the SDN controller periodically collects the network consumption bandwidth of each Ceph storage node in the system at a set time interval t and collection number N; After N times of sampling, the SDN controller obtains the average of the cluster read performance in the N times of collection processes and the average of the cluster write performance Step 2.4, the SDN controller judges whether the average value of the current cluster read performance is greater than the average value of the last cluster read performance Step 2.5, the SDN controller judges whether the average value of the current cluster write performance is greater than the average value of the last cluster write performance Step 2.6, the SDN controller judges whether the average value of the current cluster read performance is greater than the average value of the last cluster read performance Step 2.7, the SDN controller judges whether the average value of the current cluster write performance is greater than the average value of the last cluster write performance If both are true, i.e. and then go to step 2.

2. Otherwise, let i = i + 1, and return to step 2.

3.

5. The read-write optimization method for the SDN-based Ceph heterogeneous distributed storage system according to claim 1, characterized in that, In step 5, in the read decision matrix, the network residual bandwidth and the PG proportion are positive values, and the CPU utilization, memory utilization and write I / O request data volume are negative values; In the write decision matrix, the network residual bandwidth and the PG proportion are positive values, and the CPU utilization, memory utilization and read I / O request data volume are negative values.

6. The SDN-based Ceph heterogeneous distributed storage system implementing the read-write optimization method of claim 1, characterized in that, The Ceph node, the OpenFlow switch and the SDN controller are composed of; The Ceph node is located at the bottom layer of the system architecture; the Ceph node includes at least one Ceph storage node and at least one Ceph monitoring node, and the Ceph storage node and the Ceph monitoring node are topologically connected with each other; each Ceph storage node contains at least one object storage device; the Ceph monitoring node maintains the global configuration information of all Ceph nodes in the cluster; The OpenFlow switch is located at the middle layer of the system architecture; the OpenFlow switch connects the SDN controller and the Ceph node, and is responsible for the transmission between data; The SDN controller is located at the top layer of the system architecture; The SDN controller monitors the required object storage device information, and the Ceph monitoring node remotely calls the information collected by the SDN controller to make a decision reference for selecting the object storage device on the Ceph storage node.

Images