Ceph cluster performance tuning method, device, equipment, medium and product
By constructing a fitness function and using a genetic algorithm to optimize the configuration parameters of the Ceph cluster, the problems of low tuning efficiency and high resource cost in existing technologies are solved, and efficient and accurate Ceph cluster performance tuning is achieved.
Patent Information
- Application Number
- CN202610151273.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-02-03
- Publication Date
- 2026-05-01
AI Technical Summary
Current technologies for Ceph cluster performance tuning are inefficient, costly in terms of computing resources and time, and rely heavily on human experience and modeling.
By constructing a fitness function, a genetic algorithm is used to optimize the configuration parameters of the Ceph cluster, including read/write IOPS, throughput, read/write latency, and error rate. The target configuration parameter scheme is automatically searched, reducing reliance on human experience and improving tuning efficiency and accuracy.
It enables efficient and accurate Ceph cluster performance tuning, reduces computing resources and time costs, and improves the automation of the tuning process and the robustness of the results.
Smart Images
Figure CN121957495A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a Ceph cluster performance tuning method, apparatus, equipment, medium and product. Background Technology
[0002] Ceph clusters are unified distributed storage systems. Currently, Ceph cluster performance tuning is mainly achieved by adjusting cluster configuration parameters. The mainstream parameter optimization methods include empirical tuning and reinforcement learning tuning. However, empirical tuning methods often rely on trial and error adjustments based on human experience, resulting in low tuning efficiency; reinforcement learning tuning methods require system modeling, which incurs high computational resources and time costs. Summary of the Invention
[0003] This application provides a Ceph cluster performance tuning method, apparatus, equipment, media, and product to solve the problems of low tuning efficiency and high computing resource and time costs in the prior art.
[0004] To achieve the above objectives, this application provides a Ceph cluster performance tuning method, including: The configuration parameter scheme of the Ceph cluster is treated as an individual to build an initial population; A fitness function is constructed based on the performance metrics of the Ceph cluster; wherein the performance metrics include at least one of the following: read / write IOPS, throughput, read / write latency, and error rate; Calculate the fitness of each individual in the current population based on the fitness function; Genetic operations are performed on the current population to generate a new population; Determine whether the iteration termination condition is met. If yes, output the target configuration parameter scheme of the Ceph cluster. If no, use the new population to calculate fitness.
[0005] As an improvement to the above scheme, the step of constructing a fitness function based on the performance metrics of the Ceph cluster includes: The performance metrics are normalized; The fitness function is constructed using the normalized performance metrics; wherein the read / write IOPS and the throughput are positively correlated with the fitness, and the read / write latency and the error rate are negatively correlated with the fitness.
[0006] As an improvement to the above scheme, after constructing an initial population by treating the Ceph cluster configuration parameter scheme as individuals, the Ceph cluster performance tuning method further includes: Individuals in the current population are assigned to different datasets; wherein, the computing nodes of each different dataset are used to perform fitness calculations and generate new populations in parallel using their own datasets.
[0007] As an improvement to the above scheme, the step of determining whether the iteration termination condition is met, if yes, outputs the target configuration parameter scheme of the Ceph cluster; if no, the fitness calculation is performed using the new population, including: The computing nodes of the different datasets aggregate the new populations they generate into the same dataset; The computing nodes of the same dataset determine whether the iteration termination condition is met. If yes, the population with the highest fitness is output as the target configuration parameter scheme. If no, the population that meets the preset fitness condition is selected from the new population as the current population, and the process returns to the step of calculating the fitness of each individual in the current population according to the fitness function.
[0008] As an improvement to the above scheme, the step of performing genetic operations on the current population to generate a new population includes: Multiple pairs of parent individuals are selected from the current population using the roulette wheel selection method. Cross over the multiple pairs of parent individuals to obtain multiple pairs of crossed individuals; The individuals after the multiple crossovers are mutated to obtain multiple pairs of mutated individuals, which are then merged into the current population.
[0009] As an improvement to the above scheme, before constructing the initial population by treating the Ceph cluster configuration parameter scheme as an individual, the Ceph cluster performance tuning method further includes: Deploy the software environment for the Ceph cluster and perform streaming operations on the Ceph cluster; Configure Prometheus to monitor and collect performance metrics of the Ceph cluster.
[0010] To achieve the above objectives, this application also provides a Ceph cluster performance tuning device, comprising: The population building module is used to build an initial population by taking the configuration parameter scheme of the Ceph cluster as individuals; The function construction module is used to construct a fitness function based on the performance metrics of the Ceph cluster; wherein the performance metrics include at least one of the following: read / write IOPS, throughput, read / write latency, and error rate; The calculation module is used to calculate the fitness of each individual in the current population according to the fitness function; The genetic module is used to perform genetic operations on the current population to generate a new population. The judgment module is used to determine whether the iteration termination condition is met. If yes, it outputs the target configuration parameter scheme of the Ceph cluster; otherwise, it uses the new population to calculate fitness.
[0011] To achieve the above objectives, this application also provides a Ceph cluster performance tuning device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the Ceph cluster performance tuning method as described above.
[0012] To achieve the above objectives, embodiments of this application also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the Ceph cluster performance tuning method as described above.
[0013] To achieve the above objectives, embodiments of this application also provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the Ceph cluster performance tuning method as described above.
[0014] Compared with existing technologies, the Ceph cluster performance tuning method, apparatus, device, medium, and product provided in this application embodiment constructs an initial population by treating the Ceph cluster configuration parameter scheme as individuals; constructs a fitness function based on the performance indicators of the Ceph cluster; wherein the performance indicators include at least one of the following: read / write IOPS, throughput, read / write latency, and error rate; calculates the fitness of each individual in the current population according to the fitness function; performs genetic operations on the current population to generate a new population; determines whether the iteration termination condition is met; if so, outputs the target configuration parameter scheme of the Ceph cluster; otherwise, uses the new population to calculate fitness. Therefore, this application embodiment automatically searches for the target configuration parameter scheme of the Ceph cluster through fitness function-driven optimization, reducing reliance on human experience and the workload of manual tuning, improving the efficiency and accuracy of the tuning process, and eliminating the need for system modeling, thus greatly reducing computational resources and time costs. Attached Figure Description
[0015] Figure 1 This is a flowchart of a Ceph cluster performance tuning method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a cross-operation provided in an embodiment of this application; Figure 3This is a structural block diagram of a Ceph cluster performance tuning device provided in an embodiment of this application; Figure 4 This is a structural block diagram of a Ceph cluster performance tuning device provided in an embodiment of this application. Detailed Implementation
[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0017] In the description of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0018] In this application description, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.
[0019] In this application description, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The term "based on" means "at least partially based on." The term "according to" means "at least partially according to." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments." The term "and / or" means at least one of the connected objects, such as A and / or B, indicating three cases: including only A, only B, and both A and B. Unless otherwise stated, the term "multiple" means two or more.
[0020] See Figure 1, Figure 1 This is a flowchart of a Ceph cluster performance tuning method provided in an embodiment of this application. The Ceph cluster performance tuning method includes: S1. Treat the Ceph cluster configuration parameter scheme as an individual and build an initial population; It is worth noting that in this embodiment of the application, the configuration parameter scheme of the Ceph cluster is used as an individual to construct an initial population. That is, each individual is a configuration parameter scheme of the Ceph cluster, which includes at least one configuration parameter of the Ceph cluster. Multiple configuration parameter schemes are initialized to form an initial population.
[0021] Taking BlueStore (BlueStore is a storage engine in Ceph) configuration parameters as an example, the main configuration parameters are as follows: bluestore_cache_size_hdd and bluestore_cache_size_ssd represent the cache size occupied by hard disk drives (HDDs) and solid-state drives (SSDs), respectively; bluestore_cache_kv_ratio and bluestore_cache_meta_ratio represent the proportion of key-value data and metadata in the BlueStore cache, respectively; bluestore_max_blob_size and bluestore_min_alloc_size represent the maximum and minimum allocated space size of binary large objects, respectively, determining the efficiency of storage and retrieval. bluestore_prefer_deferred_size controls the write operation in BlueStore to delay write operations that are less than or equal to this threshold. By optimizing the above configuration parameters and selecting appropriate values, the target configuration parameter scheme can be obtained, which can significantly improve the read and write performance of the Ceph cluster. The specific configuration parameters are shown in Table 1 below: Table 1
[0022] Furthermore, based on the default or empirical values, initial values are assigned to each configuration parameter within the configuration parameter scheme, and these initial values are combined into multiple (e.g., 50) initial configuration parameter schemes (each configuration parameter scheme is an individual). Each individual is a potential solution in the solution space; for example, each configuration parameter scheme includes 7 configuration parameters, and a single individual config... i =(cfg1,cfg2,...,cfg7); where cfg1,cfg2,...,cfg7 are the respective configuration parameters.
[0023] S2. Construct a fitness function based on the performance metrics of the Ceph cluster; wherein the performance metrics include at least one of the following: read / write IOPS, throughput, read / write latency, and error rate; This application embodiment utilizes the read / write IOPS, throughput, read / write latency, and error rate of the Ceph cluster to construct the fitness function.
[0024] Read / write IOPS refers to the number of read and write operations per second within a Ceph cluster, specifically the average read / write IOPS. Throughput refers to the amount of data that a Ceph cluster processes or transmits within a certain period of time.
[0025] Read / write latency refers to the time it takes for a Ceph cluster to process a single request, that is, the total time from when the client sends a request to the Ceph cluster until the cluster completes the request processing and returns the response result to the client; Error rate refers to the proportion of requests that result in processing errors to the total number of requests in a Ceph cluster within a certain period of time.
[0026] S3. Calculate the fitness of each individual in the current population according to the fitness function; This application's embodiments substitute the collected performance metrics into a fitness function to calculate the fitness of each individual in the current population. This function quantifies the impact of configuration parameters on cluster performance, including key metrics such as read / write IOPS, throughput, read / write latency, and error rate. This not only more accurately identifies high-performance configuration parameter schemes, thus guiding the optimization process in the right direction, but also reduces dependence on specific environments, improving the applicability and scalability of this application's embodiments. The introduction of this function plays a decisive role in the accuracy and effectiveness of Ceph cluster performance tuning.
[0027] Specifically, if it is the first time, the current population is the initialized population; if it is not the first time, the current population is a new population, or a population selected from the new population.
[0028] S4. Perform genetic operations on the current population to generate a new population; The embodiments of this application perform genetic operations on the current population to generate a new population, ensuring the priority inheritance of excellent individuals, while increasing population diversity and avoiding the problem of premature convergence.
[0029] S5. Determine whether the iteration termination condition is met. If yes, output the target configuration parameter scheme of the Ceph cluster. If no, use the new population to calculate fitness.
[0030] In this embodiment, when the iteration termination condition is met, the target configuration parameter scheme of the Ceph cluster is output. This target configuration parameter scheme is the optimal configuration parameter access, which is applied to the actual Ceph cluster to improve system performance. If the iteration termination condition is not met, the fitness calculation is performed using a new population to continue the iteration until the iteration termination condition is met, at which point the target configuration parameter scheme of the Ceph cluster is output.
[0031] This application embodiment uses a fitness function-driven approach to automatically search for target configuration parameter schemes for Ceph clusters, reducing reliance on human experience and the workload of manual tuning, improving the efficiency and accuracy of the tuning process, and eliminating the need for system modeling, thus greatly reducing computing resources and time costs.
[0032] In one optional embodiment, constructing the fitness function based on the performance metrics of the Ceph cluster includes: The performance metrics are normalized; The fitness function is constructed using the normalized performance metrics; wherein the read / write IOPS and the throughput are positively correlated with the fitness, and the read / write latency and the error rate are negatively correlated with the fitness.
[0033] In one specific embodiment, the fitness function is constructed as follows:
[0034] Where α, β, and μ are the normalization coefficients for average read / write IOPS, throughput, and read / write latency, respectively; n is the average read / write IOPS; w is the throughput; t is the read / write latency; and r is the error rate. This function evaluates the performance of configuration parameters based on these performance metrics. Higher average read / write IOPS, higher throughput, lower read / write latency, and lower error rate result in higher fitness, indicating better cluster performance. However, the fitness function is not limited to this formula.
[0035] In an optional embodiment, after constructing an initial population by treating the Ceph cluster configuration parameter scheme as individuals, the Ceph cluster performance tuning method further includes: Individuals in the current population are assigned to different datasets; wherein, the computing nodes of each different dataset are used to perform fitness calculations and generate new populations in parallel using their own datasets.
[0036] This application fully utilizes the distributed computing capabilities of the Ceph cluster. By distributing individuals of the current population to different datasets, the computing nodes of these datasets can perform fitness calculations and new population generation in parallel using their respective datasets. This not only significantly improves search efficiency but also reduces the risk of getting trapped in local optima, enabling rapid convergence to a better solution and improving robustness. Fitness calculation refers to step S3, and new population generation refers to step S4.
[0037] Specifically, individuals in the current population are evenly distributed to different datasets. For example, based on the number of datasets m, individuals in the current population are evenly divided into m subpopulations, and the m subpopulations are distributed to m datasets, so that the subpopulation size of each computing node is consistent and the data processing load is balanced.
[0038] In one optional embodiment, the determination of whether the iteration termination condition is met, if yes, outputs the target configuration parameter scheme of the Ceph cluster; if no, uses the new population to calculate fitness, including: The computing nodes of the different datasets aggregate the new populations they generate into the same dataset; The computing nodes of the same dataset determine whether the iteration termination condition is met. If yes, the population with the highest fitness is output as the target configuration parameter scheme. If no, the population that meets the preset fitness condition is selected from the new population as the current population, and the process returns to the step of calculating the fitness of each individual in the current population according to the fitness function.
[0039] In one specific embodiment, the number of datasets is 5. Individuals in the current population are evenly distributed to each dataset, and each dataset corresponds to a computing node. The fitness calculation and the generation of the new population are performed in parallel by the computing nodes of the 5 datasets. Each dataset aggregates all new populations into the same dataset (e.g., aggregated into dataset 1), and the computing node of that dataset (e.g., dataset 1) determines whether the iteration termination condition is met. If it is met, the population with the highest fitness is output as the target configuration parameter scheme. If it is not met, a population that meets the preset fitness condition is selected from the new population as the current population. For example, a population with high fitness (the 50 individuals with the highest fitness) is selected as the current population, and the process returns to step S3 until the iteration termination condition is met.
[0040] In this embodiment, one dataset (e.g., dataset 1) is used as the main dataset, which is responsible for the allocation and management of the iterative tasks of configuration parameters, while the other datasets are used as auxiliary datasets to participate in the iterative process of configuration parameters, which effectively improves the efficiency of parameter iteration and the robustness of the results.
[0041] Specifically, the iteration termination condition includes the current iteration number being greater than or equal to the preset iteration number. The iteration number here is the global iteration number, which means that the global iteration number is incremented by 1 only after all participating computing nodes have completed a full round of core operations (including fitness calculation and new population generation). If only a single or partial dataset completes the operation, it is not counted in the global iteration number.
[0042] The preset fitness conditions include at least one of the following: fitness greater than a preset fitness threshold, the top q individuals with the highest fitness, where q is a positive integer, preferably the number of individuals in the initial population.
[0043] In one optional embodiment, performing genetic operations on the current population to generate a new population includes: Multiple pairs of parent individuals are selected from the current population using the roulette wheel selection method. Cross over the multiple pairs of parent individuals to obtain multiple pairs of crossed individuals; The individuals after the multiple crossovers are mutated to obtain multiple pairs of mutated individuals, which are then merged into the current population.
[0044] The embodiments of this application generate new populations through genetic operations such as selection, crossover, and mutation, ensuring the priority inheritance of superior individuals while increasing population diversity and avoiding the problem of premature convergence.
[0045] In one specific embodiment, the genetic operation is as follows: Selection operation: A roulette wheel selection algorithm is used to select 25 pairs of parent individuals for subsequent operations. The probability of each individual being selected is ps. i Due to its adaptability The selection probability is determined by the proportion of the total fitness of the k individuals in the dataset in this iteration. The specific selection probability formula is as follows:
[0046] Crossover operation: Crossover is performed between each pair of parent individuals obtained from the above selection operation to generate crossover individuals. Specifically, a uniform crossover method is used, with a base crossover probability pc = 0.5. For each gene of the parent individual, a random number within the range (0, 1) is selected as the crossover probability for that gene position. If the crossover probability of that gene position is greater than or equal to the base crossover probability, then the gene segments at that position in this pair of parent individuals are swapped; if the crossover probability of that gene position is less than the base crossover probability, no swapping occurs. A schematic diagram of the crossover operation is shown below. Figure 2As shown, a pair of parent individuals are parent1 and parent2. parent1 includes 1 to 5 gene loci, and parent2 includes a to e gene loci. The crossover probabilities of these gene loci are 0.3, 0.9, 0.4, 0.8, and 0.1, respectively, with a basic crossover probability of 0.5. The crossover probability of gene loci 2 and gene loci 4 is greater than 0.5. Therefore, gene loci 2 and gene loci b are swapped, and gene loci 4 and gene loci d are swapped to obtain the crossover individuals child1 and child2.
[0047] Mutation operation: Mutate individuals after crossover with a certain probability to increase population diversity and avoid premature convergence. For example, set the basic mutation probability pm=0.1, and take a random number in the range (0,1) as the mutation probability of the crossover individual. If the individual mutation probability value is less than the basic mutation probability 0.1, then randomly select a gene segment of the individual for gene rearrangement to obtain the mutated individual. If the individual mutation probability value is greater than or equal to the basic mutation probability value, then no mutation operation is performed.
[0048] In an optional embodiment, before constructing the initial population by treating the Ceph cluster configuration parameter scheme as individuals, the Ceph cluster performance tuning method further includes: Deploy the software environment for the Ceph cluster and perform streaming operations on the Ceph cluster; Configure Prometheus to monitor and collect performance metrics of the Ceph cluster.
[0049] This application provides a Ceph cluster deployment software environment to ensure the stable operation of the Ceph cluster and to perform streaming operations on the cluster, enabling precise capture of the cluster's performance.
[0050] By configuring Prometheus (an open-source system monitoring and alerting toolkit), we can ensure that it can collect Ceph cluster performance metrics in real time. This allows us to directly use the actual performance metrics of the system to iteratively optimize parameters, enabling us to respond to and resolve performance issues more quickly and accurately, and ensuring that parameter tuning improves in the right direction for performance optimization.
[0051] In practical applications, the computing nodes of each dataset apply their own configuration parameter schemes to the Ceph cluster, and use Prometheus to monitor the performance under this configuration parameter scheme in real time and collect performance indicators in real time. The real-time collected performance indicators are then substituted into the fitness function to obtain the fitness of each individual in the current population.
[0052] See Figure 3 , Figure 3This is a structural block diagram of a Ceph cluster performance tuning device 10 provided in an embodiment of this application. The Ceph cluster performance tuning device 10 includes: Population building module 11 is used to build an initial population by taking the configuration parameter scheme of the Ceph cluster as individuals; The function construction module 12 is used to construct a fitness function based on the performance metrics of the Ceph cluster; wherein the performance metrics include at least one of the following: read / write IOPS, throughput, read / write latency, and error rate; Calculation module 13 is used to calculate the fitness of each individual in the current population according to the fitness function; Genetic module 14 is used to perform genetic operations on the current population to generate a new population; The judgment module 15 is used to determine whether the iteration termination condition is met. If yes, the target configuration parameter scheme of the Ceph cluster is output; otherwise, the fitness is calculated using the new population.
[0053] Optionally, constructing the fitness function based on the performance metrics of the Ceph cluster includes: The performance metrics are normalized; The fitness function is constructed using the normalized performance metrics; wherein the read / write IOPS and the throughput are positively correlated with the fitness, and the read / write latency and the error rate are negatively correlated with the fitness.
[0054] Optionally, the Ceph cluster performance tuning device 10 further includes: The allocation module is used to assign individuals of the current population to different datasets; wherein, the computing nodes of each different dataset are used to perform fitness calculations and generate new populations in parallel using their own datasets.
[0055] Optionally, the determination of whether the iteration termination condition is met, if yes, outputs the target configuration parameter scheme of the Ceph cluster; if no, uses the new population to calculate fitness, including: The computing nodes of the different datasets aggregate the new populations they generate into the same dataset; The computing nodes of the same dataset determine whether the iteration termination condition is met. If yes, the population with the highest fitness is output as the target configuration parameter scheme. If no, the population that meets the preset fitness condition is selected from the new population as the current population, and the process returns to the step of calculating the fitness of each individual in the current population according to the fitness function.
[0056] Optionally, performing genetic operations on the current population to generate a new population includes: Multiple pairs of parent individuals are selected from the current population using the roulette wheel selection method. Cross over the multiple pairs of parent individuals to obtain multiple pairs of crossed individuals; The individuals after the multiple crossovers are mutated to obtain multiple pairs of mutated individuals, which are then merged into the current population.
[0057] Optionally, the Ceph cluster performance tuning device 10 further includes: The deployment module is used to deploy the software environment for the Ceph cluster and perform streaming operations on the Ceph cluster. The configuration module is used to configure Prometheus to monitor and collect performance metrics of the Ceph cluster. It is worth noting that the working process of each module in the Ceph cluster performance tuning device 10 described in this embodiment can be referred to the working process of the Ceph cluster performance tuning method described in the above embodiments, and will not be repeated here.
[0058] This application provides a Ceph cluster performance tuning device 10, which constructs an initial population by treating the Ceph cluster's configuration parameter schemes as individuals; constructs a fitness function based on the Ceph cluster's performance indicators, wherein the performance indicators include at least one of the following: read / write IOPS, throughput, read / write latency, and error rate; calculates the fitness of each individual in the current population based on the fitness function; performs genetic operations on the current population to generate a new population; determines whether the iteration termination condition is met; if so, outputs the target configuration parameter scheme of the Ceph cluster; otherwise, uses the new population to calculate fitness. Therefore, this application embodiment, driven by the fitness function, automatically searches for the target configuration parameter scheme of the Ceph cluster, reducing reliance on human experience and the workload of manual tuning, improving the efficiency and accuracy of the tuning process, and eliminating the need for system modeling, thus greatly reducing computational resources and time costs.
[0059] Furthermore, this application also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, it controls the device where the computer-readable storage medium is located to execute the Ceph cluster performance tuning method as described in any of the above embodiments.
[0060] Furthermore, this application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the Ceph cluster performance tuning method as described in any of the above embodiments.
[0061] See Figure 4 , Figure 4 This is a structural block diagram of a Ceph cluster performance tuning device 20 provided in an embodiment of this application. The Ceph cluster performance tuning device 20 includes: a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the Ceph cluster performance tuning method embodiments described above. Alternatively, when the processor 21 executes the computer program, it implements the functions of each module / unit in the various device embodiments described above.
[0062] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the Ceph cluster performance tuning device 20.
[0063] The Ceph cluster performance tuning device 20 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of the Ceph cluster performance tuning device 20 and does not constitute a limitation on the Ceph cluster performance tuning device 20. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the Ceph cluster performance tuning device 20 may also include input / output devices, network access devices, buses, etc.
[0064] The processor 21 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the Ceph cluster performance tuning device 20, connecting all parts of the Ceph cluster performance tuning device 20 via various interfaces and lines.
[0065] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements various functions of the Ceph cluster performance tuning device 20 by running or executing the computer programs and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0066] The modules / units integrated into the Ceph cluster performance tuning device 20, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.
[0067] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.
[0068] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.
Claims
1. A method for performance tuning of a Ceph cluster, characterized in that, include: The configuration parameter scheme of the Ceph cluster is treated as an individual to build an initial population; A fitness function is constructed based on the performance metrics of the Ceph cluster; wherein the performance metrics include at least one of the following: read / write IOPS, throughput, read / write latency, and error rate; Calculate the fitness of each individual in the current population based on the fitness function; Genetic operations are performed on the current population to generate a new population; Determine whether the iteration termination condition is met. If yes, output the target configuration parameter scheme of the Ceph cluster. If no, use the new population to calculate fitness.
2. The Ceph cluster performance tuning method as described in claim 1, characterized in that, The step of constructing a fitness function based on the performance metrics of the Ceph cluster includes: The performance metrics are normalized; The fitness function is constructed using the normalized performance metrics; wherein the read / write IOPS and the throughput are positively correlated with the fitness, and the read / write latency and the error rate are negatively correlated with the fitness.
3. The Ceph cluster performance tuning method as described in claim 1, characterized in that, After constructing an initial population by treating the Ceph cluster configuration parameter scheme as individuals, the Ceph cluster performance tuning method further includes: Individuals in the current population are assigned to different datasets; wherein, the computing nodes of each different dataset are used to perform fitness calculations and generate new populations in parallel using their own datasets.
4. The Ceph cluster performance tuning method as described in claim 3, characterized in that, If the iteration termination condition is met, the target configuration parameter scheme of the Ceph cluster is output; otherwise, fitness calculation is performed using the new population, including: The computing nodes of the different datasets aggregate the new populations they generate into the same dataset; The computing nodes of the same dataset determine whether the iteration termination condition is met. If yes, the population with the highest fitness is output as the target configuration parameter scheme. If no, the population that meets the preset fitness condition is selected from the new population as the current population, and the process returns to the step of calculating the fitness of each individual in the current population according to the fitness function.
5. The Ceph cluster performance tuning method as described in claim 1, characterized in that, The step of performing genetic operations on the current population to generate a new population includes: Multiple pairs of parent individuals are selected from the current population using the roulette wheel selection method. Cross over the multiple pairs of parent individuals to obtain multiple pairs of crossed individuals; The individuals after the multiple crossovers are mutated to obtain multiple pairs of mutated individuals, which are then merged into the current population.
6. The Ceph cluster performance tuning method as described in claim 1, characterized in that, Before constructing the initial population by treating the Ceph cluster configuration parameter scheme as an individual, the Ceph cluster performance tuning method further includes: Deploy the software environment for the Ceph cluster and perform streaming operations on the Ceph cluster; Configure Prometheus to monitor and collect performance metrics of the Ceph cluster.
7. A Ceph cluster performance tuning device, characterized in that, include: The population building module is used to build an initial population by taking the configuration parameter scheme of the Ceph cluster as individuals; The function construction module is used to construct a fitness function based on the performance metrics of the Ceph cluster; wherein the performance metrics include at least one of the following: read / write IOPS, throughput, read / write latency, and error rate; The calculation module is used to calculate the fitness of each individual in the current population according to the fitness function; The genetic module is used to perform genetic operations on the current population to generate a new population. The judgment module is used to determine whether the iteration termination condition is met. If yes, it outputs the target configuration parameter scheme of the Ceph cluster; otherwise, it uses the new population to calculate fitness.
8. A Ceph cluster performance tuning device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the Ceph cluster performance tuning method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the Ceph cluster performance tuning method as described in any one of claims 1 to 6.
10. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, implements the Ceph cluster performance tuning method as described in any one of claims 1 to 6.