Configuration parameter determination method and device, equipment, medium and product

Abstract

The invention discloses a configuration parameter determination method and device, equipment, a medium and a product. The method comprises the steps of generating an initial population based on configuration parameter constraints; taking the initial population as a current population, and calculating individual fitness of each chromosome individual in the current population; screening the current population according to the individual fitness of each chromosome individual to obtain a screened population; processing the screened population according to the variation factor and the individual space sparseness of the screened population to obtain a new generation of population; if the iteration ending condition is not met, the new-generation population serves as the current population, the operation of calculating the individual fitness of all chromosome individuals in the current population is executed again till the iteration ending condition is met, and the target configuration parameters are obtained.By means of the technical scheme, the configuration parameters with high performance and low cost can be determined.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular to a method, apparatus, device, medium, and product for determining configuration parameters. Background Art

[0002] With the construction of network infrastructure and the deepening of digital transformation strategies, the scale of data generated by Internet business has shown explosive growth. In order to mine effective information and ensure high-quality application services, technologies such as big data and cloud computing have been vigorously developed and applied. In addition, service providers are also continuously increasing their investment in hardware resources.

[0003] While the use of big data and cloud computing technologies has effectively improved application computing performance and brought considerable benefits to service providers, some challenges remain. For one thing, the initial configuration parameters of software fail to achieve optimal application performance. Insufficient performance can lead to user churn and losses for service providers. For example, big data software has diverse application scenarios. To improve the processing power of computing tasks, developers set a variety of configuration parameters to adjust the performance of the software and even the hardware. However, initial configuration is often designed to ensure versatility, guaranteeing a lower performance limit while locking in an upper performance limit. Furthermore, the cost of large-scale, high-performance computing clusters cannot be ignored. While operators pursue high-performance computing capabilities and improve service quality, the cost of computing resources also places a burden on their sustainable operations. Furthermore, the computing resources required for tasks at different times of day or in different scenarios vary. For example, traffic increases significantly during the day compared to at night, resulting in overcapacity with the same computing resource configuration. Summary of the Invention

[0004] Embodiments of the present invention provide a configuration parameter determination method, apparatus, device, medium, and product to solve the above-mentioned problem.

[0005] According to one aspect of the present invention, a method for determining a configuration parameter is provided, comprising:

[0006] Based on the configuration parameter constraints, an initial population is generated, wherein the chromosome individuals in the initial population are candidate configuration parameters, and the configuration parameters include: application configuration parameters, system configuration parameters, and target task volume of the target application;

[0007] The initial population is used as the current population, and the individual fitness of each chromosome individual in the current population is calculated, wherein the individual fitness is obtained by inputting each set of candidate configuration parameters in the current population and the target task amount of the target application into the target prediction model, and the individual fitness includes: target application execution time and target application energy consumption parameter;

[0008] Screening the current population according to the individual fitness of each chromosome individual to obtain a screened population;

[0009] Processing the screened population according to the variation factor and the individual spatial sparsity of the screened population to obtain a new generation population;

[0010] If the iteration end condition is not met, the new generation population is used as the current population, and the operation of calculating the individual fitness of each chromosome individual in the current population is returned to be executed until the iteration end condition is met to obtain the target configuration parameters.

[0011] According to another aspect of the present invention, a configuration parameter determination device is provided, the configuration parameter determination device comprising:

[0012] An initial population generation module is used to generate an initial population based on configuration parameter constraints, wherein the chromosome individuals in the initial population are candidate configuration parameters, and the configuration parameters include: application configuration parameters, system configuration parameters, and target task volume of the target application;

[0013] an individual fitness determination module, configured to use the initial population as the current population and calculate the individual fitness of each chromosome individual in the current population, wherein the individual fitness is obtained by inputting each set of candidate configuration parameters in the current population and the target task amount of the target application into a target prediction model, and the individual fitness includes: target application execution time and target application energy consumption parameters;

[0014] A screening module, configured to screen the current population according to the individual fitness of each chromosome individual to obtain a screened population;

[0015] a processing module, configured to process the screened population according to the variation factor and the individual spatial sparsity of the screened population to obtain a new generation population;

[0016] The individual fitness determination module, the screening module and the processing module are called cyclically until an iteration end condition is met to obtain target configuration parameters.

[0017] According to another aspect of the present invention, an electronic device is provided, comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the configuration parameter determination method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the configuration parameter determination method according to any embodiment of the present invention when executed.

[0022] According to another aspect of the present invention, a computer program product is provided. When the computer program is executed by a processor, the computer program implements the configuration parameter determination method as described in any one of the embodiments of the present invention.

[0023] The embodiment of the present invention first generates an initial population based on configuration parameter constraints; then uses the initial population as the current population and calculates the individual fitness of each chromosome individual in the current population through a target prediction model; finally, the current population is screened according to the individual fitness of each chromosome individual to obtain a screened population; the screened population is processed according to the mutation factor and the individual spatial sparsity of the screened population to obtain a new generation population; if the iteration end condition is not met, the new generation population is used as the current population, and the operation of calculating the individual fitness of each chromosome individual in the current population is returned to execute until the iteration end condition is met, and the target configuration parameters are obtained. The target prediction model is used to explore and obtain high-performance and low-cost configuration parameters in different scenarios.

[0024] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 is a flow chart of a method for determining configuration parameters in an embodiment of the present invention;

[0027] Figure 2 is a schematic structural diagram of a prediction device in an embodiment of the present invention;

[0028] Figure 3is a structural diagram of a configuration parameter determination device in an embodiment of the present invention;

[0029] Figure 4 It is a structural diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0033] Example 1

[0034] Figure 1 This is a flow chart of a method for determining configuration parameters provided by an embodiment of the present invention. This embodiment is applicable to the case of determining configuration parameters. The method can be executed by a configuration parameter determination device in an embodiment of the present invention. The device can be implemented in software and / or hardware. Figure 1 As shown, the method specifically includes the following steps:

[0035] S110: Generate an initial population based on configuration parameter constraints.

[0036] In this embodiment, the chromosome individuals in the initial population are candidate configuration parameters. The configuration parameters include application layer parameters and system layer parameters. The application layer parameters may be application configuration parameters, and the system layer parameters may be system configuration parameters.

[0037] In this embodiment, the configuration parameters may include application configuration parameters, system configuration parameters, and the target task volume of the target application. The system configuration parameters refer to configuration parameters at the operating system environment level during software execution. The system configuration parameters include container configuration parameters, which may include Java virtual machine parameters, processor parameters, and disk size. The processor parameters may include the number of kernel threads, and the Java virtual machine parameters may include virtual memory size.

[0038] In this embodiment, the application configuration parameters include: configuration parameters provided when the application software is designed, including: database configuration parameters, permission configuration parameters, business rule configuration parameters, log configuration parameters, etc.

[0039] Containers are used to package the relevant program code, function libraries, and environment configuration files required by an application to establish a sandbox execution environment.

[0040] In this embodiment, the configuration parameter constraints may include application configuration parameter constraints and system configuration parameter constraints. The system configuration parameter constraints include container configuration parameter constraints. The container configuration parameter constraints include upper and lower limits on the number of kernel threads, upper and lower limits on virtual memory size, and upper and lower limits on disk size. The application configuration parameter constraints include configuration parameter constraints provided during application software design, such as database connection pool size.

[0041] It should be noted that the values ​​of the candidate configuration parameters are different.

[0042] In this embodiment, the initial population is generated based on configuration parameter constraints by determining a configuration space based on the constraints and employing Latin hypercube sampling to achieve uniform sampling and comprehensive coverage of the configuration space. The configuration space includes a diverse and complex combination of system and application configuration parameters, including both discrete and continuous parameters. Discrete parameters are uniformly mapped to real values ​​suitable for computation. Specifically, discrete parameter values ​​are mapped using an index array, and each discrete parameter value can be accessed via an index subscript.

[0043] In this embodiment, the probability distribution is divided into different levels, and the sampling probability of different levels is the same. Compared with random sampling, the coverage is wider and the sampling cost is reduced to a certain extent. p =(x p1,…,x pj ), x pj Represents the sampled value of the jth configuration parameter of a single chromosome individual.

[0044] S120: Taking the initial population as the current population, calculating the individual fitness of each chromosome individual in the current population.

[0045] In this embodiment, each set of candidate configuration parameters in the current population is input into a target prediction model to obtain the individual fitness of each chromosome in the current population. The individual fitness includes: target application execution time and target application energy consumption parameter, where the target application energy consumption parameter is the energy consumption during the execution of the target application, for example, the CPU energy consumption during the execution of the target application.

[0046] In this embodiment, the generator and discriminator in the generative adversarial network are iteratively trained using a target sample set, and the trained generator is used as the target prediction model.

[0047] In this embodiment, the target prediction model includes: a cost prediction model and a performance prediction model.

[0048] In this embodiment, if the sampling observation overhead is controlled, the training model will be limited. In addition, the robustness of the model is also challenged by the differences between different scenarios. To build a high-precision prediction model, this embodiment of the present invention uses a generative adversarial network as a foundation to construct a low-overhead prediction model. By increasing the discrimination of the target application's workload, the prediction model's ability to identify different loads is enhanced, improving its robustness.

[0049] Optionally, the training process of the target prediction model includes:

[0050] Get a sample set of configuration parameters.

[0051] In this embodiment, the configuration parameter sample set includes multiple groups of configuration parameter samples and the task volume of the target application corresponding to each group of configuration parameter samples.

[0052] It should be noted that the multiple groups of configuration parameter samples in the configuration parameter sample set are different from the candidate configuration parameters in the initial population.

[0053] Deploy application images based on each set of configuration parameter samples, create containers, and run target applications corresponding to the application images in the containers.

[0054] In this embodiment, the target application refers to various programs that utilize configuration parameters and implement specific functions, such as a database.

[0055] In this embodiment, the application image includes the application software package and the necessary system environment when the software is running. The application image can be created by using an existing image or building a local image file.

[0056] It should be noted that in different scenarios, application images are deployed based on various sets of configuration parameter samples, and thus the performance parameters corresponding to different configuration parameter samples in different scenarios can be collected.

[0057] In this embodiment, a cluster communication channel is pre-established using the Secure Shell Protocol (SSH). Information such as the location and number of application images to be deployed is specified based on the remote host directory. Once the cluster is connected, the application images can be distributed. When starting a container, the configuration parameters for the container to be started can be specified via command lines. The corresponding configuration files can be mounted to a local directory to synchronize configuration parameters.

[0058] The target application execution time and the target application energy consumption parameter are used as performance parameters corresponding to each group of configuration parameter samples.

[0059] In this embodiment, the target application execution time is used to represent the processing capability of the target application when executing a task.

[0060] An initial prediction model is iteratively trained based on a training sample set generated according to multiple groups of configuration parameter samples, the task volume of the target application corresponding to each group of configuration parameter samples, and the performance parameters corresponding to each group of configuration parameter samples to obtain a target prediction model.

[0061] In this embodiment, the initial prediction model includes: an initial generator and an initial discriminator, the target prediction model includes: a target generator, and the target generation model includes: a performance prediction model and a cost prediction model.

[0062] It should be noted that since the size of the sample set is limited, the limited number of samples will affect the prediction accuracy of the trained model. Therefore, the embodiment of the present invention uses a generative adversarial network as the initial prediction model for training to obtain the target prediction model.

[0063] In this embodiment, a generative adversarial network leverages game theory to enable the discriminator and generator to achieve joint optimization through a mutually adversarial model. The generator is responsible for generating simulated samples, while the discriminator verifies the authenticity of the generator's output. During training, the discriminator is able to detect fake samples to the greatest extent possible, forcing the trained generator's output to approximate real samples. This training mechanism effectively ensures model accuracy while reducing reliance on initial samples and sample overhead.

[0064] While training the generator and discriminator, we obtain the feature importance index of each parameter in the configuration parameters of the input generator for use in the subsequent mutation process. It should be noted that during the training process, each parameter in the configuration parameters is randomly set to zero with a certain probability, and the fluctuation of the loss function or the change of the generated sample quality is observed. The feature importance of each parameter is determined based on the fluctuation of the loss function or the change of the generated sample quality.

[0065] Optionally, the target prediction model includes: a performance prediction model, and the initial prediction model includes: an initial generator and an initial discriminator;

[0066] Iteratively training an initial prediction model based on a training sample set generated based on multiple groups of configuration parameter samples, the task volume of the target application corresponding to each group of configuration parameter samples, and the performance parameters of each group of configuration parameter samples to obtain a target prediction model, including:

[0067] The initial generator and the initial discriminator are alternately trained based on a training sample set generated according to multiple groups of configuration parameter samples, the task volume of the target application corresponding to each group of configuration parameter samples, and the target application execution time of each group of configuration parameter samples to obtain a performance prediction model.

[0068] In this embodiment, the training process of the initial discriminator includes: generating a first real sample set based on multiple groups of configuration parameter samples and the target application execution time corresponding to each group of configuration parameter samples; generating a first fake sample set based on the initial generator; training the parameters of the discriminator based on the first real sample set and the first fake sample set; the training process of the initial generator includes: inputting randomly generated configuration parameter samples into the initial generator to obtain the predicted target application execution time, inputting the randomly generated configuration parameter samples and the predicted target application execution time corresponding to the randomly generated configuration parameter samples into the discriminator, and training the parameters of the initial generator based on the discrimination results output by the discriminator.

[0069] In this embodiment, a batch of fake samples is generated by the generator; these fake samples are discriminated by the discriminator; the loss of the generator is calculated based on the discrimination result of the discriminator (the goal is to make the discriminator misjudge), and the parameters of the generator are back-propagated and updated.

[0070] Optionally, the target prediction model further includes: a cost prediction model;

[0071] Also includes:

[0072] The initial generator and the initial discriminator are alternately trained according to the training sample sets generated by multiple groups of configuration parameter samples, the task volume of the target application corresponding to each group of configuration parameter samples, and the target application energy consumption parameters of each group of configuration parameter samples to obtain a cost prediction model, wherein the training process of the initial discriminator includes: generating a second real sample set according to the multiple groups of configuration parameter samples and the target application energy consumption parameters corresponding to each group of configuration parameter samples; generating a third fake sample set based on the initial generator; training the parameters of the discriminator based on the second real sample set and the third fake sample set; the training process of the initial generator includes: inputting randomly generated configuration parameter samples into the initial generator to obtain predicted target application energy consumption parameters, inputting the randomly generated configuration parameter samples and the predicted target application energy consumption parameters corresponding to the randomly generated configuration parameter samples into the discriminator, and training the parameters of the initial generator based on the discrimination results output by the discriminator.

[0073] In this embodiment, the training process of the cost prediction model is similar to the training process of the performance prediction model, and will not be described in detail here.

[0074] S130, screening the current population according to the individual fitness of each chromosome individual to obtain a screened population.

[0075] In this embodiment, the current population is screened according to the individual fitness of each chromosome individual, and the method for obtaining the screened population can be: weighted summation of the cost prediction results and performance prediction results output by the target prediction model corresponding to each chromosome individual to obtain the individual fitness of the chromosome individual, and the current population is screened according to the individual fitness of each chromosome individual to obtain the screened population.

[0076] In this embodiment, the screening rule can be to select chromosome individuals in the current population whose individual fitness is within the target fitness range, or to select chromosome individuals in the current population whose cost prediction result is less than the cost threshold and whose performance prediction result is less than the performance threshold.

[0077] S140, processing the screened population according to the variation factor and the individual spatial sparsity of the screened population to obtain a new generation population.

[0078] In this embodiment, the screened population is processed according to the mutation factor and the individual spatial sparsity of the screened population to obtain a new generation population in the following manner: based on the mutation operator, the candidate configuration parameters in the screened population are mutated to obtain an offspring population; based on the offspring population and the screened population, a target population is generated; and based on the individual spatial sparsity of the target population, the target population is screened to obtain a new generation population.

[0079] Optionally, processing the screened population according to the variation factor and the individual spatial sparsity of the screened population to obtain a new generation population includes:

[0080] Based on the mutation operator, the candidate configuration parameters in the screened population are mutated to obtain a descendant population.

[0081] In this embodiment, the method for mutating the candidate configuration parameters in the screened population based on the mutation operator to obtain the offspring population may be: determining the mutation parameter based on the configuration parameter constraint and the candidate configuration parameters in the screened population; and mutating the candidate configuration parameters in the screened population based on the mutation parameter to obtain the offspring population. The method for mutating the candidate configuration parameters in the screened population based on the mutation operator to obtain the offspring population may also be: mutating the candidate configuration parameters in the screened population based on the mutation operator and an importance index of each parameter in the candidate configuration parameters in the population to obtain the offspring population.

[0082] It should be noted that during the training process, each parameter in the configuration parameters is randomly set to zero with a certain probability, and the loss function fluctuation or the change in the quality of the generated samples is observed. The importance index of each parameter is determined based on the loss function fluctuation or the change in the quality of the generated samples.

[0083] A target population is generated according to the progeny population and the screened population.

[0084] In this embodiment, a target population may be generated according to the offspring population and the screened population by merging the offspring population and the screened population to obtain the target population.

[0085] The target population is screened based on the individual spatial sparsity of the target population to obtain a new generation population.

[0086] Optionally, based on a mutation operator, mutating the candidate configuration parameters in the screened population to obtain a child population includes:

[0087] A mutation parameter is determined according to the configuration parameter constraints and the candidate configuration parameters in the screened population.

[0088] In this embodiment, the variation parameter θ is calculated based on the following formula:

[0089]

[0090] Where u is a random number with a value range of [0,1] and is randomly generated during calculation. mis the distribution index, which is specified by the user during calculation, such as 5, 10, 20. The real-coded population makes each eigenvector of the individual undergo polynomial conflict according to the mutation rate, resulting in real-valued mutation.

[0091] The candidate configuration parameters in the screened population are mutated based on the mutation parameters to obtain a progeny population.

[0092] In this embodiment, the candidate configuration parameters in the screened population are mutated based on the mutation parameter to obtain the offspring population. This may be accomplished by mutating the candidate configuration parameters in the screened population based on the configuration parameter constraints and the mutation parameter to obtain the offspring population. For example, the difference between the upper and lower limits of the configuration parameter multiplied by the mutation parameter may be used as the adjustment amount, and the sum of each candidate configuration parameter in the screened population and the adjustment amount may be used as each chromosome individual in the offspring population.

[0093] In this embodiment, the chromosome individuals in the offspring population are determined based on the following formula:

[0094] vc'=vc i +θ(u i -l i )

[0095] Among them, vc′ is the chromosome individual in the offspring population, u i is the upper limit of configuration parameter i, l i is the lower limit of the configuration parameter i, θ is the variation parameter, vc i is the parent chromosome individual (chromosome individual in the population after screening)

[0096] S150, determine whether the iteration end condition is met, if so, execute S160, if not, take the new generation population as the current population, and return to execute S120.

[0097] In this embodiment, the iteration end condition includes at least one of the following:

[0098] The number of iterations is greater than the threshold;

[0099] After multiple consecutive iterations, the individual fitness change is less than the change threshold;

[0100] The running time is greater than the upper limit of the running time;

[0101] After multiple consecutive iterations, the new generation population obtained is the same.

[0102] S160, the iteration ends and the target configuration parameters are output.

[0103] In this embodiment, the target configuration parameters may include: multiple groups of configuration parameters.

[0104] If the iteration end condition is not met, the new generation population is used as the current population, and the operation of calculating the individual fitness of each chromosome individual in the current population is returned to be executed until the iteration end condition is met to obtain the target configuration parameters.

[0105] In this embodiment, the target configuration parameters include multiple groups of application configuration parameters and system configuration parameters.

[0106] It should be noted that container technology has become a core tool for modern software development and deployment. Although container technology has reduced dependence on the operating environment to a certain extent, in the face of diverse computing tasks, the default configuration parameters provided by software designers cannot maximize the advantages of the software; in addition, excess performance during non-peak hours will also cause certain cost overheads. Directly deploying and applying container-based software cannot achieve optimal performance, and the operating performance under the default configuration cannot meet the requirements of maximizing performance and minimizing cost overheads, nor can it achieve a balance. Ultimately, it causes continuous losses to the sustainable development of operating organizations. Therefore, it is necessary to perform multi-objective configuration optimization for containerized deployed software. However, there are the following difficulties in this field that need to be solved urgently:

[0107] (1) It is difficult to quickly locate the optimal configuration: The configuration parameter space of software is usually huge. Taking Spark as an example, its adjustable parameters have at least 100 dimensions. The integration of cross-layer parameters further expands the original configuration space, and continuous parameters and discrete parameters also form a diverse parameter value space. At the same time, the relationship between parameters and optimization objectives is not a simple linear relationship, and the time for configuration evaluation cannot be ignored. Therefore, it is necessary to quickly locate the optimal configuration in a complex configuration space with low overhead.

[0108] (2) Multi-objective conflict: Generally speaking, there are conflicts among multiple optimization objectives, such as performance and cost. Some work will transform multi-objective optimization into a single-objective problem through weighting. However, the coefficients in the linear weighting method need to be weighed and explored in the actual generation environment, and simple processing methods will also lose some optimal solutions.

[0109] (3) Dynamic load adaptability: Workloads change dynamically, including input dataset size and application type. This may cause the optimal configuration to change. If the established model cannot perceive this, the prediction model needs to be rebuilt.

[0110] The low computing performance of software will greatly limit the execution efficiency and restrict the development of operating organizations. In addition, although the energy consumption cost comes from the hardware equipment, the energy consumption generated by the hardware equipment is inseparable from the software. Therefore, while ensuring the performance of the software, the operating organization is also constantly optimizing and reducing the energy consumption cost to alleviate the burden on sustainable operations caused by the huge energy consumption and cost overhead generated by the operation of high-performance clusters. Based on the above situation, an embodiment of the present invention provides a method for determining the configuration parameters of application software deployed using containers. For the adjustable configuration parameters in the application image, the advantages of containerization technology are utilized to construct a training sample set through automated application image deployment and operation indicator collection, and an artificial intelligence algorithm is used to train the initial prediction model for predicting the performance and cost of unknown configurations, and finally quickly locate the optimal configuration parameters.

[0111] In a specific example, Figure 2 As shown, the prediction device includes a configuration generation module, an operation monitoring module, and a data prediction module. The configuration generation module defines the cross-layer configuration space within the application image to be optimized, namely, the configuration parameter constraints (configuration parameter upper and lower limits). It uses container technology to implement automated deployment and execution of application software and provides dynamic cross-layer configuration parameter updates. Container technology can reduce interference from system environment differences. Furthermore, this module utilizes uniform sampling to achieve low-cost configuration sampling. The operation monitoring module is primarily responsible for collecting system-level parameters corresponding to the executing application in the application image. These system-level parameters include the number of kernel threads, virtual memory size, and disk size. After the application task is completed, the application-level parameters, system-level parameters, application execution time, and CPU power consumption are persistently stored as a positive sample. The data prediction module is responsible for training the prediction model based on the positive samples, constructing a low-cost initial prediction model using artificial intelligence algorithms. To increase adaptability to varying task loads, the target application's task load is also processed synchronously to improve the prediction model's accuracy in different scenarios. The configuration exploration module is based on the multiple models constructed by the data prediction module and uses a multi-objective evolutionary algorithm to iteratively explore the optimal configuration solution set. In order to accelerate exploration efficiency and improve convergence speed, the importance of each configuration parameter calculated when building the prediction model will be used to guide the evolutionary process, such as mutation.

[0112] In another specific example, the configuration parameter determination method includes the following process:

[0113] Step 1: Define a cross-layer configuration space, wherein the cross-layer configuration space includes: application configuration parameter constraints and system configuration parameter constraints.

[0114] Step 2: Implement uniform sampling in the configuration space to generate the initial population, reducing sample observation overhead and improving coverage.

[0115] Step 3: Deploy the application image based on each set of configuration parameter samples, create a container, and run the target application corresponding to the application image in the container.

[0116] Step 4: Collect the performance parameters corresponding to each set of configuration parameter samples and construct a training sample set.

[0117] Step 5: For the training sample set, use the generative adversarial network to train and obtain the target prediction model.

[0118] Step 6: Based on the target prediction model, determine the individual fitness of the chromosome individuals in the population;

[0119] Step 7: Iterate and explore the optimal configuration parameters.

[0120] The ultimate goal of the embodiment of the present invention is to locate the optimal configuration parameters based on the performance and cost model, so as to dynamically adjust the performance and cost overhead of the application mirror task execution. Therefore, based on the prediction results of the target prediction model as the evaluation function of the search algorithm, in order to resolve the conflicting relationship between the optimization objectives, a multi-objective evolutionary algorithm is adopted to optimize the objectives of multiple dimensions and explore the optimal configuration parameters under the multi-objective scenario. For example, the second-generation non-dominated sorting genetic algorithm achieves the optimization of the configuration population through operations such as evolution, selection, and mutation. The specific steps are as follows:

[0121] Obtain candidate configuration parameters whose values ​​satisfy the configuration parameter constraints and generate chromosome individuals through real number encoding. Use Latin hypercube sampling to define multiple groups of chromosome individuals to form the initial population.

[0122] The probability of an individual surviving to the next generation is determined by its fitness, which is determined by the output of the target prediction model. The number of individuals to be compared is determined, and then the individuals with the best fitness within the small population are selected and added to the pool. This process is repeated until the required number of individuals is met. To increase the range of exploration within each generation, a simulated binary crossover operator is used on the parent's real-valued gene fragments to preserve their value in the new generation while maintaining individual diversity.

[0123] Based on the importance index of the parameters, one or more genes on the chromosome segment are guided to mutate, avoiding multiple mutation operations on non-critical parameters, enriching the population diversity, and avoiding falling into the local optimal solution. The chromosome individuals in the offspring population are determined based on the following formula:

[0124] vc'=vc i +θ(u i -l i )

[0125] Among them, vc′ is the chromosome individual in the offspring population, u i is the upper limit of configuration parameter i, l iis the lower limit of the configuration parameter i, θ is the variation parameter, vc i is the parent chromosome individual (the chromosome individual in the screened population), and θ is calculated based on the following formula:

[0126]

[0127] Where u is a random number with a value range of [0,1] and is randomly generated during calculation. m is the distribution index, which is specified by the user during calculation, such as 5, 10, 20. The real-coded population makes each eigenvector of the individual undergo polynomial conflict according to the mutation rate, resulting in real-valued mutation.

[0128] The resulting offspring population is combined with the parent population to form a new population. After evaluating the individual spatial sparseness, the new population is selectively evolved into a new generation. The above steps are repeated until the iteration termination condition is met, and the target configuration parameters are obtained.

[0129] The embodiment of the present invention takes advantage of the container to ensure environmental consistency, automatically deploys application images, trains to obtain a target prediction model, and explores the optimal configuration parameters of performance and cost in different scenarios through the target prediction model.

[0130] The embodiments of the present invention address the problem of insufficient potential mining of configuration space and maximize the potential optimization space of applications by integrating system configuration parameters and application configuration parameters.

[0131] The embodiment of the present invention addresses the problem of difficulty in quickly locating the optimal configuration: a model is used to capture the complex relationship between the target application execution time and the target application energy consumption parameters and the configuration parameters, and the probability distribution of the target application execution time and the target application energy consumption parameters is obtained.

[0132] To address the high overhead of establishing the initial population, Latin hypercube sampling is used to evenly cover the sampling range and reduce the probability of repeated configuration parameters. The sample generator is trained through a generative adversarial network, and a high-precision target prediction model can be obtained from a small sample set.

[0133] For multi-objective optimization problems: Based on multiple prediction models, evolutionary algorithms are used to explore the optimal configuration under multiple objectives, and the mutation process is guided by the importance index of the parameters to accelerate the efficiency of the optimal configuration exploration.

[0134] Compared to traditional configuration parameter optimization solutions, the embodiments of this invention offer advantages such as time savings, wide adaptability, high accuracy, and deep optimization. They utilize container-based software deployment to reduce environmental variability. A generative adversarial network is used to train a high-precision, load-aware prediction model at low cost. Finally, a multi-objective evolutionary algorithm is used to rapidly locate optimal configuration parameters, enabling optimal configuration exploration under multiple objectives.

[0135] Example 2

[0136] Figure 3 This is a schematic diagram of the structure of a configuration parameter determination device provided by an embodiment of the present invention. This embodiment is applicable to the case of configuration parameter determination. The device can be implemented in software and / or hardware. The device can be integrated into any device that provides a configuration parameter determination function, such as Figure 3 As shown, the configuration parameter determination device specifically includes: an initial population generation module 310 , an individual fitness determination module 320 , a screening module 330 and a processing module 340 .

[0137] The initial population generation module is used to generate an initial population based on configuration parameter constraints, wherein the chromosome individuals in the initial population are candidate configuration parameters, and the configuration parameters include: application configuration parameters, system configuration parameters and target task volume of the target application;

[0138] an individual fitness determination module, configured to use the initial population as the current population and calculate the individual fitness of each chromosome individual in the current population, wherein the individual fitness is obtained by inputting each set of candidate configuration parameters in the current population and the target task amount of the target application into a target prediction model, and the individual fitness includes: target application execution time and target application energy consumption parameters;

[0139] A screening module, configured to screen the current population according to the individual fitness of each chromosome individual to obtain a screened population;

[0140] a processing module, configured to process the screened population according to the variation factor and the individual spatial sparsity of the screened population to obtain a new generation population;

[0141] The individual fitness determination module, the screening module and the processing module are called cyclically until an iteration end condition is met to obtain target configuration parameters.

[0142] The above-mentioned product can execute the method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0143] Example 3

[0144] Figure 4A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0145] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0146] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0147] The processor 11 may be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the configuration parameter determination method.

[0148] In some embodiments, the configuration parameter determination method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the configuration parameter determination method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the configuration parameter determination method in any other appropriate manner (e.g., by means of firmware).

[0149] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0150] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0151] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0152] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0153] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0154] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0155] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0156] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the configuration parameter determination method according to any embodiment of the present invention.

[0157] The computer program product may be implemented by writing computer program code for performing the operations of the present invention in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0158] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for determining configuration parameters, characterized in that: include: Based on the configuration parameter constraints, an initial population is generated, wherein the chromosome individuals in the initial population are candidate configuration parameters, and the configuration parameters include: application configuration parameters, system configuration parameters, and target task volume of the target application; The initial population is used as the current population, and the individual fitness of each chromosome individual in the current population is calculated, wherein the individual fitness is obtained by inputting each set of candidate configuration parameters in the current population and the target task amount of the target application into the target prediction model, and the individual fitness includes: target application execution time and target application energy consumption parameter; Screening the current population according to the individual fitness of each chromosome individual to obtain a screened population; Processing the screened population according to the variation factor and the individual spatial sparsity of the screened population to obtain a new generation population; If the iteration end condition is not met, the new generation population is used as the current population, and the operation of calculating the individual fitness of each chromosome individual in the current population is returned to be executed until the iteration end condition is met to obtain the target configuration parameters.

2. The method according to claim 1, characterized in that The screened population is processed according to the variation factor and the individual spatial sparsity of the screened population to obtain a new generation population, including: Based on a mutation operator, mutating the candidate configuration parameters in the screened population to obtain a progeny population; generating a target population according to the progeny population and the screened population; The target population is screened based on the individual spatial sparsity of the target population to obtain a new generation population.

3. The method according to claim 2, characterized in that Based on the mutation operator, the candidate configuration parameters in the screened population are mutated to obtain a child population, including: determining a mutation parameter according to the configuration parameter constraints and the candidate configuration parameters in the screened population; The candidate configuration parameters in the screened population are mutated based on the mutation parameters to obtain a progeny population.

4. The method according to claim 1, wherein The training process of the target prediction model includes: Acquire a configuration parameter sample set, wherein the configuration parameter sample set includes multiple groups of configuration parameter samples and a task volume of a target application corresponding to each group of configuration parameter samples; Deploy an application image based on each set of configuration parameter samples, create a container, and run the target application corresponding to the application image in the container; The target application execution time and target application energy consumption parameters are used as performance parameters corresponding to each group of configuration parameter samples; An initial prediction model is iteratively trained based on a training sample set generated based on multiple groups of configuration parameter samples, the task volume of the target application corresponding to each group of configuration parameter samples, and the performance parameters corresponding to each group of configuration parameter samples to obtain a target prediction model, wherein the initial prediction model includes: an initial generator and an initial discriminator, the target prediction model includes: a target generator, and the target generation model includes: a performance prediction model and a cost prediction model.

5. The method according to claim 4, characterized in that The target prediction model includes: a performance prediction model, and the initial prediction model includes: an initial generator and an initial discriminator; Iteratively training an initial prediction model based on a training sample set generated based on multiple groups of configuration parameter samples, the task volume of the target application corresponding to each group of configuration parameter samples, and the target application execution time of each group of configuration parameter samples to obtain a target prediction model, including: The initial generator and the initial discriminator are alternately trained according to the training sample sets generated by multiple groups of configuration parameter samples, the task volume of the target application corresponding to each group of configuration parameter samples, and the target application execution time of each group of configuration parameter samples to obtain a performance prediction model, wherein the training process of the initial discriminator includes: generating a first real sample set according to multiple groups of configuration parameter samples and the target application execution time corresponding to each group of configuration parameter samples; generating a first fake sample set based on the initial generator; training the parameters of the discriminator based on the first real sample set and the first fake sample set; the training process of the initial generator includes: inputting randomly generated configuration parameter samples into the initial generator to obtain the predicted target application execution time, inputting the randomly generated configuration parameter samples and the predicted target application execution time corresponding to the randomly generated configuration parameter samples into the discriminator, and training the parameters of the initial generator based on the discrimination result output by the discriminator.

6. The method according to claim 5, characterized in that The target prediction model also includes: a cost prediction model; Also includes: The initial generator and the initial discriminator are alternately trained according to the training sample sets generated by multiple groups of configuration parameter samples, the task volume of the target application corresponding to each group of configuration parameter samples, and the target application energy consumption parameters of each group of configuration parameter samples to obtain a cost prediction model, wherein the training process of the initial discriminator includes: generating a second real sample set according to the multiple groups of configuration parameter samples and the target application energy consumption parameters corresponding to each group of configuration parameter samples; generating a third fake sample set based on the initial generator; training the parameters of the discriminator based on the second real sample set and the third fake sample set; the training process of the initial generator includes: inputting randomly generated configuration parameter samples into the initial generator to obtain predicted target application energy consumption parameters, inputting the randomly generated configuration parameter samples and the predicted target application energy consumption parameters corresponding to the randomly generated configuration parameter samples into the discriminator, and training the parameters of the initial generator based on the discrimination results output by the discriminator.

7. A configuration parameter determination device, characterized in that: include: An initial population generation module is used to generate an initial population based on configuration parameter constraints, wherein the chromosome individuals in the initial population are candidate configuration parameters, and the configuration parameters include: application configuration parameters, system configuration parameters, and target task volume of the target application; an individual fitness determination module, configured to use the initial population as the current population and calculate the individual fitness of each chromosome individual in the current population, wherein the individual fitness is obtained by inputting each set of candidate configuration parameters in the current population and the target task amount of the target application into a target prediction model, and the individual fitness includes: target application execution time and target application energy consumption parameters; A screening module, configured to screen the current population according to the individual fitness of each chromosome individual to obtain a screened population; a processing module, configured to process the screened population according to the variation factor and the individual spatial sparsity of the screened population to obtain a new generation population; The individual fitness determination module, the screening module and the processing module are called cyclically until an iteration end condition is met to obtain target configuration parameters.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the configuration parameter determination method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the configuration parameter determination method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the configuration parameter determination method according to any one of claims 1 to 6.

Images