Adjusting and optimizing method and device and computing equipment

By automatically tuning the operating parameters of the WRF model in stages, and using optimization algorithms such as genetic algorithms, the problem of low tuning efficiency of WRF model is solved, and the effect of reducing communication time and I/O time is achieved, and the calculation efficiency and tuning are improved. accuracy.

CN119065826APending Publication Date: 2024-12-03HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310654892.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-02
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The prior art is difficult to effectively tune the WRF model of meteorological research and forecasting, resulting in an increase in communication time and I/O time, affecting computing efficiency.

Method used

Provide an automatic tuning method, by obtaining the operating parameters of the WRF model, tuning each parameter in stages, using optimization algorithms such as genetic algorithms to reduce communication time and I/O time, and store the tuning results in the database to improve Accuracy of subsequent tuning.

Benefits of technology

Automatic WRF model parameter tuning is realized, reducing communication time and I/O time, speeding up tuning convergence speed, and enriching database data to improve the accuracy of subsequent tuning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065826A_ABST
    Figure CN119065826A_ABST
Patent Text Reader

Abstract

The invention discloses an adjusting and optimizing method, an adjusting and optimizing device and computing equipment. In this way, the computing device can automatically optimize the operating parameters for operating the WRF model, and the communication time exposed to the outside and the total I / O time of the WRF model can be reduced as much as possible. In the automatic tuning process of the computing device, each parameter of the operation parameters is tuned and optimized in stages, a small number of parameters are tuned and optimized each time, and the convergence speed of tuning and optimization can be increased. After all the parameters of the operation parameters are adjusted and optimized by the computing device, the adjustment and optimization result is formed and stored in the database, data of the database can be enriched, and the accuracy of follow-up adjustment and optimization is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of cloud service technology, and in particular to a tuning method, device and computing equipment. Background Art

[0002] The Weather Research and Forecasting (WRF) model is the most advanced atmospheric modeling system used in meteorological research and numerical weather forecasting. It is a mesoscale numerical weather forecasting model. The WRF model is written in Fortran 90 and can run on a variety of different platforms. The WRF model can be applied to the research and operationalization of numerical weather forecasting, the study of physical parameterization schemes, data assimilation, driving air quality models, mesoscale climate simulations, and coupling of ocean-atmosphere models. In the process of operational research and scientific research on weather forecasting, being able to quickly and efficiently select the optimal settings for the WRF model is of great research significance for improving the efficiency of numerical forecasting. Summary of the invention

[0003] In order to solve the above problems, a tuning method is provided in an embodiment of the present application, which automatically tunes the operating parameters of the WRF model, which can minimize the communication time exposed to the outside and the total I / O time of the WRF model. In the automatic tuning process, each parameter of the operating parameter is tuned in stages, and a small amount of parameters are tuned each time, which can speed up the convergence speed of the tuning. After all parameters of the operating parameters are tuned, the tuning results are formed and stored in the database, which can enrich the data in the database to improve the accuracy of subsequent tuning. In addition, the present application also provides a tuning device and a computing device corresponding to the tuning method.

[0004] To this end, the following technical solutions are adopted in the embodiments of the present application:

[0005] In a first aspect, an embodiment of the present application provides a tuning method, which is applied to a computing device, wherein the computing device is used to run a meteorological research and forecasting WRF model, the method comprising: obtaining operating parameters for running the WRF model; generating multiple parameter values ​​for each parameter based on each parameter of a first group in the operating parameters; dividing the multiple parameters in the operating parameters into multiple groups, the multiple groups including the first group, the first group including one or more parameters; inputting the multiple parameter values ​​of each parameter into a tuning model to obtain the tuned parameter values ​​of each parameter.

[0006] In this implementation, the computing device can automatically tune the operating parameters of the WRF model, which can minimize the communication time exposed to the outside and the total I / O time of the WRF model. During the automatic tuning process, the computing device tunes the various parameters of the operating parameters in stages, and tunes a small amount of parameters each time, which can speed up the convergence speed of the tuning.

[0007] In one embodiment, the operating parameters include an operating environment and an example name, and the generating of multiple parameter values ​​of each parameter based on each parameter of the first group in the operating parameters includes: comparing with the tuning results stored in the database according to the operating environment in the operating parameters; in response to the similarity between the operating environment in the target tuning result and the operating environment in the operating parameters being greater than a set threshold, detecting whether the target tuning result has the example name of the operating parameter; in response to the existence of the example name of the operating parameter in the target tuning result, obtaining the parameter values ​​of each parameter in the target tuning result, as well as a set number of parameter values ​​near the parameter values ​​of each parameter, to obtain multiple parameter values ​​of each parameter.

[0008] In this embodiment, the computing device can search the database for tuning results that match the operating environment and the example name in the operating parameters, and obtain the tuning results with the highest similarity to the operating environment in the operating parameters. The parameter values ​​of each parameter of the first group can be quickly found to improve computing efficiency.

[0009] In one embodiment, the method further includes: in response to the target tuning result not having the calculation instance name of the operating parameter, randomly generating a plurality of parameter values ​​of each parameter.

[0010] In one embodiment, the inputting of multiple parameter values ​​of the various parameters into the tuning model to obtain the tuned parameter values ​​of the various target parameters includes: randomly combining multiple parameter values ​​of different parameters to obtain multiple groups of parameter values, the multiple groups of parameter values ​​including a parameter value of each parameter; simulating the benchmark workload of the WRF model based on the multiple groups of parameter values ​​to obtain multiple benchmark workloads of the WRF model; weighted calculation of the end-to-end performance and input / output performance of the benchmark workloads of the multiple WRF models to obtain multiple single-value performance data; and inputting the multiple single-value performance data into the tuning model to obtain the tuned parameter values ​​of the various target parameters.

[0011] In this implementation, after the computing device obtains several different parameter values ​​of the target parameter, it can randomly match several different parameter values ​​of the selected target parameters to obtain multiple sets of parameter values, and obtain a corresponding number of single-value performance data based on the multiple sets of parameter values. The single-value performance data can be used as the fitness of the subsequent tuning algorithm. The computing device can increase the number of fitness of the tuning algorithm by random matching, thereby improving the individual richness of the tuning algorithm.

[0012] In one implementation, the tuning model is one of a genetic algorithm, a simulated annealing algorithm, and an ant colony algorithm.

[0013] In one embodiment, the tuning model is a genetic algorithm, and the step of inputting the multiple single-value performance data into the genetic algorithm to obtain the tuned parameter values ​​of the various target parameters includes: obtaining tuning parameters, wherein the tuning parameters include the population size of the genetic algorithm and the number of iterations of the genetic algorithm; inputting the multiple single-value performance data, the population size of the genetic algorithm, and the number of iterations of the genetic algorithm into the genetic algorithm to obtain the tuned single-value performance data; and determining the parameter values ​​of different parameters corresponding to the tuned single-value performance data based on the tuned single-value performance data.

[0014] In this embodiment, the computing device can input the tuning parameters and single-value performance data into the genetic algorithm to tune the several target parameters selected this time, obtain the optimal single-value performance data, and obtain the optimal parameter value of the target parameter based on the best single-value performance data.

[0015] In one embodiment, the method further includes: detecting whether the parameters of the operating parameters in groups other than the first group have been tuned; in response to the completion of tuning of the operating parameters in groups other than the first group, storing the tuned operating parameters in a database.

[0016] In this implementation, after the computing device completes the tuning of all parameters of the operating parameters, it forms a tuning result and stores it in a database, which can enrich the data in the database to improve the accuracy of subsequent tuning.

[0017] In a second aspect, an embodiment of the present application provides a tuning device, comprising: a transceiver, for acquiring operating parameters for running the WRF model; a processor, for generating multiple parameter values ​​of each parameter based on each parameter of a first group in the operating parameters; the multiple parameters in the operating parameters are divided into multiple groups, the multiple groups include the first group, and the first group includes one or more parameters; the multiple parameter values ​​of each parameter are input into the tuning model to obtain the tuned parameter values ​​of each parameter.

[0018] In one embodiment, the operating parameters include an operating environment and an instance name, and the processor is specifically used to compare the tuning results stored in the database according to the operating environment in the operating parameters; in response to the similarity between the operating environment in the target tuning result and the operating environment in the operating parameters being greater than a set threshold, detect whether the target tuning result has the instance name of the operating parameter; in response to the target tuning result having the instance name of the operating parameter, obtain the parameter values ​​of each parameter in the target tuning result, as well as a set number of parameter values ​​near the parameter value of each parameter, to obtain multiple parameter values ​​of each parameter.

[0019] In one embodiment, the processor is further configured to randomly generate a plurality of parameter values ​​for each parameter in response to the target tuning result not having a calculation instance name for the operating parameter.

[0020] In one embodiment, the processor is specifically used to randomly combine multiple parameter values ​​of different parameters to obtain multiple groups of parameter values, wherein the multiple groups of parameter values ​​include a parameter value of each parameter; simulate the benchmark workload of the WRF model based on the multiple groups of parameter values ​​to obtain multiple benchmark workloads of the WRF model; perform weighted calculations on the end-to-end performance and input / output performance of the benchmark workloads of the multiple WRF models to obtain multiple single-value performance data; input the multiple single-value performance data into a tuning model to obtain the tuned parameter values ​​of each target parameter.

[0021] In one implementation, the tuning model is one of a genetic algorithm, a simulated annealing algorithm, and an ant colony algorithm.

[0022] In one embodiment, the tuning model is a genetic algorithm, and the processor is specifically used to obtain tuning parameters, wherein the tuning parameters include the population size of the genetic algorithm and the number of iterations of the genetic algorithm; the multiple single-value performance data, the population size of the genetic algorithm and the number of iterations of the genetic algorithm are input into the genetic algorithm to obtain tuned single-value performance data; based on the tuned single-value performance data, the parameter values ​​of different parameters corresponding to the tuned single-value performance data are determined.

[0023] In one embodiment, the processor is also used to detect whether the parameters of other groups among the operating parameters except the first group have been tuned; in response to the completion of tuning of other groups among the operating parameters except the first group, the tuned operating parameters are stored in the database.

[0024] In a third aspect, an embodiment of the present application provides a computing device, comprising: at least one memory, and at least one processor, wherein the at least one processor is used to execute instructions stored in the at least one memory, so that the computing device executes various possible implementation embodiments of the first aspect.

[0025] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute various possible implementation embodiments of the first aspect.

[0026] In a fifth aspect, an embodiment of the present application provides a computer program product, characterized in that the computer program product stores instructions, and when the instructions are executed by a computer, the computer implements each possible implementation embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The following is a brief introduction to the drawings required for describing the embodiments or prior art.

[0028] Figure 1 It is a schematic diagram of the architecture of a WRF model system;

[0029] Figure 2 A schematic diagram of the architecture of a high performance computing cluster provided in an embodiment of the present application;

[0030] Figure 3 A flowchart of a tuning method provided in an embodiment of the present application;

[0031] Figure 4 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0032] Figure 5 A schematic diagram of the architecture of a computing device cluster provided in an embodiment of the present application;

[0033] Figure 6 A schematic diagram of the architecture of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0035] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0036] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.

[0037] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0038] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0039] Figure 1 A schematic diagram of the WRF model architecture. Figure 1 As shown, the WRF model 100 may include a WRF pre-processing system (WRF pre-processing system, WPS) 110 , a WRF data assimilation system (WRF data assimilation system, WRFDA) 120 , a power kernel 130 and a post-processing system 140 .

[0040] WPS 110 is used to simulate real data to define the simulation domain, interpolate land data (such as topography, land use and soil type) into the simulation domain, and reduce the dimension of meteorological data of another model and interpolate it into the simulation domain. WRFDA120 is used to provide initial conditions for WRF model forecasts, verify the differences between observations and models, and design, detect and evaluate observation systems. The power kernel 130 includes an advanced research WRF (ARW) solver. The ARW solver consists of several initialization programs for idealized and real data simulations and numerical integration programs. The post-processing system 140 can include the National Center for Atmospheric Research (NCAR) command language (NCAR command language, NCL), the fourth generation raster image processor (4th raster image processor, RIP 4), the grid analysis and display system (grid analysis and display system, GrADS), and conversion programs for other off-the-shelf graphics software packages. The post-processing system 140 is used to analyze and process the output results of the model, convert the physical quantities of the model surface to the standard isobaric surface, diagnose and analyze the physical field and convert graphic data, etc.

[0041] During the installation of the WRF model, the WRF model not only needs to configure the Linux operating system in advance, but also needs to prepare a large number of dependent libraries. For example, when the WRF model is installed in WPS, the WRF model also needs to install programs such as zlib, libpng, and Jasper to read meteorological data in the second-generation binary general regularly distributed information (2nd general regularly distributed information in binary form, GRIB 2) format. For another example, before the WRF model installs the power kernel, it is necessary to install programs such as Intel Fortran, netCDF4, and HDF5 to support dependent libraries. When the WRF model has parallel computing requirements, it is also necessary to install programs such as OpenMPI and MPICH. When the WRF model is installed on a traditional small server, it is necessary to use the udo apt-get install command to install the software, and enter bashrc to set the corresponding software and dependent library path environment variables. Therefore, the installation process of the WRF model is very troublesome, and often due to compatibility issues of software versions, interdependencies, and system environments, the WRF model installation will report errors and take too much time.

[0042] The WRF model is generally used for forecasting on a high-performance computing cluster. The operating environment of the WRF model on the high-performance computing cluster, input / output (I / O) hints, number of nodes, number of open multi-processing (OMP) threads, number of I / O process groups, number of I / O processes per group, decomposition parameters of the computing process in the X-axis and Y-axis directions (nproc_x, nproc_y), quilt-related parameters, Numtiles value and other factors will affect the computational efficiency of the WRF model.

[0043] In order to improve the computational efficiency of the WRF model, it is generally necessary to optimize various factors of the high-performance computing cluster. There are many ways to optimize the high-performance computing cluster. The high-performance computing cluster can be based on the optimization of some parameterization schemes (physical parameters) of the WRF model itself. For example, some physical modules provide support for graphics processing unit (GPU) acceleration. The high-performance computing cluster can be based on the optimization of computing algorithms. For example, I / O optimization algorithm, thread optimization algorithm, etc. And other optimization methods.

[0044] In a related optimization scheme, the WRF model uses quilt technology to make a specified number of processes I / O processes to perform I / O tasks. The WRF model makes the remaining processes computing processes to perform computing tasks. The WRF model overlaps the computing process and the I / O process to reduce the time of displaying I / O during the weather simulation process.

[0045] However, when the WRF model turns on the quilt technology, the calculation process will be reduced, resulting in an increase in calculation time. When the WRF model runs the I / O process, communication between nodes and within nodes will occur, and the communication time will not be hidden. In addition, the time when the WRF model performs the last write I / O operation will not be hidden. The mapping of the calculation process and the I / O process in the WRF model involves a two-dimensional decomposition of the calculation process, and the decomposition parameters need to be optimized. The quilt-related parameters are uncertain, and users need to manually find the optimal value.

[0046] In another related optimization scheme, the WRF model uses quilt technology to optimize the distribution of I / O processes. If the WRF model only modifies the distribution of I / O processes and does not modify the distribution of computing processes, there will be no way to eliminate inter-node communication. If the WRF model modifies the distribution of I / O processes and computing processes at the same time, the computing processes responsible for the same I / O process are distributed on the same nodes as much as possible. However, it is difficult for the WRF model to make the computing processes responsible for the same I / O process distributed on the same nodes as much as possible.

[0047] It can be seen that when the WRF model in the related art optimizes one or several factors, other factors may deteriorate or become more difficult to optimize, resulting in the inability to improve the computational efficiency of the WRF model.

[0048] In order to solve the defects of the optimization scheme of the related art, the embodiments of the present application provide a tuning method, a high-performance computing cluster and a computing device. When the high-performance computing cluster runs the tuning method, the operating parameters of the WRF model can be automatically tuned, which can minimize the communication time exposed to the outside and the total I / O time of the WRF model. During the automatic tuning process, the high-performance computing cluster tunes each parameter of the operating parameters in stages, and tunes a small amount of parameters each time, which can speed up the convergence speed of the tuning. After the high-performance computing cluster completes the tuning of all parameters of the operating parameters, it constitutes the tuning results and stores them in the database, which can enrich the data in the database to improve the accuracy of subsequent tuning.

[0049] Figure 2 Schematic diagram of the architecture of a high performance computing cluster provided in an embodiment of the present application. Figure 2 As shown, the high-performance computing cluster 200 can be divided into a WRF model module 210, a tuning module 220 and a database module 230 according to the execution function. Among them, the WRF model module 210, the tuning module 220 and the database module 230 can all be implemented by software, or can be implemented by hardware, or can be implemented by a combination of software and hardware. Exemplarily, the implementation of the WRF model module 210 is introduced below by taking the WRF model module 210 as an example. Similarly, the implementation of the tuning module 220 and the database module 230 can refer to the implementation of the WRF model module 210.

[0050] As an example of a software functional unit, the WRF model module 210 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the WRF model module 210 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0051] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0052] As an example of a hardware functional unit, the WRF model module 210 may include at least one computing device, such as a computing device, etc. Alternatively, the WRF model module 210 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0053] The multiple computing devices included in the WRF model module 210 can be distributed in the same region or in different regions. The multiple computing devices included in the WRF model module 210 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the WRF model module 210 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as computing devices, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0054] It should be noted that, in other embodiments, the WRF model module 210 can be used to execute any step in the tuning method, the tuning module 220 can be used to execute any step in the tuning method, and the database module 230 can be used to execute any step in the tuning method. The steps that the WRF model module 210, the tuning module 220 and the database module 230 are responsible for implementing can be specified as needed. The full functions of the high-performance computing cluster 200 are realized by respectively implementing different steps in the tuning method through the WRF model module 210, the tuning module 220 and the database module 230.

[0055] The tuning module 220 can receive the tuning parameters sent by other modules, and the tuning parameters can include the number of genetic algorithm populations, the number of genetic algorithm iterations, etc. When the tuning module 220 obtains the tuning parameters, the initial population and the maximum evolutionary generation of the genetic algorithm can be determined. The tuning module 220 can receive the operating parameters of the WRF model run by the WRF model module 210, and the operating parameters can be tuned using the genetic algorithm. The tuning module 220 can receive the existing tuning results stored in the database module 230, and find the parameter values ​​of the operating parameters based on the tuning results, which can improve the tuning efficiency. The tuning module 220 can input the tuning results after tuning into the WRF model module 210 to improve the calculation efficiency of the WRF model. The WRF model module 210 can input the tuning results after tuning into the database module 230, which can enrich the data of the database to improve the accuracy of subsequent tuning.

[0056] In an embodiment of the present application, the operating parameters of the WRF model obtained by the tuning module 220 may be the operating environment, the name of the example, and the number of nodes for running the WRF model, as well as the number of OMP threads, the number of I / O process groups, the number of I / O processes in each group, the decomposition parameters of the calculation process in the X-axis direction and the Y-axis direction (nproc_x, nproc_y), quilt related parameters, Numtiles value, I / O hints, etc. Among them, the operating environment refers to the type of central processing unit (CPU) of the high-performance computing cluster running the WRF model, the size of the memory, the interaction status between nodes, etc. The example name refers to the name of the algorithm for calculating the WRF model running in the high-performance computing cluster, such as copper atoms, water molecule movement, salt dissolution, etc. nproc_x represents the number of blocks cut by the calculation integration in the X-axis direction. nproc_y represents the number of blocks cut by the calculation integration in the Y-axis direction.

[0057] The quilt related parameters indicate the distribution of IO processes and computing processes. For example, IO processes are evenly distributed across nodes to avoid the process entering a node and needing to be forwarded to a node with IO processes, thereby reducing computing.

[0058] The Numtiles value is to further divide the data in a single process into Numtiles blocks for thread processing. During the operation of the WRF model, the specific Numtiles value is related to the domain size and the number of process threads. A reasonable Numtiles value can adapt to the cache size and speed up the execution of the WRF model by improving the cache hit rate.

[0059] I / O hints are used for prompts. For example, a user can give I / O hints to a computing program of a high-performance computing cluster so as to speed up the computing program of the high-performance computing cluster according to the hints.

[0060] When the tuning module 220 tunes the operating parameters, it is necessary to tune the number of OMP threads, the number of I / O process groups, the number of I / O processes in each group, the decomposition parameters of the computing process in the X-axis and Y-axis directions (nproc_x, nproc_y), quilt related parameters, Numtiles values, I / O hints and other parameters.

[0061] The tuning results may include the operating environment, instance name, number of nodes, number of OMP threads, number of I / O process groups, number of I / O processes in each group, decomposition parameters of the computing process in the X-axis and Y-axis directions (nproc_x, nproc_y), quilt-related parameters, Numtiles value, etc.

[0062] The database module 230 refers to a database, such as SQLite, MySQL, SQL Server, etc. The database module 230 is used to receive the tuning results input by the tuning module 220 and store them in the database. The database module 230 can search for the tuning results corresponding to the specific parameters according to the specific parameters. The specific parameters can be one or more of the operating environment, the name of the example, and the number of nodes.

[0063] Figure 3 Schematic diagram of a tuning method provided in an embodiment of the present application. Figure 3 As shown, the tuning method is performed by the tuning module 220, and the specific implementation process is as follows:

[0064] Step S301 : the tuning module 220 obtains operating parameters for running the WRF model.

[0065] Specifically, the tuning module 220 can automatically find the operating parameters of the WRF model when it is running. The operating parameters can be the operating environment, the name of the example, the number of nodes, the number of OMP threads, the number of I / O process groups, the number of I / O processes in each group, the decomposition parameters of the calculation process in the X-axis direction and the Y-axis direction (nproc_x, nproc_y), quilt related parameters, Numtiles value and other parameters. In the embodiment of the present application, the tuning module 220 can simulate the benchmark workload of the WRF model based on several or all of the parameters in the operating parameters. The tuning module 220 uses the benchmark workload to test the end-to-end performance of the benchmark workload, such as the total running time, and the performance of the I / O of the benchmark workload, such as the total I / O time.

[0066] In step S302, the tuning module 220 detects whether the database has the operating parameters based on the operating parameters and the tuning results of the database. In one case, when the tuning module 220 determines that the database has the operating parameters, step S303 is executed. In another case, when the tuning module 220 determines that the database does not have the operating parameters, step S304 is executed.

[0067] Step S303 : the tuning module 220 marks the operating parameter as “flag=1”.

[0068] In step S304 , the tuning module 220 marks the operating parameter as “flag=0”.

[0069] Specifically, the tuning module 220 is configured with a database, and the database stores a large number of tuning results that have been tuned. The tuning results generally include the operating environment, and may not include the name of the example. After the tuning module 220 obtains the operating parameters, it obtains the operating environment and the name of the example in the operating parameters. The tuning module 220 can search for the tuning results that match the operating environment in the operating parameters in the database according to the operating environment in the operating parameters, and obtain the tuning results with the highest similarity to the operating environment in the operating parameters. Then, the tuning module 220 can detect whether the tuning result with the highest similarity has the name of the example in the operating parameters according to the name of the example in the operating parameters. In one case, when the tuning module 220 determines that the tuning result with the highest similarity has the name of the example in the operating parameters, the operating parameter can be marked as "flag=1". In another case, when the tuning module 220 determines that the tuning result with the highest similarity does not have the name of the example in the operating parameters, the operating parameter can be marked as "flag=0".

[0070] Step S305: the tuning module 220 selects several parameters that need to be tuned from the operating parameters. For the convenience of description, the "several parameters that need to be tuned" are defined as "target parameters".

[0071] Exemplarily, the order in which the tuning module 220 tunes the various parameters of the operating parameters may be "number of OMP threads → number of I / O process groups → number of I / O processes in each group → nproc_x and nproc_y → quilt related parameters → Numtiles value → I / O hints". In the embodiment of the present application, when the tuning module 220 tunes the operating parameters, one or a small number of parameters may be selected from the operating parameters for tuning. After completing one tuning, the tuning module 220 again selects one or a small number of parameters from other parameters in the operating parameters for tuning, and so on, until all parameters in the operating parameters are tuned. The tuning module 220 divides the operating parameters into multiple stages for tuning, which can speed up the convergence of the tuning.

[0072] In one example, the tuning module 220 selects groups of target parameters multiple times: the first group includes the number of OMP threads, the number of I / O process groups, and the number of I / O processes in each group; the second group includes quilt related parameters, nproc_x, and nproc_y; and the third group includes Numtiles values ​​and I / O hints.

[0073] In the embodiment of the present application, when the tuning module 220 selects one or a few target parameters from the operating parameters for tuning, other parameters in the operating parameters remain unchanged, which can avoid other parameters in the operating parameters affecting the tuning results of these parameters.

[0074] In step S306, the tuning module 220 detects whether the operating parameter is "flag=1". In one case, when the tuning module 220 determines that the operating parameter is "flag=1", step S307 is executed. In another case, when the tuning module 220 determines that the operating parameter is not "flag=0", step S308 is executed.

[0075] Step S307 : the tuning module 220 uses the tuning results in the database to search for several parameter values ​​corresponding to the target parameters.

[0076] Step S308 : the tuning module 220 generates several parameter values ​​corresponding to the target parameters.

[0077] Specifically, the tuning module 220 is generally connected to a database, and the database can store previously completed tuning results, which can improve the efficiency of tuning the operating parameters by the tuning module 220. When the tuning module 220 determines that the operating parameter is "flag = 1", the tuning module 220 sets the parameter value of the target parameter to the parameter value of the corresponding part of the parameters in the tuning result matched by the database, and several parameter values ​​near the parameter value of the corresponding part of the parameters in the tuning result matched by the database, and obtains several different parameter values ​​of each parameter, providing an initial group for the subsequent operation of the tuning algorithm.

[0078] When the tuning module 220 determines that the operating parameter is "flag = 0", there is no tuning result matching the operating parameter in the database. At this time, the tuning module 220 can randomly generate several different parameter values ​​of the target parameter based on the operation logic set by the user. Alternatively, the tuning module 220 sends the requirement of generating the parameter value of the target parameter to the user, allowing the user to actively set several different parameter values ​​for the target parameter according to the actual situation. Alternatively, other methods are used to obtain several different parameter values ​​corresponding to the target parameter.

[0079] Step S309 : the tuning module 220 obtains a plurality of single-value performance data corresponding to different parameter values ​​based on several parameter values ​​of the target parameter.

[0080] Specifically, after the tuning module 220 obtains several different parameter values ​​of the target parameters, it can randomly match several different parameter values ​​of the target parameters selected this time to obtain multiple groups of parameter values. Each group of parameter values ​​includes a parameter value of the several target parameters selected this time. The tuning module 220 can simulate the benchmark workload of the WRF model based on multiple groups of parameter values ​​to obtain benchmark workloads of multiple WRF models. The tuning module 220 uses the benchmark workload to test the end-to-end performance of the benchmark workload and the I / O performance of the benchmark workload. The tuning module 220 obtains single-value performance data according to the set weighting method based on the end-to-end performance and I / O performance obtained by the test. The tuning module 220 obtains the single-value performance data corresponding to the benchmark workload of each WRF model through multiple calculations.

[0081] In one example, the reduction in the total running time of the end-to-end performance and the total I / O time of the I / O performance compared to the benchmark time is multiplied by 0.5 weights and then added to obtain single-value performance data as fitness.

[0082] Step S310, the tuning module 220 uses a tuning algorithm to tune multiple single-value performance data to obtain the optimal parameter value of the target parameter. The tuning algorithm can be a genetic algorithm, a simulated annealing algorithm, an ant colony algorithm, etc. This application takes the "genetic algorithm" as an example below.

[0083] Genetic algorithm is a randomized search method that evolved from the evolutionary law of the biological world (the genetic mechanism of survival of the fittest). In the initialization phase of the genetic algorithm, the evolutionary algebra counter t=0 is set, the maximum evolutionary algebra T is set, and several individuals are randomly generated as the initial population. In the evaluation phase, the genetic algorithm calculates the fitness of each individual in the population. In the selection phase, the genetic algorithm applies the selection operator to the population, and the optimized individuals are directly inherited to the next generation or the new individuals generated by pairing crossover are inherited to the next generation. In the crossover phase, the genetic algorithm applies the crossover operator to the population. The crossover operator plays a core role in the genetic algorithm. In the mutation phase, the genetic algorithm applies the mutation operator to the population to change the gene values ​​on certain loci of the individual strings in the population. After the selection, crossover and mutation operations, the next generation population is obtained. When the evolutionary algebra counter t=T, the individual with the maximum fitness obtained in the evolution process is taken as the optimal solution, and the optimal solution is output.

[0084] In the embodiment of the present application, the tuning module 220 can input the tuning parameters and the single-value performance data into the genetic algorithm to tune the several target parameters selected this time, obtain the optimal single-value performance data, and obtain the optimal parameter value of the target parameter based on the optimal single-value performance data. Among them, the number of genetic algorithm populations of the tuning parameters determines the initial population of the genetic algorithm. The number of genetic algorithm iterations of the tuning parameters determines the maximum evolutionary algebra T of the genetic algorithm. The single-value performance data calculated from the parameter values ​​of some parameters with different numerical values ​​can be used as the fitness of the genetic algorithm. The optimal single-value performance data is the optimal solution of the genetic algorithm.

[0085] In step S311, the tuning module 220 detects whether all parameters of the operating parameters have been tuned. In one case, when the tuning module 220 determines that all parameters of the operating parameters have been tuned, step S312 is executed. In another case, when the tuning module 220 determines that all parameters of the operating parameters have not been tuned, step S305 is executed.

[0086] Step S312: the tuning module 220 constructs the tuning result and stores it in the database.

[0087] After the tuning module 220 completes the tuning of the target parameter, it detects whether the other parameters of the operating parameter have completed the tuning operation. When the tuning module 220 determines that the other parameters of the operating parameter have not completed the tuning operation, it selects one or more parameters from the other parameters for tuning again. And so on, until all parameters in the operating parameters are tuned. After the tuning module 220 completes the tuning of all parameters in the operating parameters, the tuned operating parameters are recorded in the database, which can enrich the data in the database to improve the accuracy of subsequent tuning.

[0088] The tuning results output by the tuning module 220 include:

[0089] 1. Operating environment and example name. The operating environment and example names are represented by lowercase strings. For example, "DPC environment" is represented by "'dpc'", and "CONUS_2.5KM example" is represented by "'conus2.5'".

[0090] 2. I / O hints, number of nodes, number of I / O process groups, and number of I / O processes per group. I / O hints, number of nodes, number of I / O process groups, and number of I / O processes per group are specified in the run script.

[0091] 3. I / O API. The “I / O API” is also “io_form_****”, specified in the namelist, and located in “&time_control”.

[0092] 4. Quilt related parameters. Among them, quilt related parameters include the number of I / O process groups specified when quilt technology is enabled. nio_groups, specified in namelist, located in "&namelist_quilt". Quilt related parameters also include the number of I / O processes in each I / O process group specified when quilt technology is enabled. nio_tasks_per_group, specified in namelist, located in "&namelist_quilt".

[0093] 5. The decomposition parameters of the computational process in the x-axis direction and the y-axis direction (nproc_x, nproc_y). The domain decomposition parameters include the number of processes in the x-direction when the computational process is divided into a two-dimensional grid. For example, nproc_x, specified in namelist, is located in "&domains". ). The domain decomposition parameters also include the number of processes in the y-direction when the computational process is divided into a two-dimensional grid. For example, nproc_y, specified in namelist, is located in "&domains".

[0094] 6. Numtiles value. The Numtiles value is the further division method of the partial input data obtained by each process, which is specified in namelist and located in "&domains".

[0095] In the embodiment of the present application, the tuning module 220 can automatically tune the operating parameters of the WRF model, which can minimize the communication time exposed to the outside and the total I / O time of the WRF model. During the automatic tuning process, the tuning module 220 tunes the various parameters of the operating parameters in stages, and tunes a small amount of parameters each time, which can speed up the convergence speed of the tuning. After the tuning module 220 completes the tuning of all parameters of the operating parameters, it forms a tuning result and stores it in the database, which can enrich the data in the database to improve the accuracy of subsequent tuning.

[0096] Figure 4 Schematic diagram of a computing device provided in an embodiment of the present application. Figure 4 As shown, the computing device 400 includes a bus 410, a processor 420, a memory 430, and a communication interface 440. The processor 420, the memory 430, and the communication interface 440 communicate with each other through the bus 410. The computing device 400 can be a server, a computer, a portable notebook, a cabinet, etc. It should be understood that the present application does not limit the number of processors and memories in the computing device 400.

[0097] The bus 410 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 The bus 410 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 410 may include a path for transmitting information between various components of the computing device 400 (eg, the processor 420, the memory 430, and the communication interface 440).

[0098] The processor 420 may include any one or more processors such as a CPU, a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0099] The memory 430 may include a volatile memory, such as a random access memory (RAM). The memory 430 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0100] The memory 430 stores executable program codes, and the processor 420 executes the executable program codes to respectively implement the functions of the aforementioned WRF model module 210, the tuning module 220, and the database module 230, thereby implementing the tuning method. That is, the memory 430 stores instructions for executing the tuning method.

[0101] Alternatively, the memory 430 stores executable codes, and the processor 420 executes the executable codes to respectively implement the functions of the aforementioned high-performance computing cluster 200, thereby implementing the tuning method. That is, the memory 430 stores instructions for executing the tuning method.

[0102] The communication interface 440 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 400 and other devices or a communication network.

[0103] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0104] like Figure 5 As shown, the computing device cluster includes at least one computing device 400. The memory 430 in one or more computing devices 400 in the computing device cluster may store the same instructions for executing the tuning method.

[0105] In some possible implementations, the memory 430 of one or more computing devices 400 in the computing device cluster may also store partial instructions for executing the tuning method. In other words, the combination of one or more computing devices 100 may jointly execute instructions for executing the tuning method.

[0106] It should be noted that the memory 430 in different computing devices 400 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the high-performance computing cluster 200. That is, the instructions stored in the memory 430 in different computing devices 400 can implement the functions of one or more modules in the WRF model module 210, the tuning module 220 and the database module 230.

[0107] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Figure 6 A possible implementation is shown. Figure 6 As shown, two computing devices 400A and computing device 400B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device. In this type of possible implementation, the memory 430 in the computing device 400A stores instructions for executing the functions of the WRF model module 210. At the same time, the memory 430 in the computing device 400B stores instructions for executing the functions of the tuning module 220 and the database module 230.

[0108] Figure 6 The connection method between the computing device clusters shown may be based on the consideration that the tuning method provided in the present application requires a large amount of data storage, and therefore the functions implemented by the tuning module 220 and the database module 230 may be handed over to the computing device 400B for execution.

[0109] It should be understood that Figure 6The functions of the computing device 400A shown in FIG. 4 may also be completed by multiple computing devices 400. Similarly, the functions of the computing device 400B may also be completed by multiple computing devices 400.

[0110] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Figure 5 and Figure 6 The connection mode of the computing device cluster is different in that the memory 430 in one or more computing devices 400 in the computing device cluster may store the same instructions for executing the tuning method.

[0111] In some possible implementations, the memory 430 of one or more computing devices 400 in the computing device cluster may also store partial instructions for executing the tuning method. In other words, the combination of one or more computing devices 400 may jointly execute instructions for executing the tuning method.

[0112] It should be noted that the memory 430 in different computing devices 400 in the computing device cluster may store different instructions for executing partial functions of the computing device 400. That is, the instructions stored in the memory 430 in different computing devices 400 may implement the functions of one or more devices in the high-performance computing cluster 200.

[0113] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the tuning method.

[0114] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the tuning method.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tuning method, characterized in that: Applied to a computing device, the computing device is used to run a meteorological research and forecasting WRF model, the method comprising: Obtaining operating parameters for running the WRF model; Based on the parameters of the first group in the operating parameters, multiple parameter values ​​of the parameters are generated; the multiple parameters in the operating parameters are divided into multiple groups, the multiple groups include the first group, and the first group includes one or more parameters; Input multiple parameter values ​​of each parameter into the tuning model to obtain the tuned parameter value of each parameter.

2. The method according to claim 1, characterized in that The operating parameters include the operating environment and the example name. The generating a plurality of parameter values ​​of each parameter based on each parameter of the first group in the operating parameters comprises: Compare the optimization results stored in the database according to the operating environment in the operating parameters; In response to the similarity between the operating environment in the target tuning result and the operating environment in the operating parameter being greater than a set threshold, detecting whether the target tuning result contains a calculation example name of the operating parameter; In response to the target tuning result having the calculation instance name of the running parameter, the parameter value of each parameter in the target tuning result and a set number of parameter values ​​near the parameter value of each parameter are obtained to obtain multiple parameter values ​​of each parameter.

3. The method according to claim 2, characterized in that The method further comprises: In response to the target tuning result not having the calculation instance name of the running parameter, a plurality of parameter values ​​of each parameter are randomly generated.

4. The method according to any one of claims 1 to 3, characterized in that: The step of inputting the multiple parameter values ​​of the various parameters into the tuning model to obtain the tuned parameter values ​​of the various target parameters includes: Randomly combining multiple parameter values ​​of different parameters to obtain multiple groups of parameter values, wherein the multiple groups of parameter values ​​include a parameter value of each parameter; Simulating the benchmark workloads of the WRF model based on the multiple groups of parameter values ​​respectively to obtain multiple benchmark workloads of the WRF model; Performing weighted calculations on the end-to-end performance and input / output performance of the benchmark workloads of the plurality of WRF models to obtain a plurality of single-value performance data; The plurality of single-value performance data are input into the tuning model to obtain the tuned parameter values ​​of the respective target parameters.

5. The method according to claim 4, characterized in that The tuning model is one of a genetic algorithm, a simulated annealing algorithm and an ant colony algorithm.

6. The method according to claim 4 or 5, characterized in that: The tuning model is a genetic algorithm. The step of inputting the plurality of single-value performance data into the genetic algorithm to obtain the optimized parameter values ​​of the respective target parameters comprises: Acquire tuning parameters, wherein the tuning parameters include the number of the genetic algorithm population and the number of iterations of the genetic algorithm; Inputting the plurality of single-value performance data, the population size of the genetic algorithm and the number of iterations of the genetic algorithm into the genetic algorithm to obtain the optimized single-value performance data; Based on the tuned single-value performance data, parameter values ​​of different parameters corresponding to the tuned single-value performance data are determined.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Detecting whether the parameters of the other groups among the operating parameters except the first group have been tuned; In response to the fact that the optimization of the operating parameters of the groups other than the first group is completed, the optimized operating parameters are stored in a database.

8. A tuning device, characterized in that: include: A transceiver, for obtaining operating parameters for running the WRF model; A processor, configured to generate a plurality of parameter values ​​for each parameter based on each parameter of a first group in the operating parameters; A plurality of parameters in the operating parameters are divided into a plurality of groups, the plurality of groups including the first group, the first group including one or more parameters; Input multiple parameter values ​​of each parameter into the tuning model to obtain the tuned parameter value of each parameter.

9. The device according to claim 8, characterized in that The operating parameters include the operating environment and the example name. The processor is specifically configured to compare the tuning results stored in the database according to the operating environment in the operating parameters; In response to the similarity between the operating environment in the target tuning result and the operating environment in the operating parameter being greater than a set threshold, detecting whether the target tuning result contains a calculation example name of the operating parameter; In response to the target tuning result having the calculation instance name of the running parameter, the parameter value of each parameter in the target tuning result and a set number of parameter values ​​near the parameter value of each parameter are obtained to obtain multiple parameter values ​​of each parameter.

10. The device according to claim 9, characterized in that The processor is further configured to randomly generate a plurality of parameter values ​​for each parameter in response to the target tuning result not having a calculation instance name for the operating parameter.

11. The device according to any one of claims 8 to 10, characterized in that: The processor is specifically configured to randomly combine multiple parameter values ​​of different parameters to obtain multiple groups of parameter values, wherein the multiple groups of parameter values ​​include a parameter value of each parameter; Simulating the benchmark workloads of the WRF model based on the multiple groups of parameter values ​​respectively to obtain multiple benchmark workloads of the WRF model; Performing weighted calculations on the end-to-end performance and input / output performance of the benchmark workloads of the plurality of WRF models to obtain a plurality of single-value performance data; The plurality of single-value performance data are input into the tuning model to obtain the tuned parameter values ​​of the respective target parameters.

12. The device according to claim 11, characterized in that The tuning model is one of a genetic algorithm, a simulated annealing algorithm and an ant colony algorithm.

13. The device according to claim 10 or 11, characterized in that The tuning model is a genetic algorithm. The processor is specifically used to obtain tuning parameters, wherein the tuning parameters include the number of the genetic algorithm population and the number of iterations of the genetic algorithm; Inputting the plurality of single-value performance data, the population size of the genetic algorithm and the number of iterations of the genetic algorithm into the genetic algorithm to obtain the optimized single-value performance data; Based on the tuned single-value performance data, parameter values ​​of different parameters corresponding to the tuned single-value performance data are determined.

14. The device according to any one of claims 8 to 13, characterized in that: The processor is further configured to detect whether the parameters of the other groups of the operating parameters except the first group have been tuned; In response to the fact that the optimization of the operating parameters of the groups other than the first group is completed, the optimized operating parameters are stored in a database.

15. A computing device, characterized in that: include: at least one memory, At least one processor, wherein the at least one processor is configured to execute instructions stored in the at least one memory, so that the computing device executes the method according to any one of claims 1 to 7.

16. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product, characterized in that The computer program product stores instructions, and when the instructions are executed by a computer, the computer implements the method according to any one of claims 1 to 7.