Meteorological numerical simulation multi-parameter dynamic collaborative optimization processing method and system based on artificial intelligence

Abstract

The invention discloses a meteorological numerical simulation multi-parameter dynamic collaborative optimization processing method and system based on artificial intelligence, and relates to the technical field of analysis and evaluation of meteorological simulation output data. Comprising the following steps: S1, parameter combination setting; s2, generating a configuration file; s3, data preprocessing; s4, meteorological simulation; s5, copying a result; s6, data processing; s7, merging the files; s8, cleaning temporary files; s9, data analysis; and S10, performing cyclic judgment. According to the method, a large number of different physical parameter combinations are automatically generated through multi-layer nested for circulation, so that the complexity and errors of manually setting parameters are avoided, and the comprehensiveness and efficiency of simulation are improved; a plurality of steps of meteorological simulation are integrated into one script, so that automation of the whole simulation process is realized, manual intervention is reduced, and the working efficiency and the simulation accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of analysis and evaluation of meteorological simulation output data, and in particular to an artificial intelligence-based meteorological numerical simulation multi-parameter dynamic collaborative optimization processing method and system. Background Art

[0002] The current meteorological simulation process has the following shortcomings and deficiencies:

[0003] 1. Difficulty managing parameter combinations: In meteorological simulations, manually managing a large number of different physical parameter combinations is a tedious and error-prone task. Different physical parameter combinations can significantly impact simulation results, requiring extensive experimentation to determine the optimal combination. Traditional methods may require manual modification of configuration files, which can easily lead to omissions or incorrect parameter settings.

[0004] ② Insufficient automation in the simulation process: Weather simulations typically involve multiple steps, such as data preparation, model execution, and result processing. If these steps cannot be automated, they will consume significant time and manpower and be prone to human error.

[0005] ③ Complex data comparison and evaluation: In meteorological simulations, different simulation results need to be compared and evaluated to determine the accuracy and reliability of the simulation. Traditional methods may require manual code to calculate various statistical indicators and are inefficient when processing large amounts of data.

[0006] ④ Inconvenience in result management and accumulation: As the number of simulation tests increases, a large number of result files will be generated. Without an effective management and accumulation mechanism, the result files will be scattered, difficult to find and compare.

[0007] Therefore, it is an urgent problem for those skilled in the art to propose a multi-parameter dynamic collaborative optimization processing method and system for meteorological numerical simulation based on artificial intelligence to solve the difficulties existing in the existing technology. Summary of the Invention

[0008] In view of this, the present invention provides an artificial intelligence-based multi-parameter dynamic collaborative optimization processing method and system for meteorological numerical simulation, which automatically generates a large number of different physical parameter combinations through multi-layer nested for loops, avoiding the tediousness and errors of manual parameter setting and improving the comprehensiveness and efficiency of the simulation.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] A multi-parameter dynamic collaborative optimization processing method for meteorological numerical simulation based on artificial intelligence includes the following steps:

[0011] S1. Parameter combination setting: Through multi-layer nested for loops, multiple physical parameters are combined with different values, and a set of parameter values is selected in each loop;

[0012] S2. Configuration file generation: Generate an input file based on the parameter value combination selected in multiple cycles;

[0013] S3. Data preprocessing: Run the first design program to preprocess the input data;

[0014] S4. Weather simulation: Use mpirun to run the second design program in parallel, perform weather simulation based on the information in the input file, generate output files, and obtain the initial simulation results;

[0015] S5. Copy results: copy the generated output files to the specified directory;

[0016] S6. Data processing: Run the first script to process the initial simulation results and generate an intermediate CSV file;

[0017] S7. File merging: Create a new CSV file and append the contents of the intermediate CSV file to the new file to complete the data merging.

[0018] S8. Temporary file cleanup: Delete the temporary CSV files generated in the middle to free up disk space;

[0019] S9. Data analysis: Run the second script to perform data analysis on the merged CSV file, calculate multiple statistical indicators, and save the results to the results.csv file;

[0020] S10. Loop judgment: Check whether there are any unprocessed parameter combinations. If so, return to S2 to continue processing. Otherwise, end the program and output the final processing result.

[0021] Optionally, the physical parameters in S1 include mp_physics, bl_pbl_physics, cu_physics, ra_sw, ra_lw, and sf_surface_physics.

[0022] Optionally, the input file generated in S2 includes various configuration information required for running the meteorological model WRF, wherein the configuration information includes time control, regional settings, and physical parameters.

[0023] Optionally, the first script in S6 is an NCL script including country.ncl and moonlight.ncl.

[0024] Optionally, the statistical indicators in S9 include standardized mean error, standardized mean deviation and correlation coefficient.

[0025] An artificial intelligence-based meteorological numerical simulation multi-parameter dynamic collaborative optimization processing system, applying any of the above-mentioned artificial intelligence-based meteorological numerical simulation multi-parameter dynamic collaborative optimization processing methods, comprising: a parameter combination setting module, a configuration file generation module, a data preprocessing module, a meteorological simulation module, a result replication module, a data processing module, a file merging module, a temporary file cleaning module, a data analysis module and a loop judgment module;

[0026] The parameter combination setting module is connected to the input end of the configuration file generation module and is used to combine different values of multiple physical parameters through multi-layer nested for loops, selecting a set of parameter values in each loop;

[0027] A configuration file generation module is connected to the input terminal of the data preprocessing module and is used to generate an input file based on the combination of parameter values selected in multiple cycles;

[0028] A data preprocessing module is connected to the input terminal of the meteorological simulation module and is used to run the first design program to preprocess the input data;

[0029] The meteorological simulation module is connected to the input terminal of the result copy module, and uses mpirun to run the second design program in parallel, performs meteorological simulation according to the information in the input file, generates an output file, and obtains the initial simulation results;

[0030] A result copy module is connected to the input terminal of the data processing module and is used to copy the generated output file to a specified directory;

[0031] A data processing module is connected to the input end of the file merging module and is used to run the first script to process the initial simulation results and generate an intermediate CSV file;

[0032] The file merging module is connected to the input end of the temporary file cleaning module, and is used to create a new CSV file and append the content of the intermediate CSV file to the new file to complete the data merging;

[0033] The temporary file cleaning module is connected to the input end of the data analysis module to delete the temporary CSV files generated in the middle and free up disk space.

[0034] A data analysis module is connected to the input end of the loop judgment module and is used to run the second script to perform data analysis on the merged CSV file, calculate multiple statistical indicators, and save the results to the results.csv file;

[0035] The loop judgment module is used to check whether there are any unprocessed parameter combinations. If so, it returns to the configuration file generation module to continue processing. Otherwise, the program ends and the final processing result is output.

[0036] It can be seen from the above technical solutions that, compared with the prior art, the present invention provides a method and system for dynamic collaborative optimization of multiple parameters in meteorological numerical simulation based on artificial intelligence, which has the following beneficial effects:

[0037] (1) A large number of different physical parameter combinations are automatically generated through multi-layer nested for loops, avoiding the tediousness and errors of manual parameter setting and improving the comprehensiveness and efficiency of the simulation;

[0038] (2) Integrating multiple steps of meteorological simulation (data preprocessing, simulation operation, result processing, and data analysis) into one script, automating the entire simulation process, reducing manual intervention, and improving work efficiency and simulation accuracy;

[0039] (3) The analysis data of all simulation results are saved uniformly in the results.csv file, and new results are appended when the file exists, so as to facilitate subsequent comparison and further research of different simulation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0041] Figure 1 A flowchart of a multi-parameter dynamic collaborative optimization processing method for meteorological numerical simulation based on artificial intelligence provided by the present invention;

[0042] Figure 2 This is a structural block diagram of a multi-parameter dynamic collaborative optimization processing system for meteorological numerical simulation based on artificial intelligence provided by the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] Reference Figure 1 As shown, the present invention discloses a multi-parameter dynamic collaborative optimization processing method for meteorological numerical simulation based on artificial intelligence, comprising the following steps:

[0045] S1. Parameter combination setting: Through multi-layer nested for loops, multiple physical parameters are combined with different values, and a set of parameter values is selected in each loop;

[0046] S2. Configuration file generation: Generate an input file based on the parameter value combination selected in multiple cycles;

[0047] S3. Data preprocessing: Run the first design program to preprocess the input data;

[0048] S4. Weather simulation: Use mpirun to run the second design program in parallel, perform weather simulation based on the information in the input file, generate output files, and obtain the initial simulation results;

[0049] S5. Copy results: copy the generated output files to the specified directory;

[0050] S6. Data processing: Run the first script to process the initial simulation results and generate an intermediate CSV file;

[0051] S7. File merging: Create a new CSV file and append the contents of the intermediate CSV file to the new file to complete the data merging.

[0052] S8. Temporary file cleanup: Delete the temporary CSV files generated in the middle to free up disk space;

[0053] S9. Data analysis: Run the second script to perform data analysis on the merged CSV file, calculate multiple statistical indicators, and save the results to the results.csv file;

[0054] S10. Loop judgment: Check whether there are any unprocessed parameter combinations. If so, return to S2 to continue processing. Otherwise, end the program and output the final processing result.

[0055] Furthermore, the physical parameters in S1 include mp_physics, bl_pbl_physics, cu_physics, ra_sw, ra_lw, and sf_surface_physics.

[0056] Furthermore, the input file generated in S2 includes various configuration information required for the operation of the meteorological model WRF, wherein the configuration information includes time control, regional settings, and physical parameters.

[0057] Furthermore, the first script in S6 is an NCL script including country.ncl and moonlight.ncl.

[0058] Furthermore, the statistical indicators in S9 include standardized mean error, standardized mean deviation and correlation coefficient.

[0059] In a specific embodiment, the following are included:

[0060] 1. Experimental conditions and related parameters:

[0061] Hardware environment: The same high-performance computing cluster is equipped with multiple computing nodes, each with a certain number of CPU cores and memory.

[0062] Software environment: The same version of the weather simulation software WRF (Weather Research and Forecasting Model), as well as the supporting NCL (NCAR Command Language) and R language environments.

[0063] Simulation area: Select the same meteorological simulation area, which has typical meteorological characteristics, such as complex terrain, diverse climatic conditions, etc.

[0064] Simulation period: Select the same time period for meteorological simulation, for example, December 20, 2021 to February 10, 2022.

[0065] Physical parameters:

[0066] The present invention automatically generates multiple physical parameter combinations through scripts, covering mp_physics (14 values), bl_pbl_physics (14 values), cu_physics (13 values), ra_sw (8 values), ra_lw (6 values), and sf_surface_physics (7 values), with a total of 14×14×13×8×6×7=987840 combinations.

[0067] Traditional method: Manually select 10 different physical parameter combinations for simulation.

[0068] 2. Experimental process

[0069] The present invention: Run the 3-WRFAutoOptimize.sh script to automatically complete the entire process of parameter setting, data preprocessing, simulation operation, result processing and analysis. The specific steps are:

[0070] S1. Parameter combination setting: Through multi-layer nested for loops, multiple physical parameters are combined with different values, and a set of parameter values is selected in each loop;

[0071] S2. Configuration file generation: Generate an input file (namelist.input) based on the parameter value combinations selected in multiple cycles;

[0072] S3. Data preprocessing: Run the first design (real.exe) program to preprocess the input data;

[0073] S4. Weather simulation: Use mpirun to run the second design (wrf.exe) program in parallel, perform weather simulation based on the information in the input (namelist.input) file, generate the output (wrfout) file, and obtain the initial simulation results.

[0074] S5. Copy results: Copy the generated output (wrfout) file to the specified directory ( / modpub / data / DataShare / data / );

[0075] S6. Data processing: Run the first (NCL) script to process the initial simulation results and generate an intermediate CSV file;

[0076] S7. File merging: Create a new CSV file and append the contents of the intermediate CSV file to the new file to complete the data merging.

[0077] S8. Temporary file cleanup: Delete the temporary CSV files generated in the middle to free up disk space;

[0078] S9. Data analysis: Run the second (R) script to perform data analysis on the merged CSV file, calculate multiple statistical indicators, and save the results to the results.csv file;

[0079] S10. Loop judgment: Check whether there are any unprocessed parameter combinations. If so, return to S2 to continue processing. Otherwise, end the program and output the final processing result.

[0080] Traditional methods:

[0081] Manually set 10 different physical parameter combinations and modify the namelist.input file respectively.

[0082] Manually execute real.exe for data preprocessing and mpirun-np 23. / wrf.exe for meteorological simulation.

[0083] Manually copy the simulation result file, run the NCL script to process the data, and generate an intermediate CSV file.

[0084] Manually merge the intermediate CSV files and delete the temporary files.

[0085] Manually run R scripts for data analysis.

[0086] 3. Experimental results

[0087] ① Efficiency comparison

[0088] The present invention: It takes about 100 hours to complete the simulation and analysis of all parameter combinations (the specific time depends on the computing resources and the complexity of the simulation).

[0089] The traditional method takes about 30 hours to complete the simulation and analysis of 10 parameter combinations. If the simulation of the same number of parameter combinations as the present invention is to be completed, it is estimated to take about 30×(987840÷10)=2963520 hours, which is much longer than the present invention method.

[0090] ②Accuracy comparison

[0091] The accuracy is evaluated by calculating statistical indicators such as the correlation coefficient and root mean square error between the simulation results and the actual observation data.

[0092] The present invention: in the simulation results of all parameter combinations, the correlation coefficient corresponding to the found optimal parameter combination reaches 0.9, and the root mean square error is 0.5.

[0093] Traditional method: Due to the limited number of manually selected parameter combinations, the correlation coefficient corresponding to the optimal parameter combination is only 0.8, and the root mean square error is 0.8.

[0094] ③ Comprehensive comparison

[0095] The present invention simulates 987,840 different combinations of physical parameters, comprehensively covering all possible situations.

[0096] Traditional method: Only 10 parameter combinations were simulated, which made it impossible to fully explore the impact of different parameter combinations on the simulation results.

[0097] In another specific embodiment, the implementation method is:

[0098] ①Parameter setting part

[0099] Method 1: Set the parameter value range by hard-coding in the script, as shown in the current code:

[0100] form_physics in 1234567891013141617do

[0101] #...done

[0102] Method 2: Store the parameter value range in an external file, and read the file to obtain the parameter value when the script is running. For example, create a parameters.txt file with the following content:

[0103]

[0104] ②Result processing part

[0105] Method 1: Use NCL scripts for data processing, as shown in the current code:

[0106] ncl~ / country.ncl

[0107] ncl~ / yuehua.ncl#...

[0108] Method 2: You can use Python scripts instead of NCL scripts for data processing. For example, create a process_data.py script:

[0109] import pandas as pd

[0110] #Read wrfout file and process it

[0111] #...

[0112] #Save the processing results as a CSV file

[0113] df.to_csv('output.csv',index=False)

[0114] Then call the Python script in your script:

[0115] pythonprocess_data.py

[0116] The above comparative tests and descriptions of the implementation methods fully demonstrate the advantages of the present invention in meteorological simulation, as well as the replacement methods of key steps, and provide ideas for further optimization and expansion of the invention.

[0117] and Figure 1 Corresponding to the method described above, the embodiment of the present invention also provides a multi-parameter dynamic collaborative optimization processing system for meteorological numerical simulation based on artificial intelligence, which is used to Figure 1 The specific implementation of the method is shown in the following diagram: Figure 2 As shown, it includes: parameter combination setting module, configuration file generation module, data preprocessing module, meteorological simulation module, result copy module, data processing module, file merging module, temporary file cleaning module, data analysis module and loop judgment module;

[0118] The parameter combination setting module is connected to the input end of the configuration file generation module and is used to combine different values of multiple physical parameters through multi-layer nested for loops, selecting a set of parameter values in each loop;

[0119] A configuration file generation module is connected to the input terminal of the data preprocessing module and is used to generate an input file based on the combination of parameter values selected in multiple cycles;

[0120] A data preprocessing module is connected to the input terminal of the meteorological simulation module and is used to run the first design program to preprocess the input data;

[0121] The meteorological simulation module is connected to the input terminal of the result copy module, and uses mpirun to run the second design program in parallel, performs meteorological simulation according to the information in the input file, generates an output file, and obtains the initial simulation results;

[0122] A result copy module is connected to the input terminal of the data processing module and is used to copy the generated output file to a specified directory;

[0123] A data processing module is connected to the input end of the file merging module and is used to run the first script to process the initial simulation results and generate an intermediate CSV file;

[0124] The file merging module is connected to the input end of the temporary file cleaning module, and is used to create a new CSV file and append the content of the intermediate CSV file to the new file to complete the data merging;

[0125] The temporary file cleaning module is connected to the input end of the data analysis module to delete the temporary CSV files generated in the middle and free up disk space.

[0126] A data analysis module is connected to the input end of the loop judgment module and is used to run the second script to perform data analysis on the merged CSV file, calculate multiple statistical indicators, and save the results to the results.csv file;

[0127] The loop judgment module is used to check whether there are any unprocessed parameter combinations. If so, it returns to the configuration file generation module to continue processing. Otherwise, the program ends and the final processing result is output.

[0128] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0129] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-parameter dynamic collaborative optimization processing method for meteorological numerical simulation based on artificial intelligence, characterized in that: The following steps are involved: S1. Parameter combination setting: Through multi-layer nested for loops, multiple physical parameters are combined with different values, and a set of parameter values is selected in each loop; S2. Configuration file generation: Generate an input file based on the parameter value combination selected in multiple cycles; S3. Data preprocessing: Run the first design program to preprocess the input data; S4. Weather simulation: Use mpirun to run the second design program in parallel, perform weather simulation based on the information in the input file, generate output files, and obtain the initial simulation results; S5. Copy results: copy the generated output files to the specified directory; S6. Data processing: Run the first script to process the initial simulation results and generate an intermediate CSV file; S7. File merging: Create a new CSV file and append the contents of the intermediate CSV file to the new file to complete the data merging. S8. Temporary file cleanup: Delete the temporary CSV files generated in the middle to free up disk space; S9. Data analysis: Run the second script to perform data analysis on the merged CSV file, calculate multiple statistical indicators, and save the results to the results.csv file; S10. Loop judgment: Check whether there are any unprocessed parameter combinations. If so, return to S2 to continue processing. Otherwise, end the program and output the final processing result.

2. The method for multi-parameter dynamic collaborative optimization processing of meteorological numerical simulation based on artificial intelligence according to claim 1, characterized in that: The physical parameters in S1 include mp_physics, bl_pbl_physics, cu_physics, ra_sw, ra_lw, and sf_surface_physics.

3. The method for multi-parameter dynamic collaborative optimization processing of meteorological numerical simulation based on artificial intelligence according to claim 1 is characterized in that: The input file generated in S2 includes various configuration information required for the operation of the meteorological model WRF, where the configuration information includes time control, regional settings, and physical parameters.

4. The method for multi-parameter dynamic collaborative optimization of meteorological numerical simulation based on artificial intelligence according to claim 1, characterized in that: The first script in S6 is the NCL script including country.ncl and moonlight.ncl.

5. The method for multi-parameter dynamic collaborative optimization processing of meteorological numerical simulation based on artificial intelligence according to claim 1 is characterized in that: The statistical indicators in S9 include standardized mean error, standardized mean deviation and correlation coefficient.

6. A multi-parameter dynamic collaborative optimization processing system for meteorological numerical simulation based on artificial intelligence, characterized in that: An artificial intelligence-based meteorological numerical simulation multi-parameter dynamic collaborative optimization processing method according to any one of claims 1 to 5 is applied, comprising: a parameter combination setting module, a configuration file generation module, a data preprocessing module, a meteorological simulation module, a result copy module, a data processing module, a file merging module, a temporary file cleaning module, a data analysis module, and a loop judgment module; The parameter combination setting module is connected to the input end of the configuration file generation module and is used to combine different values of multiple physical parameters through multi-layer nested for loops, selecting a set of parameter values in each loop; A configuration file generation module is connected to the input terminal of the data preprocessing module and is used to generate an input file based on the combination of parameter values selected in multiple cycles; A data preprocessing module is connected to the input terminal of the meteorological simulation module and is used to run the first design program to preprocess the input data; The meteorological simulation module is connected to the input terminal of the result copy module, and uses mpirun to run the second design program in parallel, performs meteorological simulation according to the information in the input file, generates an output file, and obtains the initial simulation results; A result copy module is connected to the input terminal of the data processing module and is used to copy the generated output file to a specified directory; A data processing module is connected to the input end of the file merging module and is used to run the first script to process the initial simulation results and generate an intermediate CSV file; The file merging module is connected to the input end of the temporary file cleaning module, and is used to create a new CSV file and append the content of the intermediate CSV file to the new file to complete the data merging; The temporary file cleaning module is connected to the input end of the data analysis module to delete the temporary CSV files generated in the middle and free up disk space. A data analysis module is connected to the input end of the loop judgment module and is used to run the second script to perform data analysis on the merged CSV file, calculate multiple statistical indicators, and save the results to the results.csv file; The loop judgment module is used to check whether there are any unprocessed parameter combinations. If so, it returns to the configuration file generation module to continue processing. Otherwise, the program ends and the final processing result is output.

Images