An Ensemble Forecast Verification System

Abstract

The invention relates to the technical field of weather forecast, in particular to an ensemble forecast verification system. Parameters are set and module flow is controlled by shell script language, score calculation and statistics are performed by Fortran90 language, and finally a map is drawn by GrADS script language. The system comprises the following functions: radar combination reflectivity test, precipitation target similarity space test, ensemble forecast En-FSS space test, score calculation, significance test and statistical drawing. Compared with the prior art, the invention integrates radar combination emissivity test, precipitation target similarity space test, ensemble forecast En-FSS space test, score calculation, significance test and statistical drawing into the system, thereby enhancing the ensemble forecast verification and evaluation capability of the tool and providing better support and service for the research and development and improvement of ensemble forecast technology.

Description

Technical Field

[0001] The invention relates to the technical field of weather forecasting, in particular to an ensemble forecast verification system. Background Art

[0002] With the continuous development of numerical weather forecasting technology, ensemble forecasting services and technologies are also constantly updated and upgraded. The inspection and evaluation of the experimental results of ensemble forecasting technology is an important part of the research work. To conduct efficient and convenient inspection and evaluation of a large number of ensemble forecasting test results, a verification system that can meet the required inspection scores and graphics is needed.

[0003] Based on the above reasons, the present invention designs an ensemble forecast verification system, which integrates the process control, data processing, score calculation, significance test and statistical drawing of ensemble forecast into one system, thereby realizing efficient and convenient evaluation of the results of a large number of ensemble forecast experiments. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an ensemble forecast verification system that integrates the process control, data processing, score calculation, significance test and statistical drawing of ensemble forecast into one system, thereby realizing efficient and convenient evaluation of the results of a large number of ensemble forecast experiments.

[0005] In order to achieve the above-mentioned purpose, the present invention provides an ensemble forecast verification system, which uses shell script language to set parameters, module flow control, Fortran90 language to perform score calculation and statistics, and finally draws a map through GrADS script language; the system includes the following functions: radar combination reflectivity test, precipitation target similarity space test, ensemble forecast En-FSS space test, score calculation, significance test and statistical drawing.

[0006] The radar combination reflectivity verification uses the radar combination reflectivity observation data used in the CMA-MESO business model data assimilation as the verification and comparison data, and calculates the probability forecast score of the radar combination reflectivity ensemble forecast, including the AROC score and the BS score.

[0007] The precipitation target similarity space test includes:

[0008] Analyze the influence of various parameters of comprehensive evaluation index of precipitation probability forecast on the evaluation of precipitation probability forecast effect:

[0009] Using ensemble forecast precipitation forecast data and high-resolution precipitation data from the National Meteorological Information Center's three-source fusion grid analysis, different influence radii, precipitation intensity thresholds, and probability thresholds were selected to identify precipitation probability forecast targets and observation targets. The morphological characteristic parameters of each precipitation target individual were calculated. The morphological characteristic parameters included center position, area, major axis direction, and aspect ratio. The morphological characteristic similarity parameters of the precipitation probability forecast target and the observation target pair included center distance, minimum edge distance, overlapping area ratio, area ratio, and axial difference. The effects of different influence radii, precipitation intensity thresholds, probability thresholds, and morphological characteristic parameters of precipitation targets on the evaluation of precipitation probability forecast effects were analyzed.

[0010] Construct a comprehensive evaluation index for spatial verification of precipitation probability forecast:

[0011] According to the analysis of the influence of various parameters of the comprehensive evaluation index of precipitation probability forecast on the evaluation of precipitation probability forecast effect, the main spatial morphological characteristic parameters of precipitation targets are selected, and the weights of various main parameters are determined through weight coefficient sensitivity tests to reasonably express the precipitation probability forecast effect; according to the influence radius, precipitation intensity threshold, probability threshold and various morphological characteristic parameters of precipitation targets determined by the test, a target-oriented precipitation probability forecast spatial verification comprehensive evaluation index is constructed;

[0012] The specific steps are:

[0013] S3-1, first select the influence radius R and precipitation intensity threshold L, smooth the grid analysis precipitation field and identify the target, assuming that X observation targets are identified;

[0014] S3-2, select the probability threshold P, smooth the probability forecast precipitation field and identify the target, assuming that Y forecast targets are identified;

[0015] S3-3, according to the matching criteria, assuming that M pairs of observations and forecast targets are successfully matched, then M≤Min(X, Y);

[0016] The target-oriented convective-scale ensemble forecast precipitation probability forecast spatial verification comprehensive evaluation index SIM can be expressed as:

[0017] Formula 1:

[0018] L is the number of precipitation intensity thresholds, ρ l is the weight of the precipitation intensity threshold l; P is the number of probability thresholds, τ p is the weight of the probability threshold p, S p is the similarity of morphological features when the probability threshold is p:

[0019] Formula 2:

[0020] M is the number of observation and forecast target pairs, A m is the area of ​​the mth observation and forecast target pair, A t is the total area of ​​all observed and predicted target pairs, K is the number of morphological features, ε k is the weight of the kth morphological feature, and the similarity function F of the kth morphological feature parameter k Determined through sensitivity tests, the value range is [0,1]; F k (m) is the contribution function of the kth morphological characteristic parameters of the mth observation and forecast target pair, namely, center distance, minimum edge distance, overlapping area ratio, area ratio, and axial difference to the morphological characteristic similarity;

[0021] The range of SIM, the comprehensive evaluation index of spatial verification of precipitation probability forecast, is [0, 1], where 0 represents no skill and 1 represents an ideal score. When the number of targets identified by the observation field and the forecast field is 0, SIM is undefined; when the number of targets identified by the observation field and the forecast field, one of them is 0, SIM = 0;

[0022] The En-FSS spatial verification of ensemble forecast is as follows: only the spatial extension is considered and the space is verified by the En-FSS scoring method.

[0023] The score calculation is responsible for the calculation and output of the main test score of the ensemble forecast for a single reporting time, and stores the precipitation test station information list information and model terrain height field data.

[0024] The significance test performs a significance test on the score difference between the two groups of test score data calculated by the score calculation module, compares the significance of the difference in the two groups of test prediction scores, and outputs the significance test results in text format to see whether they pass different significance levels.

[0025] Statistical drawing is responsible for the statistics and graphic drawing of the ensemble forecast verification score results.

[0026] Compared with the prior art, the present invention integrates radar combination emissivity test, precipitation target similarity space test, ensemble forecast En-FSS space test, score calculation, significance test and statistical drawing into the system, which can complete the parameter setting and transmission of the whole process of ensemble forecast test from ensemble forecast score calculation, score difference significance test to score statistical drawing, as well as the operation control of each functional module, thereby enhancing the ensemble forecast test and evaluation capability of the tool and providing better support and service for the research and development and improvement of ensemble forecast technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of the overall process of the system of the present invention.

[0028] Figure 2This is a schematic diagram of the evolution of the En-FSS score of the present invention over time.

[0029] Figure 3 Schematic diagram of the comprehensive scoring card of the present invention.

[0030] Figure 4 The mean and time series of CRPS scores in the present invention are shown in Fig. Figure 1 .

[0031] Figure 5 This is a schematic diagram of the mean RMSE / Spread score of the present invention.

[0032] Figure 6 Schematic diagram of the ensemble average and control forecast RMSE (left) / Bias (right) scoring time series of the present invention.

[0033] Figure 7 It is a schematic diagram comparing the ensemble average and control forecast RMSE and dispersion (solid line) of the present invention with the business CMA-GFS score (dashed line).

[0034] Figure 8 It is a schematic diagram of the mean ACC score of the present invention.

[0035] Fig. 9 It is a schematic diagram comparing the ensemble average and control forecast ACC (solid line) of the present invention with the business CMA-GFS score (dashed line).

[0036] Fig.10 This is a schematic diagram of the mean value of the forecast bias Bias score of the present invention.

[0037] Fig.11 The En-FSS score of the present invention is shown as an example of its evolution over time. Figure 2 .

[0038] Fig.12 This is a schematic diagram of the comparison of RMSE and dispersion Spread scores of the regional / global EPS (East Asia region) of the present invention.

[0039] Fig.13 Schematic diagram of the spatial distribution of ensemble forecast verification scores of the present invention.

[0040] Fig.14 Schematic diagram of the spatial distribution of the ensemble forecast Spread / RMSE consistency score of the present invention. DETAILED DESCRIPTION

[0041] The present invention will now be further described with reference to the accompanying drawings.

[0042] See also Figure 1The present invention provides an ensemble forecast verification system, which uses shell script language to set parameters, module flow control, Fortran90 language to perform score calculation and statistics, and finally draws a map through GrADS script language; the system includes the following functions: radar combination reflectivity test, precipitation target similarity space test, ensemble forecast En-FSS space test, score calculation, significance test and statistical drawing.

[0043] The process control module can be divided into the overall process control run, and two subsystem process controls: global EPS inspection control GEPS_vrfy / run and regional EPS inspection control REPS_vrfy / run. These three process control modules all contain two files: job card file *eps_vrfy.cmd (eps_vrfy.cmd, geps_vrfy.cmd, reps_vrfy.cmd) and process control script *eps_vrfy.sh (eps_vrfy.sh, geps_vrfy.sh, reps_vrfy.sh).

[0044] The job card file *eps_vrfy.cmd is used to submit the process control script to the machine job queue to wait for execution. The job name is *epsvrfy, errors are output to the file *eps_vrfy.err, and the operation records are output to *eps_vrfy.log. The content of the job card is as follows:

[0045] The process control script *eps_vrfy.sh is the operation control script for each module, which is used to set parameters such as time and path and control the operation of each module. The meanings of the parameters of the process control script *eps_vrfy.sh are as follows:

[0046] eps_tpye = 'reps' Check EPS type, regional EPS = 'reps', global EPS = 'geps'.

[0047] exp1 = 'CMA-REPS' forecast experiment 1 name;

[0048] exp2 = 'F' prediction experiment 2 name;

[0049] date_start=2022091112 The first start time of the forecast test (UTC);

[0050] date_end=2022091512 The last start time of the forecast test (UTC);

[0051] cyc_int=24 The time interval between adjacent test time reports (Hour); sgnf1=0.75 minimum significance level;

[0052] sgnf2=0.95 intermediate significance level;

[0053] sgnf3=0.995, the highest significance level;

[0054] drt_fcst_exp1 = ' / g2 / nwp_vfy / MUVOS / forecast / EPS / REPS / ' forecast experiment 1 verification data (GRIB2) path;

[0055] drt_fcst_exp2 = 'F' forecast experiment 2 verification data (GRIB2) path;

[0056] drt_grib2_fcst1 = ' / g1 / COMMONDATA / OPER / NWPC / GRAPES_REPS / Prod-grib / ' test 1 (GRIB2) path;

[0057] drt_grib2_fcst2 = 'F'

[0058] Trial 2 (GRIB2) pathway;

[0059] drt_sc_exp1 = '.. / rst / CMA-REPS / ' Forecast experiment 1 test score path;

[0060] drt_sc_exp2 = 'F' Forecast experiment 2 test score path;

[0061] drt_sgnf = '.. / rst_sgnf / CMA-REPS / ' significance test result path;

[0062] drt_scd_pic = '.. / pics / CMA-REPS / ' Check graphics path;

[0063] …

[0064] #---Parameter setting continues---

[0065] drt_obs_rain1h = ' / g2 / nwp_vfy / MUVOS / obs / station / sl / rain / rain_01 / ' hourly observed precipitation data path;

[0066] drt_clim = ' / g3 / vfy_xp / climatology / era-dailyclim / ' WMO-LCDNV standard 1.5° climate field parameter data path;

[0067] fn_stn = '.. / para / china_surf_meteo_obs_stn_info_2016.txt' Precipitation test station list;

[0068] drt_temp = ' / g1 / COMMONDATA / OPER / NWPC / GRAPES_GFS_GMF / Obs-prep / ' sounding observation data path;

[0069] drt_obs_radar = ' / g1 / COMMONDATA / obs / OPER / NWPC / mosaic / ' radar composite reflectivity observation data path;

[0070] …

[0071] sh radar_vrfy.sh$date_start$date_end$cyc_int$drt_grib2_fcst1$drt_grib2_fcst2$drt_sc_exp1$drt_sc_exp2$drt_scd_pic$drt_obs_radar

[0072] Radar combination reflectivity test;

[0073] sh*eps_sim.sh$date_start$date_end$cyc_int

[0074] Precipitation target similarity space test;

[0075] …

[0076] . / *eps_vrfy.exe$dt$drt_fcst_exp1${drt_sc_exp1}$drt_obs_rain1h$fn_stn$fn_zs$drt_clim

[0077] Calculate the ensemble forecast verification score based on the start time;

[0078] …

[0079] . / eps_fss.exe$dt${drt_ana_rain_01}$drt_fcst_exp1${drt_sc_exp1}

[0080] FSS spatial verification of precipitation ensemble forecast;

[0081] …

[0082] . / sgnf.exe$date_start$date_end$cyc_int$exp1$exp2$drt_sc_exp1s$drt_sc_exp2s$drt_sgnfs$sgnf1$sgnf2$sgnf3

[0083] Conduct significance test on the two groups of test scores;

[0084] . / sta.exe$date_start$date_end$cyc_int$exp1$exp2$drt_sc_exp1$drt_sc_exp2$drt_sgnf

[0085] Calculate APTS, PME, PMAE and other probability forecast verification scores for precipitation ensemble forecasts;

[0086] …

[0087] . / scd.exe$date_start$date_end$exp1$exp2$drt_sgnfs$drt_scd_pics$sgnf1$sgnf2$sgnf3

[0088] Use the significance test results to draw a comprehensive score card chart;

[0089] . / scd_2s.exe$date_start$date_end$exp1$exp2$drt_sgnf$drt_scd_pic$sgnf1$sgnf2$sgnf3

[0090] Use the significance test results to draw a comprehensive score card chart of the two scores;

[0091] . / sc_avg.exe$date_start$date_end$cyc_int$exp1$exp2$drt_sc_exp1s$drt_sc_exp2s$drt_scd_pics$sgnf1$sgnf2$sgnf3

[0092] Draw daily, time-averaged and time series graphs of ensemble forecast scores;

[0093] . / rms_cp.exe$date_start$date_end$cyc_int$exp1$exp2$drt_sc_exp1s$drt_sc_exp2s$drt_scd_pics$ar

[0094] Draw a comparison chart of RMSE scores of isobaric surface elements with those of the business CMA-GFS;

[0095] . / rms.exe$date_start$date_end$cyc_int$exp1$exp2$drt_sc_exp1s$drt_sc_exp2s$drt_scd_pics

[0096] Draw a comparison chart of RMSE and Spread;

[0097] . / sr_rms.exe$date_start$date_end$cyc_int$exp1$exp2$drt_sc_exp1s$drt_sc_exp2s$drt_scd_pics$ar

[0098] Draw a time series comparison chart of RMSE and Spread;

[0099] . / rmse_geps.exe$date_start$date_end$cyc_int$exp1 GEPS$drt_sc_exp1$drt_geps$drt_scd_pic EstA

[0100] Draw a comparison chart of the Spread and RMSE scores of the isobaric surface elements of CMA-REPS and CMA-GEPS;

[0101] . / rmsm.exe$date_start$date_end$cyc_int$exp1$exp2$drt_sc_exp1$drt_sc_exp2$drt_scd_pic

[0102] Draw the RMSE / Spread score graph of the isobaric surface elements with the zero field of the set member as the test comparison field;

[0103] . / fss.exe$date_start$date_end$cyc_int$exp1$exp2$drt_sc_exp1$drt_sc_exp2$drt_scd_pic EstA

[0104] Draw a FSS score graph;

[0105] . / acc_cp.exe$date_start$date_end$cyc_int$exp1$exp2$drt_sc_exp1s$drt_sc_exp2s$drt_scd_pics$ar

[0106] Draw a comparison chart of ACC scores of isobaric surface elements with the business CMA-GFS;

[0107] . / acc.exe$date_start$date_end$cyc_int$exp1$exp2$drt_sc_exp1s$drt_sc_exp2s$drt_scd_pics

[0108] Draw anomaly correlation coefficient graph;

[0109] . / bias.exe$date_start$date_end$cyc_int$exp1$exp2$drt_sc_exp1s$drt_sc_exp2s$drt_scd_pics

[0110] Plot forecast deviation maps;

[0111] . / sr_bias.exe$date_start$date_end$cyc_int$exp1$exp2$drt_sc_exp1s$drt_sc_exp2s$drt_scd_pics$ar

[0112] Draw a time series diagram of forecast deviation;

[0113] . / distrb.exe$date_start$date_end$cyc_int$exp1$drt_sc_exp1$drt_scd_pic

[0114] Plot the spatial distribution of inspection scores;

[0115] . / distrb_css.exe$date_start$date_end$cyc_int$exp1$drt_sc_exp1$drt_scd_pic

[0116] Draw the spatial distribution map of Spread / RMSE consistency score;

[0117] Functional modules:

[0118] 1. Radar combination reflectivity test:

[0119] The radar combination reflectivity verification module radar_vrfy_*eps uses the radar combination reflectivity observation data used in the CMA-MESO business mode data assimilation as the verification and comparison data to calculate the probability forecast score of the radar combination reflectivity ensemble forecast, including the AROC score and the BS score.

[0120] The main functions of each program are briefly described as follows:

[0121] radar_vrfy.f90

[0122] The main program for inspection and scoring is mainly responsible for passing parameters, calling data reading, scoring calculation, scoring output subroutines, cycle time, precipitation level, etc.;

[0123] cnst.f90

[0124] Constant parameter setting module;

[0125] get_ntm.f90

[0126] A 10-bit string-formatted time variable is added or subtracted with an integer time increment (can be positive or negative) to obtain a new time string.

[0127] read_cma_*eps.f90

[0128] Read the subroutine of the global / regional ensemble member radar combination reflectivity forecast data in GRIB2 format, paying attention to the north-south order of the data;

[0129] read_radar.f90

[0130] Read the radar combined reflectivity observation data subroutine in NC format, pay attention to the data default value setting; distrb.f90

[0131] Intermediate data output programs such as forecasts and observations help diagnose the correctness of data reading;

[0132] aroc_bs.f90

[0133] The calculation procedure of AROC score and BS score of radar combined reflectivity probability forecast;

[0134] output.f90

[0135] Scoring output program;

[0136] 2. Precipitation target similarity space test:

[0137] The target-oriented spatial verification comprehensive evaluation index method for precipitation probability forecast is:

[0138] a) Analyze the impact of various parameters of the comprehensive evaluation index of precipitation probability forecast on the evaluation of precipitation probability forecast effect.

[0139] Using ensemble forecast precipitation forecast data and high-resolution (5km) precipitation data from the three-source fusion grid analysis of the National Meteorological Information Center, different influence radii, precipitation intensity thresholds and probability thresholds were selected to identify precipitation probability forecast targets and observation targets. The morphological characteristic parameters (center position, area, major axis direction, aspect ratio, etc.) of each precipitation target individual and the morphological characteristic similarity parameters (center distance, minimum edge distance, overlapping area ratio, area ratio, axial difference, etc.) of the precipitation probability forecast target and the observation target pair were calculated. The effects of different influence radii, precipitation intensity thresholds, probability thresholds and morphological characteristic parameters of precipitation targets on the evaluation of precipitation probability forecast effects were analyzed.

[0140] b) Construct a comprehensive evaluation index for spatial verification of precipitation probability forecast.

[0141] According to the analysis of the influence of various parameters of the comprehensive evaluation index of precipitation probability forecast on the evaluation of precipitation probability forecast effect, the main spatial morphological characteristic parameters of precipitation targets are selected, and the weights of each main parameter are determined through the weight coefficient sensitivity test to reasonably express the precipitation probability forecast effect. According to the influence radius, precipitation intensity threshold, probability threshold and various morphological characteristic parameters of precipitation targets determined by the test, a target-oriented precipitation probability forecast spatial verification comprehensive evaluation index is constructed.

[0142] The technical route for the research on the target-oriented spatial verification comprehensive evaluation index of precipitation probability forecast is:

[0143] First, the influence radius R and precipitation intensity threshold L are selected to smooth the grid analysis precipitation field and identify targets. It is assumed that X observation targets are identified. Then the probability threshold P is selected to smooth the probability forecast precipitation field and identify targets. It is assumed that Y forecast targets are identified. According to the matching criteria, assuming that M observation and forecast target pairs are successfully matched, then M≤Min(X,Y). Therefore, the target-oriented convective scale ensemble forecast precipitation probability forecast spatial verification comprehensive evaluation index SIM can be expressed as:

[0144]

[0145] Where L is the number of precipitation intensity thresholds, ρ l is the weight of the precipitation intensity threshold l; P is the number of probability thresholds, τ p is the weight of the probability threshold p, S p is the similarity of morphological features when the probability threshold is p:

[0146]

[0147] Where M is the number of observation and forecast target pairs, Am is the area of ​​the mth observation and forecast target pair, At is the total area of ​​all observation and forecast target pairs, K is the number of morphological features, ε kis the weight of the k-th morphological feature, and the similarity function Fk of the k-th morphological feature parameter is determined by sensitivity test, and its value range is [0,1]; where Fk(m) is the contribution function of the k-th morphological feature parameter (center distance, minimum edge distance, overlapping area ratio, area ratio, axial difference, etc.) of the m-th observation and forecast target pair to the morphological feature similarity.

[0148] The range of SIM, a comprehensive evaluation index of spatial verification of precipitation probability forecast, is [0, 1], where 0 represents no skill and 1 represents an ideal score. When the number of targets identified by both the observation field and the forecast field is 0, SIM is undefined; when one of the number of targets identified by the observation field and the forecast field is 0, SIM = 0.

[0149] The precipitation target similarity spatial verification module sim_*eps uses the 5km resolution three-source fusion grid analysis precipitation data of the National Meteorological Information Center as the verification and comparison data, extracts the precipitation target morphological characteristic parameters (center position, area, major axis direction, aspect ratio, etc.) on the grid analysis precipitation field and the precipitation probability forecast field, as well as the morphological characteristic similarity parameters (center distance, minimum edge distance, overlapping area ratio, area ratio, axial difference, etc.) between the precipitation probability forecast target and the observed target pair, and calculates the precipitation probability forecast comprehensive evaluation index at each magnitude, as well as the overall precipitation probability forecast comprehensive evaluation index, including the AROC score of precipitation at each magnitude on the grid field, as a traditional score for comparative reference.

[0150] The main functions of each program are briefly described as follows:

[0151] *eps_sim.f90

[0152] The main program for inspection and scoring is mainly responsible for passing parameters, calling data reading, scoring calculation, scoring output subroutines, cycle time, precipitation level, etc.;

[0153] .. / run / settings.nml

[0154] Observation, forecast, result storage path and parameter setting files such as forecast range, resolution and number of grid points;

[0155] cnst.f90

[0156] Constant parameter setting module;

[0157] get_ntm.f90

[0158] A 10-bit string-formatted time variable is added or subtracted with an integer time increment (can be positive or negative) to obtain a new time string.

[0159] attribute_pair.f90

[0160] Obtain the morphological characteristic parameters of the forecast & observation target pair (center distance, minimum edge distance, overlapping area ratio, area ratio, axial difference, etc.);

[0161] attribute_single.f90

[0162] Obtain forecasts and observe the morphological characteristic parameters of a single target (center position, area, major axis direction, aspect ratio, etc.);

[0163] border.f90

[0164] Obtain numbered forecast and observation site precipitation targets;

[0165] get_ana_rain_1h.f90

[0166] Read the hourly three-source fusion grid analysis precipitation data in GRIB1 format. Pay attention to the north-south order of the data; get_ana_rain_xh.f90

[0167] Subroutine to obtain X hours of accumulated three-source fusion grid analysis precipitation data;

[0168] get_*eps_fcst.f90

[0169] Obtain precipitation probability forecasts at each precipitation level;

[0170] read_fcst.f90

[0171] Subroutine to read GRIB2 format ensemble member precipitation forecast data, pay attention to the north-south order of the data; similarity.f90

[0172] The precipitation probability forecast comprehensive evaluation index sim is calculated based on the morphological characteristic parameters of the forecast & observation target pair (center distance, minimum edge distance, overlapping area ratio, area ratio, axial difference, etc.);

[0173] smooth.f90

[0174] Smoothing the original forecast and observed precipitation fields;

[0175] transform.f90

[0176] The forecast and observed precipitation fields are interpolated to a 0.01° high-resolution uniform grid field using the nearest point method;

[0177] vrf_aroc.f90

[0178] Calculate the AROC score of precipitation probability forecast on the grid field;

[0179] 3. Such as Figure 2As shown, the En-FSS spatial verification of the ensemble forecast is:

[0180] a) En-FSS score calculation scheme for spatial verification of precipitation ensemble forecast:

[0181] The En-FSS scoring calculation scheme for the spatial verification of precipitation ensemble forecast mainly refers to the En-FSS scoring method proposed by Le Duc et al. (TellusA2013). However, in this verification software, only the spatial extension is considered, and the temporal extension is not considered:

[0182] Cumulative duration of precipitation inspection: 12 hours;

[0183] Test precipitation levels: light rain (0.1mm), moderate rain (5mm), heavy rain (15mm) and rainstorm (30mm);

[0184] Neighborhood window scale: 0.1°, 0.3°, 0.5°, 1°, and 2°;

[0185] Forecast resolution: 0.03°, 0.1° and 0.15°;

[0186] b) Results of En-FSS scoring test for precipitation ensemble forecast:

[0187] From the En-FSS scores of the 0.03° and 0.15° regional ensemble forecast REPS_3km and REPS_15km experiments, the En-FSS scores of the spatial verification of precipitation ensemble forecast can better distinguish the effects of precipitation ensemble forecasts at different resolutions. For different neighborhood window scales, the En-FSS scores of the spatial verification of precipitation ensemble forecasts show that the heavy rain (15mm) and torrential rain (30mm) forecast effects of CMA-REPS_3km are significantly better than those of CMA-REPS_15km.

[0188] The precipitation ensemble forecast spatial verification En-FSS score calculation module fss uses the 5km resolution three-source fusion grid analysis precipitation data of the National Meteorological Information Center as the verification and comparison data, and calculates the precipitation ensemble forecast En-FSS score at various levels and different spatial scales.

[0189] The main functions of each program are briefly described as follows:

[0190] eps_fss.f90

[0191] The main program for inspection and scoring is mainly responsible for passing parameters, calling data reading, scoring calculation, scoring output subroutines, cycle time, etc.;

[0192] cnst.f90

[0193] Constant parameter setting module;

[0194] get_ntm.f90

[0195] A 10-bit string-formatted time variable is added or subtracted with an integer time increment (can be positive or negative) to obtain a new time string.

[0196] cal_fss.f90

[0197] Calculate the En-FSS score for spatial verification of precipitation ensemble forecasts;

[0198] read_ana_rain_01.f90

[0199] Read the hourly three-source fusion grid analysis precipitation data in GRIB1 format. Pay attention to the north-south order of the data; get_ana_rain_xh.f90

[0200] Subroutine to obtain X hours of accumulated three-source fusion grid analysis precipitation data;

[0201] nrpt_ana.f90

[0202] Perform nearest point interpolation on the grid field;

[0203] read_fcst.f90

[0204] Read the GRIB2 format ensemble member precipitation forecast data subroutine, paying attention to the north-south order of the data;

[0205] output.f90

[0206] Scoring output program;

[0207] 4. Score calculation: The program under the score calculation module src is responsible for the calculation and output of the main test score of the ensemble forecast for a single report time. The folder para in this module stores the precipitation test site information list data, model terrain height field data, etc.

[0208] The main functions of each program are briefly described as follows:

[0209] *eps_vrfy.f90

[0210] The main program for inspection and scoring is mainly responsible for passing parameters, calling data reading, scoring calculation, scoring output subroutines, cycle time, level, elements, etc.;

[0211] reps_vrfy_terrain.f90

[0212] The main program for regional ensemble forecast verification, which considers the part below the model terrain by deducting the isobaric surface element field from the model terrain field in the verification score calculation, can be used to replace reps_vrfy.f90;

[0213] reps_vrfy_noterrain.f90

[0214] The regional ensemble forecast verification main program that does not consider the part below the model terrain by deducting the isobaric surface element field from the model terrain field in the verification score calculation is the default main program, which is the same as reps_vrfy.f90;

[0215] cnst.f90

[0216] Constant parameter setting module;

[0217] read_fcst.f90

[0218] Read the ensemble forecast data subroutine in GRIB2 format, paying attention to the north-south order of the data;

[0219] get_ntm.f90

[0220] A 10-bit string-formatted time variable is added or subtracted with an integer time increment (can be positive or negative) to obtain a new time string.

[0221] rd_clim.f90

[0222] The daily climate field data of the 1.5° isobaric surface elements of ECMWF-ERA from 1989 to 2008, which is the WMO-LCDNV verification standard climate field, are read as parameters for calculating the score;

[0223] intp_clim.f90

[0224] Interpolate the forecast data to the climate field grid;

[0225] intp_temp.f90

[0226] Interpolate ensemble forecast data to the sounding observation sites;

[0227] roc_reli.f90

[0228] Calculate the ROC area score of the isobaric surface elements and the horizontal and vertical coordinates of its curve (F and H), reliability curve parameters, etc.;

[0229] temp_vrfy.f90

[0230] Ensemble forecast verification score calculation for station sounding observation data;

[0231] rmse_sprd.f90

[0232] Calculate the ensemble mean and area-weighted RMSE of each ensemble member and the ensemble forecast dispersion on the original forecast field grid;

[0233] rmse_crps_sprd_ri.f90

[0234] Calculate ensemble forecast (taking area weight into account) CRPS, Outlier and other scores;

[0235] distr.f90

[0236] To calculate CRPS, grade histograms and other scores, the predicted values ​​of the ensemble members are sorted; distrb.f90

[0237] Output intermediate inspection data field in Micaps4 format;

[0238] rmse_acc.f90

[0239] Calculate the ensemble mean and area-weighted RMSE / ACC scores of each ensemble member on the climate grid field;

[0240] read_stif.f90

[0241] Read the site latitude and longitude and other information in the inspection site list;

[0242] read_zs.f90

[0243] Read terrain height field data in little-endian binary format stored sequentially;

[0244] read_temp.f90

[0245] Read the text format sounding observation data used for business model data assimilation;

[0246] get_obs_rain_xh.f90

[0247] Get the station observed precipitation value with a cumulative duration of X hours;

[0248] rd_obs_rain_1h.f90

[0249] Subroutine to read hourly station observation precipitation data in text format, paying attention to data default and trace precipitation value processing;

[0250] nrpt.f90

[0251] Interpolate the precipitation forecast value to the observation site using the nearest point interpolation method;

[0252] obfc.f90

[0253] Calculate and output the 5% quantile forecast values ​​of the ensemble forecast at the observation station;

[0254] op_err.f90

[0255] File open status check handler;

[0256] rd_err.f90

[0257] Display read error status prompt information;

[0258] aroc_bs_ri.f90

[0259] Calculate the AROC and BS scores of precipitation ensemble forecasts at the observing stations;

[0260] output.f90

[0261] Output the scoring data into text format in the form of a comprehensive scorecard;

[0262] output_roc_reli.f90

[0263] Output the scoring parameters such as ROC curve, reliability curve, relative economic value, etc. of the ensemble forecast isobaric surface elements under the climate field mean threshold;

[0264] output_roc_relin.f90

[0265] Output the ROC curve, reliability curve, BS and other scoring parameters of the ensemble forecast isobaric surface elements under each standard deviation threshold of the climate field;

[0266] 5. Significance test:

[0267] The significance test module performs a significance test on the score difference of the two groups of test score data calculated by the score calculation module, compares the significance of the difference in the two groups of test prediction scores, and outputs the significance test results in text format to see whether they pass different significance levels.

[0268] The main functions of each program are briefly described as follows:

[0269] get_ntm.f90

[0270] A 10-bit string-formatted time variable is added or subtracted with an integer time increment (can be positive or negative) to obtain a new time string.

[0271] inv_stdnt.f90

[0272] Get the quantile of Student's T distribution;

[0273] PROBST.f90

[0274] Calculate the Student's T distribution probability value;

[0275] STUDNT.f90

[0276] Calculate the area under the upper tail of the Student's T distribution curve;

[0277] sta.f90

[0278] Statistical calculation of the newly developed EPS precipitation probability forecast score APTS, probability error PME / PMAE, etc.;

[0279] sgnf.f90

[0280] The main program for calculating the significance test is to statistically compare the significance of the difference in the prediction scores of the two groups of tests and whether they have passed different significance levels;

[0281] The meanings of the parameters of the significance test scoring main program sgnf.f90 are as follows:

[0282] ldn=30, the total number of columns in the scoring matrix on the comprehensive scoring card;

[0283] vrn=18, the total number of rows in the scoring matrix on the comprehensive scoring card;

[0284] expn=2, indicating two groups of comparative experiments;

[0285] hdn=6, you need to skip the first 6 lines of the text format comprehensive score card;

[0286] tn=3, there are 3 significance level parameters;

[0287] nbth=3, indicating the minimum number of scores required for significance test, otherwise no test will be performed;

[0288] rms=1,rme=15,which means the first score on the comprehensive score card is columns 1-15;

[0289] acs=16,ace=30, which means the second score on the comprehensive score card is in columns 16-30;

[0290] 6. Such as Figures 3 to 14 The statistical plot shown is:

[0291] The statistical drawing module scd is responsible for the statistics and graphic drawing of the ensemble forecast test score results. The main functions of each drawing program are briefly introduced as follows:

[0292] scd.f90

[0293] Comprehensive scorecard drawing program, used to read the significance test results in text format output by the significance test module (-3, -2, -1, 0, 1, 2, 3 are codes for passing or failing at different significance levels) and draw the significance test results;

[0294] scd.gs

[0295] The GS script for drawing the comprehensive score card of the significance test maps the codes one by one to the graphics of the corresponding colors on the comprehensive score card and outputs the graph according to the significance test results (significance codes) in text format;

[0296] sc_avg.f90

[0297] The scoring time mean and scoring time series drawing program reads the scores one by one, calculates the time mean of the scores, and calls the script to draw the scoring time mean and series diagram;

[0298] sc_avg.gs

[0299] Rating time mean and time series plotting script, plotting the rating curve with time or time change; rmse.f90

[0300] The ensemble average and control forecast RMSE and dispersion comparison drawing program reads the scores of each starting time, calculates the time mean of the score, and calls the script to draw the time mean graph of the score;

[0301] rmse.gs

[0302] Rating time mean graph drawing script, draw RMSE / Spread score curve with time aging; sr_rms.f90&sr_bias.f90

[0303] The program for drawing the time series graph of the ensemble average and control forecast RMSE / Bias scores reads the scores of each starting time and calls the script to draw the time series graph of the scores;

[0304] sr_rms.gs&sr_bias.gs

[0305] Rating time series plotting script, plotting RMSE / Bias rating time series curves;

[0306] rms_cp.f90

[0307] The ensemble mean and control forecast RMSE and dispersion are compared with the business CMA-GFS score. The program reads the scores of each starting time, calculates the time mean of the score, and calls the script to draw the time mean graph of the score; rms_cp.gs

[0308] The script for drawing the mean value graph over time is used to draw the RMSE / Spread and business CMA-GFS score comparison curves that change with time.

[0309] acc.f90

[0310] The ensemble average and control forecast ACC drawing program reads the scores of each starting time, calculates the score time mean, and draws the ACC score time mean graph;

[0311] acc.gsACC score time mean graph drawing script, draw the score curve that changes with time;

[0312] acc_cp.f90

[0313] The program for comparing the ensemble average and control forecast ACC and operational CMA-GFS scores is used to read the scores of each starting time, calculate the time mean of the scores, and draw the time mean graph of the ACC scores;

[0314] acc_cp.gs

[0315] The script for drawing the time mean graph of the ensemble average and control forecast ACC score is used to draw the score curve of the ensemble average and control forecast ACC and the operational CMA-GFS score changing with time;

[0316] bias.f90

[0317] The ensemble average and control forecast Bias plotting program reads the scores of each starting time, calculates the mean Bias score, and plots the score time mean graph;

[0318] bias.gs

[0319] Forecast bias Bias score time mean graph drawing script, draw the Bias score curve with time aging; fss.f90

[0320] Ensemble forecast spatial verification En-FSS score statistical drawing program, read the score of each start time, calculate the mean En-FSS score, and draw the score map;

[0321] fss.gs

[0322] Ensemble forecast spatial verification En-FSS score drawing script, drawing En-FSS score curves with time duration and precipitation magnitude;

[0323] mse_geps.f90

[0324] The program for drawing the comparison chart of the regional ensemble forecast control forecast, ensemble mean RMSE and dispersion Spread with the corresponding scores of the global ensemble forecast in the East Asia region reads the test scores of the regional EPS and the global EPS in the East Asia region corresponding to the start time and draws the chart;

[0325] rmse_geps.gs

[0326] Script for drawing comparison charts of regional / global EPS (East Asia) RMSE and dispersion Spread scores;

[0327] distrb.f90

[0328] The program for drawing the spatial distribution map of the ensemble forecast verification score reads the grid data of the ensemble forecast score at each starting time, calculates the time mean, and draws the spatial distribution map of the ensemble forecast verification score;

[0329] distrb.gs

[0330] Script for drawing the spatial distribution map of ensemble forecast verification scores; distrb_css.f90

[0331] The program for drawing the spatial distribution map of the ensemble forecast Spread / RMSE consistency score reads the score grid data of each starting time, calculates the time mean, and draws the spatial distribution map of the ensemble forecast consistency score;

[0332] distrb_css.gs

[0333] Script for drawing spatial distribution map of ensemble forecast Spread / RMSE consistency score, which draws spatial distribution map of ensemble forecast Spread / RMSE consistency score;

[0334] The above are only preferred embodiments of the present invention, which are only used to help understand the method and core ideas of the present application. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

[0335] The present invention solves the problem in the prior art as a whole that there is a lack of a system for efficient and convenient inspection and evaluation of a large number of ensemble forecast test results. By setting and transmitting the parameters of the entire process of ensemble forecast inspection from ensemble forecast score calculation, score difference significance test, to score statistical drawing, as well as the operation control of each functional module, the ensemble forecast inspection and evaluation capability of the tool is enhanced, providing better support and service for the research and development and improvement of ensemble forecast technology.

Claims

1. An ensemble forecast verification system, characterized in that: The parameters are set and the module flow is controlled by shell script language, the score calculation and statistics are performed by Fortran90 language, and finally the map is drawn by GrADS script language; the functions of the system include: radar combination reflectivity test, precipitation target similarity space test, ensemble forecast En-FSS space test, score calculation, significance test and statistical drawing; The radar combination reflectivity test uses the radar combination reflectivity observation data used in the CMA-MESO business model data assimilation as the test comparison data, and calculates the probability forecast score of the radar combination reflectivity ensemble forecast, including the AROC score and the BS score; The precipitation target similarity space test includes: Analyze the influence of various parameters of comprehensive evaluation index of precipitation probability forecast on the evaluation of precipitation probability forecast effect: Using ensemble forecast precipitation forecast data and three-source fusion grid analysis high-resolution precipitation data from the National Meteorological Information Center, different influence radii, precipitation intensity thresholds and probability thresholds are selected to identify precipitation probability forecast targets and observation targets, and calculate the morphological characteristic parameters of each precipitation target individual, including center position, area, major axis direction, aspect ratio, and morphological characteristic similarity parameters of precipitation probability forecast targets and observation targets, including center distance, minimum edge distance, overlapping area ratio, area ratio, and axial difference. The impact of different influence radii, precipitation intensity thresholds, probability thresholds and morphological characteristic parameters of precipitation targets on the evaluation of precipitation probability forecast effects is analyzed; Construct a comprehensive evaluation index for spatial verification of precipitation probability forecast: According to the analysis of the influence of various parameters of the comprehensive evaluation index of precipitation probability forecast on the evaluation of precipitation probability forecast effect, the main spatial morphological characteristic parameters of precipitation targets are selected, and the weights of various main parameters are determined through weight coefficient sensitivity tests to reasonably express the precipitation probability forecast effect; according to the influence radius, precipitation intensity threshold, probability threshold and various morphological characteristic parameters of precipitation targets determined by the test, a target-oriented precipitation probability forecast spatial verification comprehensive evaluation index is constructed; The specific steps are: S3-1, first select the influence radius R and precipitation intensity threshold L, smooth the grid analysis precipitation field and identify the target, assuming that X observation targets are identified; S3-2, select the probability threshold P, smooth the probability forecast precipitation field and identify the target, assuming that Y forecast targets are identified; S3-3, according to the matching criteria, assuming that M pairs of observations and forecast targets are successfully matched, then M≤Min(X, Y); The target-oriented convective-scale ensemble forecast precipitation probability forecast spatial verification comprehensive evaluation index SIM can be expressed as: Formula 1: L is the number of precipitation intensity thresholds, ρ l is the weight of the precipitation intensity threshold l; P is the number of probability thresholds, τ p is the weight of the probability threshold p, S p is the similarity of morphological features when the probability threshold is p: Formula 2: M is the number of observation and forecast target pairs, A m is the area of ​​the mth observation and forecast target pair, A t is the total area of ​​all observed and predicted target pairs, K is the number of morphological features, ε k is the weight of the kth morphological feature, and the similarity function F of the kth morphological feature parameter k Determined through sensitivity tests, the value range is [0,1]; the F k (m) is the contribution function of the kth morphological characteristic parameters of the mth observation and forecast target pair, namely, center distance, minimum edge distance, overlapping area ratio, area ratio, and axial difference to the morphological characteristic similarity; The range of the SIM value of the spatial verification comprehensive evaluation index of precipitation probability forecast is [0,1], where 0 represents no skill and 1 represents an ideal score. When the number of targets identified by the observation field and the forecast field is 0, SIM is undefined; when the number of targets identified by the observation field and the forecast field, one of which is 0, SIM = 0; The En-FSS spatial verification of the ensemble forecast is: only considering the spatial extension, and verifying the space through the En-FSS scoring method; The score calculation is responsible for calculating and outputting the main verification score of the ensemble forecast for a single reporting time, and storing the precipitation verification site information list information and model terrain height field data; The significance test performs a significance test on the score difference of the two groups of test score data calculated by the score calculation module, compares the significance of the difference in the two groups of test forecast scores, and outputs the significance test results in text format to see whether they pass different significance levels; the statistical drawing is responsible for the statistics and graphic drawing of the set forecast test score results.

Images