Multi-dimensional fusion test evaluation method and system for deterministic heavy precipitation forecast

Abstract

The present application relates to the multi-dimensional fusion test evaluation method and system for the deterministic heavy precipitation forecast, comprising the following: obtaining the model precipitation forecast data and the site live data of the target evaluation area as the to-be-evaluated data; inputting the to-be-evaluated data into the pre-established heavy precipitation multi-dimensional fusion test evaluation system to obtain the evaluation result; and visually displaying the evaluation result; the present application supplements the model precipitation forecast process stability evaluation method and the spatial test similarity test method suitable for the heavy precipitation characteristics on the basis of the traditional test method, and adds more test dimensions for the model precipitation forecast. The present application further constructs the multi-dimensional fusion test evaluation method, the multi-dimensional test index can be displayed through the graph, the model development and the user can intuitively understand the prediction ability of each dimension of the model; and the comprehensive test evaluation index (CEI) formed reflects the comprehensive evaluation result of the six core precipitation prediction abilities of the model.

Description

Technical Field

[0001] This invention relates to the field of atmospheric science, and in particular to a multidimensional fusion verification and evaluation method and system for deterministic heavy precipitation forecasts. Background Technology

[0002] The commonly used precipitation forecast verification methods in the field of atmospheric science can be divided into the following three categories:

[0003] (1) Deterministic prediction test for continuous variables. Specific test methods include: relative error, absolute error, standard deviation, correlation coefficient, etc.

[0004] (2) Deterministic prediction test for binary events. Specific test methods include: accuracy, TS score, BIAS score, hit rate, false alarm rate, false alarm rate, etc.

[0005] (3) Verification of space field predictions. Commonly used verification methods include neighborhood method, MODE, CRA, etc.

[0006] The deterministic forecast test for continuous variables reflects the deviation of the forecast value from the actual value; the deterministic forecast test for binary events reflects the presence or absence of a forecast for the actual event; and the spatial field forecast test reflects the deviation between the actual field and the forecast field in terms of spatial characteristics.

[0007] Let's illustrate the characteristics of the three evaluation methods with an example. Weather station A recorded 47 mm of accumulated precipitation in 24 hours, while weather station B, located 2 km away, recorded 52 mm. The model forecasts 52 mm of precipitation for both stations. To evaluate the model's ability to forecast heavy rainfall (over 50 mm of accumulated precipitation in 24 hours) at station A, the first type of evaluation method shows a forecast error of only 5 mm, which is not significant. However, the second type of evaluation method indicates that the model failed to accurately forecast the heavy rainfall, instead providing an unrealistic forecast for station A. Combining this with the third type of evaluation method, the model accurately forecasts the heavy rainfall area within a 5 km radius of station A, indicating that the model's overall forecast performance for heavy precipitation is still relatively good. This demonstrates that different evaluation methods only express different dimensions of assessment significance; a single evaluation method cannot encompass all the information about the model's forecast performance.

[0008] In addition, the evaluation of the model's precipitation forecasting capability needs to include time-based verification methods, such as the forecast stability of the model's forecasts over multiple consecutive days and the accuracy of hourly precipitation trend forecasts.

[0009] For deterministic precipitation grid forecasts, there is currently a lack of comprehensive multi-dimensional verification methods for evaluating the capabilities of these models. Summary of the Invention

[0010] The purpose of this invention is to at least address one of the shortcomings of the prior art by providing a multidimensional fusion verification and evaluation method and system for deterministic heavy precipitation forecasts.

[0011] To achieve the above objectives, the present invention adopts the following technical solution:

[0012] Specifically, a multi-dimensional fusion verification and evaluation method for deterministic heavy precipitation forecasts is proposed, including the following:

[0013] Obtain model precipitation forecast data and station-based real-time data for the target assessment area as the data to be assessed.

[0014] The data to be evaluated is input into the pre-established multi-dimensional fusion test and evaluation system for heavy precipitation to obtain the evaluation results;

[0015] The evaluation results will be visualized.

[0016] Specifically, the process of establishing a multidimensional fusion testing and evaluation system includes:

[0017] Establish modules for deterministic forecast testing of continuous variables and deterministic forecast testing of binary events.

[0018] A model process forecast stability testing module is established based on a multidimensional testing method.

[0019] A multidimensional spatial similarity test for precipitation is established, and a spatial test module is built based on this.

[0020] Based on the deterministic forecast verification module, the process forecast stability verification module, and the spatial verification module, multidimensional fusion verification and evaluation indicators for heavy precipitation are determined, and a multidimensional fusion verification and evaluation system is established.

[0021] Furthermore, specifically, modules for deterministic forecast testing of continuous variables and deterministic forecast testing of binary events are established, including:

[0022] Establish the relative error, absolute error, and standard deviation of cumulative precipitation forecasts; establish the TS score, BIAS score, hit rate, missed rate, and false alarm rate for cumulative precipitation classification; establish the accuracy and correlation coefficient of hourly precipitation forecasts; and then establish a deterministic forecast verification module.

[0023] Furthermore, specifically, a model process forecast stability testing module is established based on the multidimensional testing method, including:

[0024] Forecast standard deviation, range, and process forecast error are used as indicators to verify the stability of model process forecasts. Based on these indicators, a model process forecast stability verification module is established.

[0025] The forecast standard deviation is calculated as follows:

[0026] ,

[0027] In the formula, For the i-th reported precipitation value in the process, This is the average of n reported precipitation values.

[0028] The range is calculated as follows:

[0029] ,

[0030] In the formula, and These are the maximum and minimum precipitation values ​​from n reported precipitation data points, respectively.

[0031] The method for calculating the process forecast error is as follows:

[0032] ,

[0033] In the formula, This is the average of n reported precipitation values. This represents the actual precipitation value.

[0034] Furthermore, the model process forecast stability verification module is also used to calculate the comprehensive stability index of the process.

[0035] The comprehensive process stability index is obtained by adding the absolute values ​​of the forecast standard deviation, range, and process forecast error according to preset weights. The smaller the value of the comprehensive process stability index, the better the forecast stability and process forecast effect of the model.

[0036] Furthermore, specifically, a multi-dimensional test for spatial similarity of precipitation is established, including:

[0037] Target object identification: The grid data in the precipitation field is processed into a two-dimensional array of size xDim*yDim, and Gaussian threshold filtering is used to identify and merge precipitation objects to obtain forecast and actual objects;

[0038] After obtaining the forecast and actual target objects, the system searches for actual objects within a square area centered on the geometric center of the forecast object. Specifically, it obtains the bounding rectangle of the forecast object and then extends a preset value of grid width outward from each side of the bounding rectangle to obtain a search rectangle. If an actual object falls within the search rectangle, it is added to the current forecast object's score set. If no actual object matches, it is considered a null forecast. If multiple actual objects match, they are scored, and the object with the highest score is selected to generate an object pair.

[0039] The scoring is based on the spatial similarity of precipitation using a multidimensional test. Specifically, the spatial similarity of precipitation using a multidimensional test is as follows:

[0040] ,

[0041] Through comparative evaluation experiments, the value of M was set to 4.

[0042] Let be the ratio of the overlapping areas of the j-th object pair. =[Number of intersecting grid points / (Actual grid points + Predicted grid points)] * 2

[0043] The area ratio of the j-th object pair:

[0044] When 0 <= R <= 0.8, =R / 0.8,

[0045] When R > 0.8, =1,

[0046] Where R is a value less than 1 in which the area of ​​the forecast object (i.e., the total number of grid points) is divided by the area of ​​the actual object (i.e., the total number of grid points), or the area of ​​the actual object (i.e., the total number of grid points) is divided by the area of ​​the forecast object (i.e., the total number of grid points).

[0047] This is the absolute value of the major axis angle difference between the j-th object pair, i.e., the absolute value of the cosine of the angle between the longest axis of the predicted object's geometry and the longest axis of the actual object's geometry.

[0048] =|COS(angleDelta)|,

[0049] Let be the geometric center distance of the j-th object pair.

[0050] Specific parameters: distance D is the distance between the geometric center of the actual object and the geometric center of the predicted object; optimal distance Dmin; maximum tolerance distance Dmax.

[0051] When D <= Dmin: =1,

[0052] When Dmin <D<= Dmax : =1-(D- Dmin) / [Dmax - Dmin],

[0053] When D > Dmax: =0,

[0054] Here, we set Dmin = 30KM and Dmax = 300KM.

[0055] Function weights w: 0.4, 0.3, 0.1, 0.2

[0056] Confidence level c: =1、 =1、 = , = The ratio of small areas to large areas in the actual and forecast objects, where and The aspect ratios of the forecast and the observed objects are respectively, when or When the confidence function approaches 1, The value is close to 0.

[0057] Furthermore, specifically, determine the multi-dimensional integrated evaluation indicators for heavy precipitation, including:

[0058] Based on the experiment of testing and evaluating multiple heavy precipitation cases, the 24-hour mean absolute error of model precipitation forecasts, TS score for heavy rain and above, heavy rain forecast bias, hourly clear and rain forecast accuracy, heavy rain spatial similarity, and comprehensive index of process stability were selected from the output results of the deterministic forecast testing module, process forecast stability testing module, and spatial testing module and dimensionlessly fused to construct a multidimensional fusion testing and evaluation index.

[0059] Furthermore, the method also includes performing equal-weighted arithmetic averaging on the dimensionless fusion of the 24-hour average absolute error, TS score for heavy rain and above, rainstorm forecast bias, hourly weather forecast accuracy, rainstorm spatial similarity, and process stability comprehensive index to obtain the comprehensive test and evaluation index (CEI). The higher the CEI score, the better the model's multi-dimensional fusion forecast capability for heavy precipitation.

[0060] Furthermore, specifically, the method for visualizing the presentation is as follows:

[0061] The evaluation results are generated into a radar chart, and the radar chart and CEI value are visualized.

[0062] This invention also proposes a multi-dimensional fusion verification and evaluation system for deterministic heavy precipitation forecasts, including the following:

[0063] The data acquisition module is used to acquire model precipitation forecast data and station real-time data for the target assessment area as the data to be assessed.

[0064] The deterministic forecast testing module is used to perform deterministic forecast testing on continuous variables and deterministic forecast testing on binary events for the data to be evaluated.

[0065] The Model Process Forecast Stability Validation Module is used to perform model process forecast stability validation on the data to be evaluated using a multidimensional test-based process forecast stability validation method.

[0066] A spatial verification module is established to perform precipitation spatial similarity verification on the precipitation spatial similarity of the data to be evaluated based on multidimensional verification.

[0067] The module for determining the multidimensional fusion test and evaluation index of heavy precipitation is used to determine the multidimensional fusion test and evaluation index of heavy precipitation based on the deterministic forecast test module, the process forecast stability test module and the spatial test module, and then obtain the evaluation results.

[0068] The visualization module is used to visualize the evaluation results.

[0069] The beneficial effects of this invention are as follows:

[0070] This invention proposes a multi-dimensional fusion verification and evaluation method and system for deterministic heavy precipitation forecasts, overcoming the limitation of existing single precipitation verification methods mentioned in the background, which can only evaluate the forecasting capability of a single dimension of the model. Based on traditional verification methods, this invention supplements them with a method for evaluating the stability of the model's precipitation forecast process and a spatial similarity verification method applicable to heavy precipitation characteristics, adding more verification dimensions to model precipitation forecasts. Furthermore, this invention constructs a multi-dimensional fusion verification and evaluation method. Multi-dimensional verification indicators can be graphically displayed, allowing model developers and users to intuitively understand the forecasting capability of each dimension of the model; the resulting Comprehensive Validation and Evaluation Index (CEI) reflects the comprehensive evaluation results of the model's six core precipitation forecasting capabilities. Attached Figure Description

[0071] The above and other features of this disclosure will become more apparent from the detailed description of the embodiments illustrated in conjunction with the accompanying drawings. In the accompanying drawings, the same reference numerals denote the same or similar elements. Obviously, the drawings described below are merely some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort. In the drawings:

[0072] Figure 1 The flowchart shown is a multi-dimensional fusion verification and evaluation method for deterministic heavy precipitation forecasts according to the present invention.

[0073] Figure 2 The diagram shows the algorithm principle of the multi-dimensional fusion verification and evaluation method for deterministic heavy precipitation forecasts in this invention.

[0074] Figure 3 The figure shown is an evaluation result diagram of the present invention in a specific application. Detailed Implementation

[0075] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The same reference numerals used throughout the accompanying drawings indicate the same or similar parts.

[0076] Example 1, referring to Figure 1 as well as Figure 2 This invention proposes a multi-dimensional fusion verification and evaluation method for deterministic heavy precipitation forecasts, including the following:

[0077] Obtain model precipitation forecast data and station-based real-time data for the target assessment area as the data to be assessed.

[0078] The data to be evaluated is input into the pre-established multi-dimensional fusion test and evaluation system for heavy precipitation to obtain the evaluation results;

[0079] The evaluation results will be visualized.

[0080] Specifically, the process of establishing a multidimensional fusion testing and evaluation system includes:

[0081] Establish modules for deterministic forecast testing of continuous variables and deterministic forecast testing of binary events.

[0082] A model process forecast stability testing module is established based on a multidimensional testing method.

[0083] A multidimensional spatial similarity test for precipitation is established, and a spatial test module is built based on this.

[0084] Based on the deterministic forecast verification module, the process forecast stability verification module, and the spatial verification module, multidimensional fusion verification and evaluation indicators for heavy precipitation are determined, and a multidimensional fusion verification and evaluation system is established.

[0085] In this embodiment 1, the one-sidedness of the single-dimensional test method in evaluating the precipitation forecasting ability of the model is overcome, and a multi-dimensional fusion precipitation forecasting evaluation method is provided, which can comprehensively evaluate the heavy precipitation forecasting ability of deterministic models and intuitively compare the forecasting capabilities of each model.

[0086] In a preferred embodiment of the present invention, specifically, a deterministic prediction test module for continuous variables and a deterministic prediction test module for binary events are established, including:

[0087] Establish the relative error, absolute error, and standard deviation of cumulative precipitation forecasts; establish the TS score, BIAS score, hit rate, missed rate, and false alarm rate of cumulative precipitation classification; establish the accuracy and correlation coefficient of hourly precipitation forecasts; and then establish a deterministic forecast verification module.

[0088] The correlation coefficient refers to the statistical value... .in, These are station observations. Here, N represents the forecast value for each station, and N is the total number of samples (number of stations) participating in the test. Furthermore, most of the indicators in the deterministic forecast testing module are not used later to synthesize multidimensional fusion test evaluation indicators; they are only used to establish the deterministic forecast testing module and provide data support for subsequent analysis.

[0089] In a preferred embodiment of the present invention, specifically, a model process forecast stability testing module is established based on a multidimensional testing method, comprising:

[0090] Forecast standard deviation, range, and process forecast error are used as indicators to verify the stability of model process forecasts. Based on these indicators, a model process forecast stability verification module is established.

[0091] The forecast standard deviation is calculated as follows:

[0092] ,

[0093] In the formula, For the i-th reported precipitation value in the process, This is the average of n reported precipitation values.

[0094] The range is calculated as follows:

[0095] ,

[0096] In the formula, and These are the maximum and minimum precipitation values ​​from n reported precipitation data points, respectively.

[0097] The method for calculating the process forecast error is as follows:

[0098] ,

[0099] In the formula, This is the average of n reported precipitation values. This represents the actual precipitation value.

[0100] Typically, n ≥ 3.

[0101] In a preferred embodiment of the present invention, the model process forecast stability verification module is further used to calculate a comprehensive process stability index.

[0102] The comprehensive process stability index is obtained by adding the absolute values ​​of the forecast standard deviation, range, and process forecast error according to preset weights. The smaller the value of the comprehensive process stability index, the better the forecast stability and process forecast effect of the model.

[0103] In this preferred embodiment, the absolute values ​​of the above three indicators are added together according to the weights (2:1:2) to form a comprehensive process stability index. The smaller the value, the better the forecast stability and process forecast effect of the model.

[0104] As a preferred embodiment of the present invention, specifically, a multi-dimensional test for spatial similarity of precipitation is established, including...

[0105] Target object identification: The gridded data within the precipitation field is processed into a two-dimensional array of size xDim*yDim. Gaussian threshold filtering is used to identify and merge precipitation objects to obtain forecast and actual objects (verification requires simultaneous acquisition of both forecast and actual precipitation field data; objects identified and merged from forecast gridded data are forecast objects; objects identified and merged from actual gridded data are actual objects).

[0106] After obtaining the forecast and actual target objects, the system searches for actual objects within a square area centered on the geometric center of the forecast object. Specifically, it obtains the bounding rectangle of the forecast object, then extends each side of the bounding rectangle outwards by a predetermined grid width to create a search rectangle. If an actual object has a point within the search rectangle, it is added to the current forecast object's score set. If no actual object matches, it is considered a null forecast. If multiple actual objects match, they are scored, and the object with the highest score is selected to generate an object pair. (Here, the points in the actual and forecast objects refer to grid points.)

[0107] The scoring is based on the spatial similarity of precipitation using a multidimensional test. Specifically, the spatial similarity of precipitation using a multidimensional test is as follows:

[0108] ,

[0109] Through comparative evaluation experiments, the value of M was set to 4.

[0110] Let be the ratio of the overlapping areas of the j-th object pair. =[Number of intersecting grid points / (Actual grid points + Predicted grid points)] * 2

[0111] The area ratio of the j-th object pair:

[0112] When 0 <= R <= 0.8, =R / 0.8,

[0113] When R > 0.8, =1,

[0114] Where R is a value less than 1 in which the area of ​​the forecast object (i.e., the total number of grid points) is divided by the area of ​​the actual object (i.e., the total number of grid points), or the area of ​​the actual object (i.e., the total number of grid points) is divided by the area of ​​the forecast object (i.e., the total number of grid points).

[0115] This is the absolute value of the major axis angle difference between the j-th object pair, i.e., the absolute value of the cosine of the angle between the longest axis of the predicted object's geometry and the longest axis of the actual object's geometry.

[0116] =|COS(angleDelta)|,

[0117] Let be the geometric center distance of the j-th object pair.

[0118] Specific parameters: distance D is the distance between the geometric center of the actual object and the geometric center of the predicted object; optimal distance Dmin; maximum tolerance distance Dmax.

[0119] When D <= Dmin: =1,

[0120] When Dmin <D<= Dmax : =1-(D- Dmin) / [Dmax - Dmin],

[0121] When D > Dmax: =0,

[0122] Here, we set Dmin = 30KM and Dmax = 300KM.

[0123] Function weights w: 0.4, 0.3, 0.1, 0.2

[0124] Confidence level c: =1、 =1、 = , = The ratio of small areas to large areas in the actual and forecast objects, where and The aspect ratios of the forecast and the observed objects are respectively, when or When the confidence function approaches 1, The value is close to 0.

[0125] In this preferred embodiment, the spatial verification mainly adopts the MODE method, which consists of three main steps: target object identification, target object matching, and target object verification.

[0126] The specific operation for target object identification is as follows: Grid data within the precipitation field is processed into a two-dimensional array of size xDim*yDim. The project uses a resolution of 5km*5km, and Gaussian threshold filtering is used to identify and merge precipitation objects. Based on local heavy rainfall characteristics, the project sets the number of grid points in the merged object area to be considered a valid target object if it exceeds 200 (approximately 70km*70km).

[0127] After obtaining the forecast and observed precipitation targets, the system searches for observed precipitation targets within a square radius centered on the geometric center of the forecast target. Specifically, it obtains the bounding rectangle of the forecast target, then extends each side of the rectangle outwards by 10 grid points (set as the `search_radius` parameter in the program) to create a search rectangle. If an observed precipitation target falls within this search rectangle, it is added to the current forecast target's score set. No matching target is detected (empty forecast); if multiple targets are detected, they are scored, and the target with the highest score is selected to form a target pair.

[0128] In a preferred embodiment of the present invention, specifically, the multi-dimensional fusion test and evaluation indicators for heavy precipitation are determined, including...

[0129] Based on the experiment of testing and evaluating multiple heavy precipitation cases, the 24-hour mean absolute error of model precipitation forecasts, TS score for heavy rain and above, heavy rain forecast bias, hourly clear and rain forecast accuracy, heavy rain spatial similarity, and comprehensive index of process stability were selected from the output results of the deterministic forecast testing module, process forecast stability testing module, and spatial testing module and dimensionlessly fused to construct a multidimensional fusion testing and evaluation index.

[0130] The six indicators here are all derived from the basic verification indicators mentioned earlier. These basic indicators correspond to different precipitation levels and can be categorized as "heavy rain or above (TS)" or "torrential rain or above (TS)" with precipitation levels. The selection of these six indicators is based on the six dimensions required for assessing torrential rain in weather forecast verification operations, and they are also the dimensions that the project is currently prioritizing for assessment.

[0131] The 24-hour mean absolute error, TS score for heavy rain and above, rainstorm forecast deviation, hourly weather forecast accuracy, and rainstorm spatial similarity correspond to the 24-hour cumulative precipitation mean absolute error, 24-hour heavy rain and above TS score, 24-hour rainstorm forecast deviation, 24-hour hourly weather forecast accuracy, and 24-hour rainstorm magnitude spatial similarity, respectively.

[0132] As a preferred embodiment of the present invention, the method further includes performing equal-weighted arithmetic averaging on the dimensionless fusion 24h mean absolute error, TS score for heavy rain and above, rainstorm forecast deviation, hourly clear and rain forecast accuracy, rainstorm spatial similarity, and process stability comprehensive index to obtain the comprehensive test evaluation index CEI. The higher the CEI score, the better the model's multi-dimensional fusion forecast capability for heavy precipitation.

[0133] The scores for each of the above indicators were all processed using a dimensionless fusion method, with scores ranging from 0 to 100. A score closer to 100 indicates a better performance in that evaluation dimension. The specific fusion methods for each indicator are shown in Table 1 below:

[0134]

[0135] Table 1

[0136] Another application example of this invention is given below.

[0137] A multi-dimensional fusion test and comparative evaluation was conducted on the forecasting effectiveness of GIFT, CMA-GFS, CMA-TRAMS, CMA-GD, and ECMWF for the 2022 Guangdong Province rainstorm event. According to... Figure 1 The process steps are inspected.

[0138] Step A: Obtain the 00-96h precipitation grid forecast values ​​from 12 UTC in 2022 for each of the above subjective and objective models; the corresponding actual precipitation data are from 86 national automatic weather stations and approximately 3,000 township and regional automatic weather stations in Guangdong Province.

[0139] Step B involves conducting a multi-dimensional fusion test of heavy precipitation.

[0140] Step C yields the multidimensional fusion test and evaluation results of precipitation, including radar charts and the Comprehensive Test and Evaluation Index (CEI).

[0141] Step B specifically involves:

[0142] Step B1: Conduct deterministic forecast tests for continuous variables, deterministic forecast tests for binary events, process forecast stability tests, and precipitation spatial similarity tests, respectively, to obtain, but not limited to, the following test results: 24-hour average absolute error of precipitation, 24-hour heavy rain or above TS, 24-hour rainstorm forecast bias, hourly clear / rainy forecast accuracy, rainstorm spatial similarity, and comprehensive process stability index, etc.

[0143] Step B2: Dimensionlessly integrate the comprehensive indicators of 24-hour average absolute error of precipitation, 24-hour heavy rain or above TS, rainstorm forecast deviation, hourly clear and rainy forecast accuracy, rainstorm spatial similarity, and process stability.

[0144] Step B3: Based on the 24-hour average absolute error of precipitation, 24-hour heavy rain or above TS, rainstorm forecast deviation, hourly clear and rain forecast accuracy, rainstorm spatial similarity, and process stability comprehensive index, draw a multi-dimensional test and evaluation chart and generate the comprehensive test and evaluation index (CEI).

[0145] Technical Results: The forecasting capabilities of various subjective and objective forecasting products for different types of heavy precipitation in 2022 were comprehensively evaluated. By comparing the models, the forecasting characteristics of each model in different dimensions can be identified.

[0146] The final multidimensional test and evaluation chart obtained in this embodiment is as follows: Figure 3 As shown; the Comprehensive Evaluation Index (CEI) is generated as shown in Table 2 below (Comprehensive Evaluation Index of Subjective and Objective Models of Different Types of Rainstorms in 2022).

[0147]

[0148] Table 2

[0149] In a preferred embodiment of the present invention, the visualization is specifically performed as follows:

[0150] The evaluation results are generated into a radar chart, and the radar chart and CEI value are visualized.

[0151] This invention also proposes a multi-dimensional fusion verification and evaluation system for deterministic heavy precipitation forecasts, including the following:

[0152] The data acquisition module is used to acquire model precipitation forecast data and station real-time data for the target assessment area as the data to be assessed.

[0153] The deterministic forecast testing module is used to perform deterministic forecast testing on continuous variables and deterministic forecast testing on binary events for the data to be evaluated.

[0154] The Model Process Forecast Stability Validation Module is used to perform model process forecast stability validation on the data to be evaluated using a multidimensional test-based process forecast stability validation method.

[0155] A spatial verification module is established to perform precipitation spatial similarity verification on the precipitation spatial similarity of the data to be evaluated based on multidimensional verification.

[0156] The module for determining the multidimensional fusion test and evaluation index of heavy precipitation is used to determine the multidimensional fusion test and evaluation index of heavy precipitation based on the deterministic forecast test module, the process forecast stability test module and the spatial test module, and then obtain the evaluation results.

[0157] The visualization module is used to visualize the evaluation results.

[0158] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0159] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or system capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0160] Although the description of the invention has been quite detailed and particularly of several described embodiments, it is not intended to limit it to any of these details or embodiments or any particular embodiment, but should be considered as providing a broad possible interpretation of the claims by referring to the appended claims and taking into account the prior art, thereby effectively covering the intended scope of the invention. Furthermore, the invention has been described above with respect to embodiments foreseeable by the inventors in order to provide a useful description, and non-substantial modifications to the invention that have not yet been foreseen may still represent equivalent modifications.

[0161] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. Any embodiment that achieves the technical effects of the present invention using the same means should fall within the protection scope of the present invention. Within the protection scope of the present invention, various modifications and variations can be made to the technical solutions and / or implementation methods.

Claims

1. A multi-dimensional fusion verification and evaluation method for deterministic heavy precipitation forecasts, characterized in that, Including the following: Obtain model precipitation forecast data and station-based real-time data for the target assessment area as the data to be assessed. The data to be evaluated is input into the pre-established multi-dimensional fusion test and evaluation system for heavy precipitation to obtain the evaluation results; The evaluation results will be visualized. Specifically, the process of establishing a multidimensional fusion testing and evaluation system includes: Establish modules for deterministic forecast testing of continuous variables and deterministic forecast testing of binary events. A model process forecast stability testing module is established based on a multidimensional testing method. A multidimensional spatial similarity test for precipitation is established, and a spatial test module is built based on this. Based on the deterministic forecast verification module, the process forecast stability verification module, and the spatial verification module, multidimensional fusion verification and evaluation indicators for heavy precipitation are determined, and a multidimensional fusion verification and evaluation system is established. Specifically, modules for deterministic forecast testing of continuous variables and deterministic forecast testing of binary events are established, including: This paper establishes the relative error, absolute error, and root mean square error of cumulative precipitation forecasts; it also establishes the TS score, BIAS score, hit rate, missed prediction rate, and false alarm rate for cumulative precipitation; it further establishes the accuracy and correlation coefficient of hourly precipitation forecasts; and finally, it establishes a deterministic forecast verification module, where the correlation coefficient refers to the statistical correlation coefficient. ; Specifically, a model process forecast stability testing module is established based on the multidimensional testing method, including: Forecast standard deviation, range, and process forecast error are used as indicators to verify the stability of model process forecasts. Based on these indicators, a model process forecast stability verification module is established. The forecast standard deviation is calculated as follows: ， In the formula, For the i-th reported precipitation value in the process, This is the average of n reported precipitation values. The range is calculated as follows: ， In the formula, and These are the maximum and minimum precipitation values ​​from n reported precipitation data points, respectively. The method for calculating the process forecast error is as follows: ， In the formula, This is the average of n reported precipitation values. This is the actual precipitation value; The model process forecast stability verification module is also used to calculate the comprehensive stability index of the process. The process stability index is obtained by adding the absolute values ​​of the forecast standard deviation, range, and process forecast error according to preset weights. The smaller the value of the process stability index, the better the forecast stability and process forecast effect of the model. Specifically, a multi-dimensional test for spatial similarity of precipitation is established, including: Target object identification: The grid data in the precipitation field is processed into a two-dimensional array of size xDim*yDim, and Gaussian threshold filtering is used to identify and merge precipitation objects to obtain forecast and actual objects; After obtaining the forecast and actual target objects, the system searches for actual objects within a square area centered on the geometric center of the forecast object. Specifically, it obtains the bounding rectangle of the forecast object and then extends a preset value of grid width outward from each side of the bounding rectangle to obtain a search rectangle. If an actual object falls within the search rectangle, it is added to the current forecast object's score set. If no actual object matches, it is considered a null forecast. If multiple actual objects match, they are scored, and the object with the highest score is selected to generate an object pair. The scoring is based on the spatial similarity of precipitation using a multidimensional test. Specifically, the spatial similarity of precipitation using a multidimensional test is as follows: ， Through comparative evaluation experiments, the value of M was set to 4. Let the ratio of the overlapping areas of the j-th object pair be denoted as . =[Number of intersecting grid points / (Actual grid points + Predicted grid points)] * 2 The area ratio of the j-th object pair: When 0 <= R <= 0.8, =R / 0.8, When R > 0.8, =1, Where R is a value less than 1 in which the area of ​​the forecast object (i.e., the total number of grid points of the forecast object) is divided by the area of ​​the actual object (i.e., the total number of grid points of the actual object), or the area of ​​the actual object (i.e., the total number of grid points of the actual object) is divided by the area of ​​the forecast object (i.e., the total number of grid points of the forecast object). It is the absolute value of the cosine of the angle difference between the major axes of the j-th object pair, that is, the absolute value of the cosine of the angle between the longest axis of the predicted object's geometry and the longest axis of the actual object's geometry; =|COS(angleDelta)| Let be the geometric center distance of the j-th object pair. Specific parameters: distance D is the distance between the geometric center of the actual object and the geometric center of the predicted object; optimal distance Dmin; maximum tolerance distance Dmax. When D <= Dmin: =1, When Dmin < D <= Dmax: = 1 - (D - Dmin) / [Dmax - Dmin], When D > Dmax: =0, Function weights w: 0.4, 0.3, 0.1, 0.2 Confidence level c: =1、 =1、 = , = The ratio of small areas to large areas in the actual and forecast objects, where and The aspect ratios of the forecast and the observed objects are respectively, when or When the confidence function approaches 1, The value is close to 0; Specifically, determine the multi-dimensional integrated evaluation indicators for heavy precipitation, including: Based on the experiment of testing and evaluating multiple heavy precipitation cases, the 24-hour mean absolute error of model precipitation forecasts, TS score for heavy rain and above, heavy rain forecast bias, hourly clear and rain forecast accuracy, heavy rain spatial similarity, and comprehensive index of process stability were selected from the output results of the deterministic forecast testing module, process forecast stability testing module, and spatial testing module and dimensionlessly fused to construct a multidimensional fusion testing and evaluation index.

2. The multi-dimensional fusion verification and evaluation method for deterministic heavy precipitation forecasts according to claim 1, characterized in that, The method further includes performing equal-weighted arithmetic averaging on the dimensionless fusion of the 24-hour average absolute error, TS score for heavy rain and above, rainstorm forecast bias, hourly weather forecast accuracy, rainstorm spatial similarity, and process stability comprehensive index to obtain the comprehensive test and evaluation index (CEI). The higher the CEI score, the better the model's multi-dimensional fusion forecast capability for heavy precipitation.

3. The multi-dimensional fusion verification and evaluation method for deterministic heavy precipitation forecasts according to claim 2, characterized in that, Specifically, the method of visualization is as follows: The evaluation results are generated into a radar chart, and the radar chart and CEI value are visualized.

4. A multi-dimensional fusion verification and evaluation system for deterministic heavy precipitation forecasts, characterized in that, The system comprising the steps of the method according to any one of claims 1-3 above, wherein the system includes the following: The data acquisition module is used to acquire model precipitation forecast data and station real-time data for the target assessment area as the data to be assessed. The deterministic forecast testing module is used to perform deterministic forecast testing on continuous variables and deterministic forecast testing on binary events for the data to be evaluated. The Model Process Forecast Stability Validation Module is used to perform model process forecast stability validation on the data to be evaluated using a multidimensional test-based process forecast stability validation method. A spatial verification module is established to perform precipitation spatial similarity verification on the precipitation spatial similarity of the data to be evaluated based on multidimensional verification. The module for determining the multidimensional fusion test and evaluation index of heavy precipitation is used to determine the multidimensional fusion test and evaluation index of heavy precipitation based on the deterministic forecast test module, the process forecast stability test module and the spatial test module, and then obtain the evaluation results. The visualization module is used to visualize the evaluation results.

Images