Meteorological early warning service quality-based effect evaluation model construction method

Abstract

The invention discloses a method for constructing an effect evaluation model based on meteorological early warning service quality, and relates to the technical field of meteorological early warning service quality evaluation, and the main scheme is as follows: calculating a structural dimension index of a precipitation field by analyzing forecast precipitation intensity data and observation precipitation intensity data; performing grade evaluation on the structural dimension index through a grade division model, and analyzing forecast occurrence time data and actual occurrence time data to obtain a timeliness score; a public influence degree score is obtained through the early warning level data, the actual occurrence position data and the received data of the mobile phone users in the urban area and the remote area, the structural dimension comprehensive score, the timeliness comprehensive score and the public influence degree score are comprehensively analyzed, and a comprehensive quality score is obtained; and summarizing the assessed contents into a comprehensive assessment report, thereby solving the problems of single assessment mode, lack of comprehensive assessment and poor assessment authenticity in the prior art.

Description

Technical Field

[0001] This invention relates to the field of meteorological early warning service quality assessment technology, specifically to a method for constructing an effectiveness assessment model based on meteorological early warning service quality. Background Technology

[0002] With advancements in science and technology and continuous progress in modern meteorological observation techniques, the accuracy and timeliness of weather forecasts have been continuously improved, providing more reliable meteorological services to various industries. In the field of meteorological forecasting system evaluation, with the increasing frequency of climate change and extreme weather events, the requirements for the accuracy and timeliness of meteorological early warning services are becoming increasingly stringent. Therefore, it is necessary to conduct quality assessments of meteorological early warning systems to ensure their reliability.

[0003] Existing meteorological forecasting system evaluation methods often focus on the evaluation of a single indicator, such as the accuracy of the forecast or the timing of the warning, lacking a comprehensive evaluation of the quality of the warning service. This results in poor evaluation results and makes it difficult to accurately reflect the actual effectiveness of the warning service. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] To address the shortcomings of existing technologies, this invention provides a method for constructing an effectiveness evaluation model based on the quality of meteorological early warning services, which at least solves the problems of single evaluation methods, lack of comprehensive evaluation, and poor evaluation accuracy in existing technologies.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for constructing an effectiveness evaluation model based on meteorological early warning service quality, comprising:

[0008] Step 1: Obtain forecast precipitation intensity data of the precipitation field through the numerical forecast model of the meteorological forecast system, obtain observed precipitation intensity data of the precipitation field through meteorological observation stations, and calculate the structural dimension index of the precipitation field by analyzing the forecast precipitation intensity data and observed precipitation intensity data.

[0009] Step 2: Evaluate the structural dimension indicators using a grading model to obtain the evaluation grades of different indicators within different structural dimension indicators, and calculate the comprehensive score of the structural dimension using different structural dimension indicators and their corresponding evaluation grades.

[0010] Step 3: Obtain forecast time data for extreme weather events through the meteorological early warning system and actual time data through meteorological observation stations; obtain a timeliness score by analyzing the forecast and actual time data.

[0011] Step 4: Obtain the warning level data of the precipitation field through the meteorological warning system, obtain the actual location data of the precipitation field through the meteorological observation station, obtain the reception data of mobile phone users in urban areas and mobile phone users in remote areas, analyze the data in Step 4 to obtain the public impact score, and determine the public impact score level.

[0012] Step 5: Based on the comprehensive analysis of the structural dimension comprehensive score, the timeliness comprehensive score, and the public impact score, a comprehensive quality score is obtained; and the quality level of the weather forecast system is determined based on the comprehensive quality score.

[0013] Step Six: Compile all the assessment results from Steps One to Five into a comprehensive assessment report.

[0014] In the preferred embodiment of the above-mentioned method for constructing an effectiveness evaluation model based on the quality of meteorological early warning services, in step one, the structural dimension indicators include precipitation field morphology similarity value, precipitation intensity deviation value, and offset distance value.

[0015] In the preferred embodiment of the above-mentioned method for constructing an effectiveness evaluation model based on meteorological early warning service quality, in step one, the precipitation field is first divided into grids, and the coordinates of different grids are defined as P(i,j); then, a data analysis model is constructed, and the predicted precipitation intensity data and observed precipitation intensity data are analyzed through the constructed data analysis model to calculate the mean of predicted precipitation intensity, the mean of observed precipitation intensity, the standard deviation of predicted precipitation intensity, the standard deviation of observed precipitation intensity, and the covariance of precipitation intensity; the formulas used are as follows:

[0016]

[0017]

[0018] Among them, P f (i,j) represents the predicted precipitation intensity at different grid points (i,j); P o (i,j) represents the observed precipitation intensity at different grid points (i,j); μ x μ represents the mean of the predicted precipitation intensity at different grid points. y σ represents the mean of observed precipitation intensity at different grid points; x σ represents the standard deviation of the predicted precipitation intensity at different grid points. y σ represents the standard deviation of observed precipitation intensity at different grid points. xy This represents the covariance between the predicted precipitation intensity and the observed precipitation intensity; M and N represent the number of grid points in the horizontal and vertical directions of the precipitation field; H represents the total number of grids, and H = M * N.

[0019] In the preferred scheme of the above-mentioned method for constructing an effectiveness evaluation model based on meteorological early warning service quality, a precipitation field morphology similarity analysis model is constructed to analyze the mean of forecast precipitation intensity, the mean of observed precipitation intensity, the standard deviation of forecast precipitation intensity, the standard deviation of observed precipitation intensity, and the covariance of precipitation intensity, thereby obtaining the precipitation field morphology similarity of precipitation intensity data. The calculation principle of the model is as follows:

[0020]

[0021] Wherein, SSIM(x,y) represents the similarity value of the precipitation field morphology between the predicted precipitation intensity data and the observed precipitation intensity data; C1 and C2 represent stability constant one and stability constant two, respectively;

[0022] The precipitation intensity analysis model is used to analyze forecasted precipitation intensity data and observed precipitation intensity data to calculate the precipitation intensity deviation. The formula used is as follows:

[0023]

[0024] Wherein, RMSE represents the precipitation intensity deviation value;

[0025] By constructing a correlation analysis model to analyze forecasted precipitation intensity data and observed precipitation intensity data, the cross-correlation value between the forecasted precipitation intensity data and observed precipitation intensity data is calculated. The formula used is as follows:

[0026] XG(dx,dy)=∑ i,j P f (i,j)×P o (i+dx,j+dy);

[0027] Where XG(dx,dy) represents the cross-correlation value between forecast precipitation intensity data and observed precipitation intensity data, and dx and dy represent the horizontal and vertical offsets of the grid, respectively;

[0028] By iterating through the cross-correlation values ​​with different offsets, we find the maximum value among the cross-correlation values. The offset (dx, dy) corresponding to the maximum value among all cross-correlation values ​​is the offset between the precipitation intensity data and the observed precipitation intensity data.

[0029] The offset distance is calculated based on the offset (dx, dy) corresponding to the maximum value among all cross-correlation values, using the following formula:

[0030]

[0031] Where JLZ represents the offset distance value.

[0032] In the preferred solution of the method for constructing the effect evaluation model based on the quality of meteorological warning services, in step two, the evaluation grades corresponding to different structural dimension indicators are as follows: The evaluation grade for the similarity of precipitation field patterns is:

[0033] 1 ≥ SSIM(x,y) ≥ 0.9: Rated as an A-level precipitation field pattern similarity score, with a score of 90 - 100 points;

[0034] 0.8 ≤ SSIM(x,y) < 0.9: Rated as a B-level precipitation field pattern similarity score, with a score of 80 - 89 points;

[0035] 0.7 ≤ SSIM(x,y) < 0.8: Rated as a C-level precipitation field pattern similarity score, with a score of 70 - 79 points; SSIM(x,y) < 0.7: Rated as a D-level precipitation field pattern similarity score, with a score of 60 points;

[0036] The evaluation grade for intensity deviation score is:

[0037] 0 < RMSE ≤ 0.1: Rated as an A-level intensity deviation score, with a score of 90 - 100 points;

[0038] 0.1 < RMSE ≤ 0.15: Rated as a B-level intensity deviation score, with a score of 80 - 89 points;

[0039] 0.15 < RMSE ≤ 0.2: Rated as a C-level intensity deviation score, with a score of 70 - 79 points;

[0040] RMSE > 0.2: Rated as a D-level intensity deviation score, with a score of 60 points;

[0041] The evaluation grade for offset distance score is:

[0042] 0 < JLZ ≤ 5km: Rated as an A-level offset distance score, with a score of 90 - 100 points; [[ID=3..]]

[0043] 5km < JLZ ≤ 10km: Rated as a B-level offset distance score, with a score of 80 - 89 points;

[0044] 10km < JLZ ≤ 15km: Rated as a C-level offset distance score, with a score of 70 - 79 points;

[0045] JLZ > 15km: Rated as a D-level offset distance, with a score of 60 points.

[0046] In the preferred solution of the method for constructing the effect evaluation model based on the quality of meteorological warning services, in step two, the calculation method of the comprehensive score of the structural dimension is as follows:

[0047]

[0048] Among them, PF k This represents the specific score for different indicator items, where k represents the index of the different indicator item, taking values ​​of 1, 2, or 3, corresponding to the precipitation field morphology similarity value, precipitation intensity deviation value, and offset distance value, respectively; SC k S1 represents the value of the k-th indicator item; S2 represents the lower limit of the level, S3 represents the upper limit of the level below, S4 represents the upper limit of the score of the level, S5 represents the upper limit of the score of the level below, and S6 represents the lower limit of the score of the level.

[0049] By analyzing the specific scores of different indicators, a comprehensive score for the structural dimensions is calculated using the following formula:

[0050]

[0051] Where ZH represents the overall score of the structural dimension; α k This represents the weighting coefficient for specific scores of different indicators.

[0052] In the preferred embodiment of the above-mentioned method for constructing an effectiveness evaluation model based on the quality of meteorological early warning services, in step three: by constructing a timeliness analysis model; analyzing the forecast and actual occurrence time data of typhoon, rainstorm, and snowstorm weather in the meteorological early warning system, and calculating the comprehensive timeliness score, the formula used is as follows:

[0053]

[0054] TQL represents the overall timeliness score; TY R This indicates the predicted time of the Rth typhoon; TS R DY represents the actual time of the Rth typhoon; R represents the typhoon number, which is a positive integer; p DS represents the predicted time of the p-th rainstorm. p This represents the actual time of the p-th rainstorm, where p represents the sequence number of the rainstorm and takes a positive integer value; BY m This indicates the predicted time of the m-th blizzard, BS. m Let represent the actual occurrence time of the m-th blizzard, where m represents the sequence number of the blizzard and takes a positive integer value; β1 is the weighting coefficient of the time lead fraction for typhoon weather; β2 is the weighting coefficient of the time lead fraction for rainstorm weather; β3 is the weighting coefficient of the time lead fraction for blizzard weather, and β1 + β2 + β3 = 1; T h D h and B h These represent the golden lead time for forecasting typhoon weather, heavy rain weather, and heavy snow weather, respectively.

[0055] In the preferred embodiment of the above-mentioned method for constructing an effectiveness evaluation model based on the quality of meteorological early warning services, in step four: a sliding window and an early warning threshold are set, the hit window is slid across the grid map of the precipitation field, the early warning value of each grid point in the hit window is compared with the early warning threshold, and if at least 30% of the grid points covered by the hit window exceed the early warning threshold, then this coverage area is defined as a hit window, the sliding window is used to cover all locations in the precipitation field, and the total number of hit windows is counted as the number of hit windows;

[0056] The actual disaster windows are counted from the actual location data.

[0057] A window coverage calculation model is constructed, and the window coverage is calculated using this model. The method used is as follows:

[0058]

[0059] Where MZ represents window coverage, CK Y CK indicates the number of windows hit. S Indicates the actual number of disaster windows;

[0060] By constructing a signal coverage calculation model, the received data of mobile phone users in urban areas and remote areas are input into the model to obtain the signal coverage. The calculation principle of the model is as follows:

[0061]

[0062] Where FG represents signal coverage; CS J This indicates the number of mobile phone users in the urban area who received the warning information; CS Z This represents the total number of mobile phone users in the urban area; PY J This indicates the number of mobile phone users in remote areas who received the warning message; PY Z γ1 represents the number of mobile phone users in remote areas who received the warning information; γ2 represents the warning information reception rate in urban areas; γ1+γ2=1.

[0063] In the preferred embodiment of the above-mentioned method for constructing an effectiveness evaluation model based on the quality of meteorological early warning services, in step four: by analyzing the window coverage and signal coverage, the public impact score is obtained, based on the following formula:

[0064]

[0065] Among them, GZ represents the public influence score; Weighting coefficients representing window coverage; The weighting coefficients representing signal coverage, and

[0066] In the preferred embodiment of the above-mentioned method for constructing an effectiveness evaluation model based on the quality of meteorological early warning services, in step five, a comprehensive quality analysis model is constructed. The structural dimension comprehensive score, the timeliness comprehensive score, and the public impact score are input into the comprehensive quality analysis model to calculate the comprehensive quality score. The calculation principle of the model is as follows:

[0067] ZL=μ1×ZH+μ2×TQL+μ3×GZ;

[0068] Where ZL represents the overall quality score; μ1 is the weight coefficient of the overall score for structural dimensions; μ2 is the weight coefficient of the overall score for timeliness; and μ3 is the weight coefficient of the score for public impact, μ1+μ2+μ3=1.

[0069] (III) Beneficial Effects

[0070] This invention provides a method for constructing an effectiveness evaluation model based on the quality of meteorological early warning services, which has the following beneficial effects:

[0071] (1) Existing evaluation methods often focus on a single indicator, such as forecast accuracy, and lack consideration for timeliness and public impact. This proposal provides a more comprehensive evaluation method by integrating three dimensions: structural dimension, timeliness, and public impact. This method can more accurately reflect the actual effect of early warning services, rather than just a single accuracy indicator.

[0072] (2) This scheme provides a scientific and systematic evaluation method by using a grading model to assess the structural dimension indicators and calculate a comprehensive score. This method can more objectively evaluate all aspects of the early warning service, avoid the arbitrariness of subjective judgment, and improve the accuracy and reliability of the evaluation.

[0073] (3) Existing assessment methods are not detailed enough in evaluating timeliness, making it difficult to accurately reflect the timeliness of early warning services. This solution analyzes the difference between the predicted and actual times of occurrence to calculate a timeliness score, thus accurately assessing the timeliness of early warning services. This is of great significance for improving the response speed of early warning services and reducing disaster losses.

[0074] (4) This plan analyzes the reception of early warning information to assess the actual impact of early warning services on the public. This helps to understand the public's satisfaction and trust in early warning services, providing a basis for service improvement.

[0075] (5) By comprehensively analyzing the structural dimension comprehensive score, the timeliness comprehensive score, and the public impact score, this scheme can derive a comprehensive quality score, thereby determining the quality level of the weather forecast system. This comprehensive scoring method can more comprehensively reflect the overall quality of the early warning service and provide users with more valuable reference information. Attached Figure Description

[0076] Figure 1 This is a schematic diagram illustrating the steps of constructing a performance evaluation model for meteorological early warning service quality according to the present invention. Detailed Implementation

[0077] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0078] Example 1

[0079] Please see Figure 1 This invention provides a method for constructing an effectiveness evaluation model based on the quality of meteorological early warning services, including:

[0080] Step 1: Obtain forecast precipitation intensity data of the precipitation field through the numerical forecasting model of the meteorological forecasting system, obtain observed precipitation intensity data of the precipitation field through meteorological observation stations, and calculate the structural dimension index of the precipitation field by analyzing the forecast precipitation intensity data and observed precipitation intensity data.

[0081] Step 101: The precipitation field is divided into gridded partitions, and the coordinates of different grids are defined as P(i,j). By dividing the precipitation field into a regular M*N grid and defining a unified coordinate P(i,j), spatial consistency processing between forecast and observation data is achieved. After gridding, the data within each cell can be compared point-to-point, avoiding analytical biases caused by differences in spatial resolution. This provides a structured data foundation for subsequent statistical calculations and avoids the spatial distribution discretization problem and spatial alignment errors caused by mismatch between observation points and forecast grid positions that exist in traditional meteorological data analysis.

[0082] Step 102: Obtain the forecast precipitation intensity data of the precipitation field through the numerical forecast model of the meteorological forecast system, and obtain the observed precipitation intensity data of the precipitation field through the meteorological observation station.

[0083] It should be noted that both forecast and observed precipitation intensity data need to undergo normalization preprocessing before subsequent calculations. This normalization eliminates the differences in dimensions between different parameters, facilitating comprehensive calculations. Extreme value normalization or Z-score standardization methods can be used to eliminate dimensional differences, mapping precipitation intensity values ​​to [0,1] or a standard normal distribution space. This processing significantly improves the calculation stability of statistics such as covariance and correlation coefficients, preventing comprehensive indicators like SSIM from being dominated by a few high-value regions due to differences in magnitude.

[0084] Step 103: Construct a data analysis model. Analyze the forecasted precipitation intensity data and observed precipitation intensity data using the constructed data analysis model, calculating the mean of forecasted precipitation intensity, the mean of observed precipitation intensity, the standard deviation of forecasted precipitation intensity, the standard deviation of observed precipitation intensity, and the covariance of precipitation intensity. The formulas used are as follows:

[0085]

[0086]

[0087] Among them, P f (i,j) represents the predicted precipitation intensity at different grid points (i,j); P o (i,j) represents the observed precipitation intensity at different grid points (i,j); μ x μ represents the mean of the predicted precipitation intensity at different grid points. y σ represents the mean of observed precipitation intensity at different grid points; x σ represents the standard deviation of the predicted precipitation intensity at different grid points. y σ represents the standard deviation of observed precipitation intensity at different grid points. xy This represents the covariance between the predicted precipitation intensity and the observed precipitation intensity; M and N represent the number of grid points in the horizontal and vertical directions of the precipitation field; H represents the total number of grids, and H = M * N.

[0088] Step 104: By constructing a precipitation field morphology similarity analysis model, the mean of predicted precipitation intensity, the mean of observed precipitation intensity, the standard deviation of predicted precipitation intensity, the standard deviation of observed precipitation intensity, and the covariance of precipitation intensity are analyzed to obtain the precipitation field morphology similarity of precipitation intensity data. The formula used is as follows:

[0089]

[0090] Wherein, SSIM(x,y) represents the similarity value of the precipitation field morphology between the forecast precipitation intensity data and the observed precipitation intensity data; C1 and C2 represent stability constant one and stability constant two, respectively, and can be taken as 0.1-1 to prevent the denominator from being zero.

[0091] The SSIM value integrates three dimensions: mean precipitation intensity, standard deviation of precipitation intensity, and covariance of precipitation intensity. The introduction of stability constant one and stability constant two avoids the instability of the denominator approaching zero in low precipitation areas and avoids the inability of traditional point-to-point error indicators, such as MAE, to assess the spatial similarity of precipitation field morphology, such as insensitivity to the shift of the rainstorm center location and the difference in precipitation band morphology.

[0092] Step 105: Analyze the forecasted precipitation intensity data and observed precipitation intensity data using the constructed precipitation intensity analysis model, and calculate the precipitation intensity deviation value. The formula used is as follows:

[0093]

[0094] RMSE represents the precipitation intensity deviation value.

[0095] By integrating the sum of squared grid-level deviations (the ΣΣ operation in the formula), both the overall deviation level and spatial distribution information are reflected. When a region has consecutive high-deviation grids, it can be determined that the model has systematic errors in that geographic unit (such as a mountain range or coastline), guiding the optimization of terrain parameterization schemes and avoiding the inability of traditional regional average deviations to distinguish spatial distribution differences.

[0096] Step 106: Analyze the forecast precipitation intensity data and observed precipitation intensity data using the constructed correlation analysis model, and calculate the cross-correlation value between the forecast precipitation intensity data and observed precipitation intensity data. The formula used is as follows:

[0097] XG(dx,dy)=∑ i,j P f (i,j)×P o (i+dx,j+dy);

[0098] Where XG(dx,dy) represents the cross-correlation value between forecast precipitation intensity data and observed precipitation intensity data, and dx and dy represent the horizontal and vertical offsets of the grid, respectively.

[0099] By quantifying spatial displacement errors, we can avoid the fact that numerical models often cause overall displacement of the precipitation field due to initial field errors or dynamic process deviations, such as typhoon path shifts, which are difficult to quantify using traditional methods.

[0100] By iterating through the cross-correlation values ​​with different offsets, we find the maximum value among all the cross-correlation values. The offset (dx, dy) corresponding to the maximum value among all the cross-correlation values ​​is the offset between the precipitation intensity data and the observed precipitation intensity data.

[0101] The offset distance is calculated based on the offset (dx, dy) between precipitation intensity data and observed precipitation intensity data, using the following formula:

[0102]

[0103] Among them, JLZ represents the offset distance value.

[0104] Through the fusion of multi-dimensional indicators, the model realizes a comprehensive evaluation of the forecast quality.

[0105] Step 2: Use the grading model to evaluate the grades of the structural dimension indicators, obtain the evaluation grades of different indicators in different structural dimension indicators, and calculate the comprehensive score of the structural dimension through different structural dimension indicators and the corresponding evaluation grades.

[0106] Step 201: Preset the precipitation field form similarity scoring grades in the grading model, specifically:

[0107] 1≥SSIM(x,y)≥0.9: Rated as grade A precipitation field form similarity score, with a score of 90-100 points.

[0108] 0.8≤SSIM(x,y)<0.9: Rated as grade B precipitation field form similarity score, with a score of 80-89 points.

[0109] 0.7≤SSIM(x,y)<0.8: Rated as grade C precipitation field form similarity score, with a score of 70-79 points.

[0110] SSIM(x,y)<0.7: Rated as grade D precipitation field form similarity score, with a score of 60 points.

[0111] Step 202: Set the intensity deviation scoring grades in the grading model, specifically:

[0112] 0<RMSE≤0.1: Rated as grade A intensity deviation score, with a score of 90-100 points.

[0113] 0.1<RMSE≤0.15: Rated as grade B intensity deviation score, with a score of 80-89 points.

[0114] 0.15<RMSE≤0.2: Rated as grade C intensity deviation score, with a score of 70-79 points.

[0115] RMSE>0.2: Rated as grade D intensity deviation score, with a score of 60 points.

[0116] Step 203: Set the offset distance scoring grades in the grading model, specifically:

[0117] 0<JLZ≤5km: Rated as grade A offset distance score, with a score of 90-100 points.

[0118] 5 km < JLZ ≤ 10 km: Rated as B - level offset distance score, with a score of 80 - 89 points.

[0119] 10 km < JLZ ≤ 15 km: Rated as C - level offset distance score, with a score of 70 - 79 points.

[0120] JLZ > 15 km: Rated as D - level offset distance, with a score of 60 points.

[0121] It should be noted that the preset similarity score levels of precipitation field patterns, intensity deviation score levels, and offset distance score levels can be set and adjusted according to the standards in industry standards or regulatory documents. Set the level thresholds based on industry standards or regulatory documents (such as "Methods for Quality Inspection of Meteorological Forecasts") to avoid human - experience intervention and improve the credibility of evaluation results. For example, SSIM ≥ 0.9 is rated as A - level (excellent), RMSE ≤ 0.1 is rated as A - level (high - precision), and JLZ ≤ 5 km is rated as A - level (high spatial matching degree) to ensure that the evaluation system complies with meteorological business specifications.

[0122] In this step, use a standardized grading system: By presetting the score levels (A - D levels) of SSIM, RMSE, and JLZ, map indicators with different dimensions to a standardized scoring range of 0 - 100 points, eliminating the interference of dimensional differences on comprehensive evaluation. For example, SSIM = 0.86 (B - level) corresponds to 80 - 89 points, JLZ = 8 km (B - level) corresponds to 80 - 89 points, and RMSE = 0.12 (B - level) corresponds to 80 - 89 points, achieving horizontal comparability between indicators. In traditional meteorological forecast quality assessment, the dimensions and numerical ranges of different evaluation indicators (such as precipitation field pattern similarity, intensity deviation, and spatial offset) vary greatly, making it difficult to directly compare and synthesize indicators. For example, the numerical scales of SSIM (range 0 - 1), RMSE (possibly 0 - +∞), and offset distance (in kilometers) are different, and it is impossible to achieve comprehensive evaluation through simple weighting or arithmetic mean. In addition, the lack of a unified grading standard easily leads to strong subjectivity in evaluation results and cannot objectively reflect the actual performance of the forecast system.

[0123] Step 204: Input the data in the structural dimension indicators into the grading model and output the evaluation levels corresponding to different structural dimension indicators

[0124] Step 205: Calculate the specific score of each indicator based on the specific value and specific score of each structural dimension indicator. The calculation method is as follows:

[0125]

[0126] Among them, PF kThis represents the specific score for different indicator items, where k represents the index of the different indicator item, taking values ​​of 1, 2, or 3, corresponding to the precipitation field morphology similarity value, precipitation intensity deviation value, and offset distance value, respectively; SC k S1 represents the value of the k-th indicator item; S2 represents the lower limit of the level, S3 represents the upper limit of the next lower level, S4 represents the upper limit of the score of the level, S5 represents the upper limit of the score of the next lower level, and S6 represents the lower limit of the score of the level.

[0127] In this step, specific scores are dynamically calculated based on the actual values ​​of indicators within the same level. This avoids the information loss caused by traditional discretization scoring (such as fixing the score to 80), ensuring a continuous positive correlation between the scoring results and forecast quality. This facilitates subsequent optimization analysis and avoids the inability of traditional grading methods (such as directly taking the median of the level) to distinguish subtle differences in indicator values ​​within the same level. For example, SSIM(x,y) = 0.89 (close to the upper limit of level B) and SSIM(x,y) = 0.81 (close to the lower limit of level B) are both rated as level B (80-89 points), but the former has better actual performance. Traditional methods cannot reflect this difference, resulting in a crude evaluation result.

[0128] For example, when SSIM(x,y) is 0.86, the precipitation field morphological similarity score is B, with a range of 80-90 points.

[0129] Step 206: Analyze the specific scores of different indicators to calculate the comprehensive score of the structural dimensions. The formula used is as follows:

[0130]

[0131] Where ZH represents the overall score of the structural dimension; α k The weighting coefficients representing the specific scores for different indicators can take the following values: α1 = 0.4, α2 = 0.2, α3 = 0.4; these can be adjusted according to specific needs, and α1 + α... 2+ α3 = 1.

[0132] In this step, a comprehensive quantitative evaluation of precipitation field forecast quality is achieved through a graded scoring and weighted synthesis of three indicators: precipitation field morphology similarity, precipitation intensity deviation, and offset distance. This avoids the one-sidedness of relying on a single indicator (such as using only RMSE). When evaluating the indicators, different indicators contribute differently to forecast quality, but traditional methods (such as equal-weighted summation) cannot reflect the priority of key indicators. For example, precipitation field morphology similarity and offset distance are more crucial for depicting the spatial morphology of the precipitation field, while precipitation intensity deviation may be significantly affected by local extreme values, requiring a reduction in its weight.

[0133] Step 3: Obtain forecast time data for extreme weather events through the meteorological early warning system and actual time data through meteorological observation stations; obtain a timeliness score by analyzing the forecast time data and actual time data.

[0134] It should be noted that the forecast time data and the actual time data need to be normalized before proceeding with the subsequent calculation steps. This is to eliminate the dimensions between different parameters, thereby facilitating comprehensive calculation.

[0135] Step 301: Construct a timeliness analysis model; extreme weather mainly includes typhoon weather, rainstorm weather, and blizzard weather; the timeliness analysis model is used to analyze the forecast and actual occurrence time data of historical typhoon weather, rainstorm weather, and blizzard weather in the meteorological early warning system, and calculate the comprehensive timeliness score. The formula used is as follows:

[0136] TQL represents the overall timeliness score; TY R This indicates the predicted time of the Rth typhoon; TS R DY represents the actual time of the Rth typhoon; R represents the typhoon number, which is a positive integer; p DS represents the predicted time of the p-th rainstorm. p This represents the actual time of the p-th rainstorm, where p represents the sequence number of the rainstorm and takes a positive integer value; BY m This indicates the predicted time of the m-th blizzard, BS. m Let represent the actual occurrence time of the m-th blizzard, where m represents the sequence number of the blizzard and takes a positive integer value; β1 is the weighting coefficient of the time lead fraction for typhoon weather, which can be 0.4; β2 is the weighting coefficient of the time lead fraction for rainstorm weather, which can be 0.3; β3 is the weighting coefficient of the time lead fraction for blizzard weather, which can be 0.3, and β1 + β2 + β3 = 1; T h D h and B h The golden lead time for forecasting typhoon, heavy rain, and blizzard weather, respectively, can be set according to industry standards or normative documents. For example, the golden lead time for typhoon weather can be 72 hours, for heavy rain weather it can be 6 hours, and for blizzard weather it can be 12 hours, etc., by referencing normative documents (such as the "Typhoon Warning Business Regulations"). h =72 hours, D in the "Technical Specifications for Rainstorm Weather Warning" h=6 minutes), to ensure that the evaluation system conforms to the actual business situation.

[0137] In this step, the timeliness scores for typhoons, severe convection, and blizzards are calculated independently to accurately match the warning characteristics of each weather type.

[0138] Step 302: Set the timeliness rating levels in the rating model, specifically as follows:

[0139] 90-100 points: rated as Grade A timeliness.

[0140] 80-89 points: rated as Grade B timeliness.

[0141] 70-79 points: rated as C-level timeliness.

[0142] Below 70 points: rated as D-level timeliness.

[0143] The timeliness comprehensive score is compared with the timeliness score level to determine the corresponding timeliness score level.

[0144] It should be noted that the timeliness rating can be set and adjusted according to industry standards or normative documents.

[0145] Step 4: Obtain the warning level data of the precipitation field through the meteorological warning system, obtain the actual location data of the precipitation field through the meteorological observation station, obtain the reception data of mobile phone users in urban areas and mobile phone users in remote areas, analyze the data in Step 4 to obtain the public impact score, and determine the public impact score level.

[0146] It should be noted that the warning level data and the actual location data need to be normalized before subsequent calculation steps to eliminate the dimensions between different parameters, thereby facilitating comprehensive calculation.

[0147] Step 401: Obtain the number of hit windows in the precipitation field. Specifically, set a sliding window and an early warning threshold. Slide the hit window across the grid map of the precipitation field. Compare the early warning value of each grid point in the hit window with the early warning threshold. If at least 30% of the grid points covered by the hit window exceed the early warning threshold, this coverage area is defined as a hit window. Extend the sliding window to cover all locations in the precipitation field and count the total number of hit windows. This avoids the problem of traditional early warning coverage assessments only counting the "total area of ​​the early warning area" or "the number of affected people," which cannot quantify the spatial matching degree between the early warning area and the actual disaster area.

[0148] Step 402: Count the windows where disasters actually occurred from the actual location data, and use this count as the actual disaster window count.

[0149] Step 403: Construct a window coverage calculation model, and calculate the window coverage using the neighborhood probability test method of the window coverage calculation model. The method used is as follows:

[0150]

[0151] Where MZ represents window coverage, CK Y CK indicates the number of windows hit. S This indicates the actual number of disaster windows.

[0152] This step directly reflects how many of the warning hit windows are actual disaster windows. The MZ value identifies warning location deviations, avoiding the shortcomings of traditional coverage calculations (such as warning area / disaster area) that ignore spatial distribution characteristics and fail to reflect the degree of overlap between the warning and disaster areas. For example, when the warning area and the disaster area do not overlap at all, the traditional method scores 0, but it cannot distinguish between cases of "partial coverage but location offset."

[0153] Step 404: Obtain the received data of mobile phone users in urban areas and mobile phone users in remote areas; calculate the signal coverage by analyzing the received data of mobile phone users in urban areas and mobile phone users in remote areas.

[0154] It should be noted that the received data of mobile phone users in urban areas and those in remote areas can be obtained through mobile signaling big data. Before proceeding with subsequent calculations, normalization preprocessing is required to eliminate the dimensions between different parameters, thereby facilitating comprehensive calculations.

[0155] Specifically, the signal coverage calculation model is constructed, and the received data is input into the model to obtain the signal coverage FG. The calculation principle of the model is as follows:

[0156]

[0157] Where FG represents signal coverage; CS J This indicates the number of mobile phone users in the urban area who received the warning information; CS Z This represents the total number of mobile phone users in the urban area; PY J This indicates the number of mobile phone users in remote areas who received the warning message; PY Z γ1 represents the number of mobile phone users in remote areas who received the warning information; γ2 represents the warning information reception rate in urban areas, which can be 0.5; γ1+γ2=1 represents the warning information reception rate in remote areas.

[0158] This solution avoids the problem that the efficiency of early warning information transmission varies significantly between urban and rural areas, and that traditional methods fail to differentiate between urban and rural user groups, leading to assessment results that are biased towards urban areas.

[0159] It should be noted that urban areas refer to the main urban areas of cities and counties; remote areas refer to villages, mountainous areas, etc. outside of cities and counties, which can be adjusted according to assessment needs.

[0160] Step 405: By analyzing window coverage and signal coverage, a public impact score is obtained, based on the following formula:

[0161] GZ=(φ1×MZ×φ2×FG)×100;

[0162] Wherein, GZ represents the public impact score; φ1 represents the weighting coefficient of window coverage, which can be 0.6; φ2 represents the weighting coefficient of signal coverage, which can be 0.4. The weighting coefficient φ1+φ2=1 can be adjusted according to actual needs.

[0163] Step 406: Set the public influence rating levels in the rating model, specifically as follows:

[0164] 90-100 points: rated as A-level public influence.

[0165] 80-89 points: rated as B-level public influence.

[0166] 70-79 points: rated as C-level public influence.

[0167] Scores below 70: Rated as D-level public impact.

[0168] The public influence score is compared with the public influence rating level to determine the corresponding public influence rating level.

[0169] It should be noted that public impact represents the influence of window coverage and signal coverage on the public. That is, the higher the window coverage and signal coverage, the better the positive impact on the public, and the higher the score.

[0170] By integrating data, the effectiveness of the quantitative early warning system across the entire chain from "technical release" to "public response" is demonstrated, reflecting the effectiveness of information transmission.

[0171] Step 5: Based on the comprehensive analysis of the structural dimension comprehensive score, the timeliness comprehensive score, and the public impact score, a comprehensive quality score is obtained; and the quality level of the weather forecast system is determined based on the comprehensive quality score.

[0172] Step 501: By constructing a comprehensive quality analysis model, input the comprehensive score of structural dimensions, the comprehensive score of timeliness, and the score of public impact into the comprehensive quality analysis model, and calculate the comprehensive quality score ZL. The calculation principle of the model is as follows:

[0173] ZL=μ1×ZH+μ2×TQL+μ3×GZ;

[0174] Where: ZL represents the overall quality score; μ1 is the weight coefficient of the structural dimension overall score, which can be 0.4; μ2 is the weight coefficient of the timeliness overall score, which can be 0.3; μ3 is the weight coefficient of the public impact score, which can be 0.3, and μ1+μ2+μ3=1, which can be adjusted as needed.

[0175] By setting weighting coefficients, the core position of structural dimensions is emphasized, which is in line with the business logic that "forecast accuracy is the foundation of early warning". It integrates the full-process indicators of "forecast accuracy - early warning timeliness - public reach", avoids system imbalance caused by local optimization, and avoids the problem in traditional meteorological early warning quality assessment that the scores of different dimensions (such as forecast accuracy, timeliness, and public impact) exist independently and lack systematic integration.

[0176] Step 502: Set the overall quality rating levels in the rating model, specifically as follows:

[0177] 90-100 points: rated as excellent.

[0178] 80-89 points: rated as good.

[0179] 70-79 points: rated as qualified.

[0180] Scores below 70: rated as unqualified.

[0181] The overall quality score is compared with the overall quality score level to determine the corresponding quality level.

[0182] By using a four-level classification (Excellent / Good / Qualified / Unqualified), complex scoring is mapped to actionable business instructions. Through a unified grading standard, horizontal comparisons of early warning systems in different regions and at different times can be achieved.

[0183] Step Six: Compile all the assessment results from Steps One to Five into a comprehensive assessment report.

[0184] Specifically, the comprehensive assessment report may include assessment results for at least the precipitation field morphological similarity rating, intensity deviation rating, offset distance rating, timeliness rating, public impact rating, and overall quality rating.

[0185] In this step, a comprehensive assessment report centrally presents all dimension scores (precipitation field morphology similarity, intensity deviation, offset distance, timeliness, public impact, and overall quality), allowing decision-makers to quickly obtain a complete assessment profile within a single document. It supports cross-dimensional analysis; for example, a "high structural score but low public impact" indicates a problem with the warning dissemination channel, rather than the forecasting technology itself. This avoids the situation in traditional meteorological warning assessments where scores for various dimensions (such as structure, timeliness, and public impact) are scattered across different reports, forcing decision-makers to search through multiple documents or systems, resulting in inefficiency and the potential for overlooking key indicators. For instance, a typhoon warning might have a precipitation field morphology similarity score of B (85 points) but a public impact score of only D (60 points). Without integrated analysis, focus might be placed on the high-scoring item while ignoring the low-scoring weakness.

[0186] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0187] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0188] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for constructing an effectiveness evaluation model based on meteorological early warning service quality, characterized in that, include: Step 1: Obtain forecast precipitation intensity data of the precipitation field through the numerical forecast model of the meteorological forecast system, obtain observed precipitation intensity data of the precipitation field through meteorological observation stations, and calculate the structural dimension index of the precipitation field by analyzing the forecast precipitation intensity data and observed precipitation intensity data. Step 2: Evaluate the structural dimension indicators using a grading model to obtain the evaluation grades of different indicators within different structural dimension indicators, and calculate the comprehensive score of the structural dimension using different structural dimension indicators and their corresponding evaluation grades. Step 3: Obtain forecast time data for extreme weather events through the meteorological early warning system and actual time data through meteorological observation stations; obtain a timeliness score by analyzing the forecast and actual time data. Step 4: Obtain the warning level data of the precipitation field through the meteorological warning system, obtain the actual location data of the precipitation field through the meteorological observation station, obtain the reception data of mobile phone users in urban areas and mobile phone users in remote areas, analyze the data in Step 4 to obtain the public impact score, and determine the public impact score level. Step 5: Based on the comprehensive analysis of the structural dimension comprehensive score, the timeliness comprehensive score, and the public impact score, a comprehensive quality score is obtained; and the quality level of the weather forecast system is determined based on the comprehensive quality score. Step Six: Compile all the assessment results from Steps One to Five into a comprehensive assessment report.

2. The method for constructing an effectiveness evaluation model based on meteorological early warning service quality according to claim 1, characterized in that, In step one, the structural dimension indicators include precipitation field morphology similarity value, precipitation intensity deviation value, and offset distance value.

3. The method for constructing an effectiveness evaluation model based on meteorological early warning service quality according to claim 2, characterized in that, In step one, the precipitation field is first divided into grids, and the coordinates of different grids are defined as P(i,j). Then, a data analysis model is constructed to analyze the predicted precipitation intensity data and observed precipitation intensity data, and to calculate the mean of predicted precipitation intensity, the mean of observed precipitation intensity, the standard deviation of predicted precipitation intensity, the standard deviation of observed precipitation intensity, and the covariance of precipitation intensity. The formulas used are as follows: Among them, P f (i,j) represents the predicted precipitation intensity at different grid points (i,j); P o (i,j) represents the observed precipitation intensity at different grid points (i,j); μ x μ represents the mean of the predicted precipitation intensity at different grid points. y σ represents the mean of observed precipitation intensity at different grid points; x σ represents the standard deviation of the predicted precipitation intensity at different grid points. y σ represents the standard deviation of observed precipitation intensity at different grid points. xy This represents the covariance between the predicted precipitation intensity and the observed precipitation intensity; M and N represent the number of grid points in the horizontal and vertical directions of the precipitation field; H represents the total number of grids, and H = M * N.

4. The method for constructing an effectiveness evaluation model based on meteorological early warning service quality according to claim 3, characterized in that, By constructing a precipitation field morphology similarity analysis model, the mean of predicted precipitation intensity, the mean of observed precipitation intensity, the standard deviation of predicted precipitation intensity, the standard deviation of observed precipitation intensity, and the covariance of precipitation intensity are analyzed to obtain the precipitation field morphology similarity of precipitation intensity data. The calculation principle of the model is as follows: Wherein, SSIM(x,y) represents the similarity value of the precipitation field morphology between the predicted precipitation intensity data and the observed precipitation intensity data; C1 and C2 represent stability constant one and stability constant two, respectively; The precipitation intensity analysis model is used to analyze forecasted precipitation intensity data and observed precipitation intensity data to calculate the precipitation intensity deviation. The formula used is as follows: Wherein, RMSE represents the precipitation intensity deviation value; By constructing a correlation analysis model to analyze forecasted precipitation intensity data and observed precipitation intensity data, the cross-correlation value between the forecasted precipitation intensity data and observed precipitation intensity data is calculated. The formula used is as follows: XG(dx,dy)=∑ i,j P f (i,j)×P o (i+dx,j+dy); Among them, XG(dx,dy) represents the cross-correlation value between the predicted precipitation intensity data and the observed precipitation intensity data, where dx and dy represent the lateral and longitudinal offsets of the grid respectively; Traverse the cross-correlation values with different offsets, find the maximum value among the cross-correlation values, and the offset (dx,dy) corresponding to the maximum value among all cross-correlation values is the offset between the precipitation intensity data and the observed precipitation intensity data; According to the offset (dx,dy) corresponding to the maximum value among all cross-correlation values, calculate the offset distance value, and the formula is as follows: Among them, JLZ represents the offset distance value.

5. The method for constructing an effectiveness evaluation model based on meteorological early warning service quality according to claim 4, characterized in that, In step two, the evaluation grades corresponding to different structural dimension indicators are as follows: The evaluation grade of the similarity of precipitation field morphology is: 1≥SSIM(x,y)≥0.9: Rated as grade A precipitation field morphology similarity score, with a score of 90-100 points; 0.8≤SSIM(x,y)<0.9: Rated as grade B precipitation field morphology similarity score, with a score of 80-89 points; 0.7≤SSIM(x,y)<0.8: Rated as grade C precipitation field morphology similarity score, with a score of 70-79 points; SSIM(x,y)<0.7: Rated as grade D precipitation field morphology similarity score, with a score of 60 points; The evaluation grade of intensity deviation is: 0<RMSE≤0.1: Rated as grade A intensity deviation score, with a score of 90-100 points; 0.1<RMSE≤0.15: Rated as grade B intensity deviation score, with a score of 80-89 points; 0.15<RMSE≤0.2: Rated as grade C intensity deviation score, with a score of 70-79 points; RMSE>0.2: Rated as grade D intensity deviation score, with a score of 60 points; The evaluation grade of offset distance is: 0<JLZ≤5km: Rated as grade A offset distance score, with a score of 90-100 points; 5km<JLZ≤10km: Rated as grade B offset distance score, with a score of 80-89 points; 10km<JLZ≤15km: Rated as grade C offset distance score, with a score of 70-79 points; JLZ>15km: Rated as grade D offset distance, with a score of 60 points.

6. The method for constructing an effectiveness evaluation model based on meteorological early warning service quality according to claim 5, characterized in that, In step two, the calculation method of the comprehensive score of the structural dimension is as follows: Among them, PF k This represents the specific score for different indicator items, where k represents the index of the different indicator item, taking values ​​of 1, 2, or 3, corresponding to the precipitation field morphology similarity value, precipitation intensity deviation value, and offset distance value, respectively; SC k S1 represents the value of the k-th indicator item; S2 represents the lower limit of the level, S3 represents the upper limit of the level below, S4 represents the upper limit of the score of the level, S5 represents the upper limit of the score of the level below, and S6 represents the lower limit of the score of the level. By analyzing the specific scores of different indicators, calculate the comprehensive score of the structural dimension, and the formula is as follows: Where ZH represents the overall score of the structural dimension; α k This represents the weighting coefficient for specific scores of different indicators.

7. The method for constructing an effectiveness evaluation model based on meteorological early warning service quality according to claim 6, characterized in that, In step three: By constructing a timeliness analysis model; analyze the forecast occurrence time data and the actual occurrence time data of typhoon weather, rainstorm weather and snowstorm weather in the meteorological warning system, and calculate the comprehensive timeliness score, and the formula is as follows: TQL represents the overall timeliness score; TY R This indicates the predicted time of the Rth typhoon; TS R DY represents the actual time of the Rth typhoon; R represents the typhoon number, which is a positive integer; p DS represents the predicted time of the p-th rainstorm. p This represents the actual time of the p-th rainstorm, where p represents the sequence number of the rainstorm and takes a positive integer value; BY m This indicates the predicted time of the m-th blizzard, BS. m Let represent the actual occurrence time of the m-th blizzard, where m represents the sequence number of the blizzard and takes a positive integer value; β1 is the weighting coefficient of the time lead fraction for typhoon weather; β2 is the weighting coefficient of the time lead fraction for rainstorm weather; β3 is the weighting coefficient of the time lead fraction for blizzard weather, and β1 + β2 + β3 = 1; T h D h and B h These represent the golden lead time for forecasting typhoon weather, heavy rain weather, and heavy snow weather, respectively.

8. The method for constructing an effectiveness evaluation model based on meteorological early warning service quality according to claim 7, characterized in that, In step four: Set a sliding window and a warning threshold, slide the hit window in the grid map of the precipitation field, compare the warning value of each grid point in the hit window with the warning threshold. If at least 30% of the grid points in the area covered by the hit window exceed the warning threshold, then this covered area is defined as a hit window. Slide the sliding window to cover all positions of the precipitation field, and count the number of all hit windows as the number of hit windows; The actual disaster windows are counted from the actual location data. A window coverage calculation model is constructed, and the window coverage is calculated using this model. The method used is as follows: Where MZ represents window coverage, CK Y CK indicates the number of windows hit. S Indicates the actual number of disaster windows; By constructing a signal coverage calculation model, the received data of mobile phone users in urban areas and remote areas are input into the model to obtain the signal coverage. The calculation principle of the model is as follows: Where FG represents signal coverage; CK J This indicates the number of mobile phone users in the urban area who received the warning information; CK Z This represents the total number of mobile phone users in the urban area; PY J This indicates the number of mobile phone users in remote areas who received the warning message; PY Z γ1 represents the number of mobile phone users in remote areas who received the warning information; γ2 represents the warning information reception rate in urban areas; and γ1+γ2=1.

9. The method for constructing an effectiveness evaluation model based on meteorological early warning service quality according to claim 8, characterized in that, In step four: By analyzing window coverage and signal coverage, a public impact score is obtained, based on the following formula: Among them, GZ represents the public influence score; Weighting coefficients representing window coverage; The weighting coefficients representing signal coverage, and 10. The method for constructing an effectiveness evaluation model based on meteorological early warning service quality according to claim 9, characterized in that, In step five, a comprehensive quality analysis model is constructed. The structural dimension comprehensive score, the timeliness comprehensive score, and the public impact score are input into the model to calculate the comprehensive quality score. The calculation principle of the model is as follows: ZL=μ1×ZH+μ2×TQL+μ3×GZ; Where ZL represents the overall quality score; μ1 is the weight coefficient of the overall score for structural dimensions; μ2 is the weight coefficient of the overall score for timeliness; and μ3 is the weight coefficient of the score for public impact, μ1+μ2+μ3=1.

Images