Method and device for detecting weather forecast accuracy by fitting high error interval of expression

Abstract

The present application belongs to the technical field of weather prediction, and particularly relates to a weather forecast accuracy detection method and device for fitting high error interval of expression, which comprises the following steps: extracting a floating point expression from a weather forecast application program, parsing the floating point expression into an abstract syntax tree, and generating an equivalent MPFR high-precision version program as a test oracle by using the abstract syntax tree; performing error detection on a starting interval of the floating point expression to obtain an error distribution graph of the floating point expression in the interval; performing image processing on the error distribution graph; performing DBSCAN clustering on the processed error distribution graph and fitting upper and lower bound polynomial functions; deriving more high error intervals based on the polynomial functions; performing error detection on each of the derived high error intervals to obtain a final maximum error value; and judging the accuracy of weather forecast prediction according to the obtained maximum error value. The present application abstracts the weather forecast application program into a floating point expression for error detection, thereby improving the detection efficiency.

Description

Technical Field

[0001] This invention belongs to the field of weather forecasting technology, and specifically relates to a method and apparatus for detecting the accuracy of weather forecasts in the high error range of a fitted expression. Background Technology

[0002] Predicting weather and climate is one of the most complex problems in the world. Numerical modeling methods achieved success starting in the 1950s. However, over the past 60 years, the required spatiotemporal accuracy for weather forecasts has increased dramatically. Numerical models have evolved from simple forward equations to some of the most complex software projects in the world, requiring massive amounts of data and millions of lines of code supported by the powerful computing capabilities of supercomputers. Because weather forecasting involves extensive floating-point calculations, errors are unavoidable, and their magnitude directly impacts the accuracy of forecasts. Therefore, error detection is a crucial task in verifying the accuracy of weather forecasts. However, analyzing floating-point applications as a whole is difficult and inefficient.

[0003] Currently popular methods for detecting floating-point computational errors include static detection and dynamic detection. Static detection relies on theories such as abstract interpretation, interval arithmetic, affine arithmetic, and symbolic execution to attempt to find the upper bound of floating-point program errors. Dynamic detection utilizes tools to generate test predictions (precise values ​​of floating-point programs). Test predictions are generally obtained using high-precision floating-point types, and the floating-point program error can be obtained by comparing the actual calculated value with the precise value of the test prediction. Dynamic analysis is essentially based on search methods to find test cases that trigger large errors in floating-point programs. This can involve searching for the range of intervals that trigger errors or searching for specific inputs that trigger the maximum error. Existing technologies have some drawbacks. For example, the results of static detection are often higher than the error limits of actual floating-point computation programs. Dynamic detection uses a search strategy to detect the maximum error. The first search strategy is not yet well-established; for example, random search may miss test cases that trigger the maximum error, binary heuristic search has high time overhead, and the second search process cannot be reused. Once the detection interval changes, the search process needs to be re-executed. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention proposes a method and apparatus for detecting the accuracy of weather forecasts by fitting expressions with high error intervals. It abstracts the numerous floating-point operations in weather forecast applications into floating-point expressions for error detection, enabling rapid and effective verification of weather forecast accuracy. Furthermore, it performs DBSCAN clustering on the error distribution map of the floating-point expressions and fits high error intervals, thereby improving detection efficiency.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] This invention provides a method for detecting the accuracy of weather forecasts in the high error range of fitted expressions, comprising the following steps:

[0007] Floating-point expressions are extracted from the weather forecast application, parsed into abstract syntax trees, and used to generate an equivalent MPFR high-precision version of the program as a test oracle. The test oracle, i.e. the precise value, is obtained for each test case and used for error detection.

[0008] Error detection is performed on the starting interval of the floating-point expression to obtain the error distribution map of the floating-point expression in that interval;

[0009] Perform image processing on the error distribution map;

[0010] DBSCAN clustering was performed on the processed error distribution map, and upper and lower bound polynomial functions were fitted.

[0011] More high error intervals can be derived based on polynomial functions;

[0012] Error detection is performed on each derived high error interval to obtain the final maximum error value;

[0013] The accuracy of weather forecasts is determined by the maximum error value obtained.

[0014] Furthermore, the process of parsing a floating-point expression into an abstract syntax tree involves parsing the floating-point expression into a syntax tree through lexical analysis and syntax analysis. The equivalent MPFR high-precision version program replaces all variables, arithmetic operations, and function operations in the syntax tree nodes with equivalent MPFR operations.

[0015] Furthermore, error detection is performed on the initial interval of the floating-point expression to obtain the error distribution map of the floating-point expression in this interval. This includes: performing large-scale sampling detection on a certain initial interval of the detection interval, with a sampling scale of m. The test case generation method is uniform sampling, that is, uniformly generating m sampling points in the initial interval. An error distribution map is plotted on the error values ​​of the m test cases. The horizontal axis of the error distribution map represents the test case, and the vertical axis represents the error value under the test case.

[0016] Furthermore, the error value of the test case is calculated using the minimum precision unit ULP and the calculated and precise values ​​of the floating-point expression under that test case.

[0017] Furthermore, the method for image processing of the error distribution map is to take the first p points with the largest error values ​​for every n test cases and retain them as error data, while removing the remaining np points with low error values.

[0018] Furthermore, DBSCAN clustering divides regions with sufficient density into clusters, obtains the maximum error point of each cluster, and fits a polynomial function to these maximum error points.

[0019] Furthermore, the maximum error value is used to determine the accuracy of weather forecasts. The larger the maximum error value, the greater the forecast deviation and the less accurate the forecast. Conversely, the smaller the maximum error value, the smaller the forecast deviation and the more accurate the forecast.

[0020] The present invention also provides a weather forecast accuracy detection device for high error range of fitted expression, comprising:

[0021] The MPFR high-precision version program generation module is used to extract floating-point expressions from weather forecast applications, parse the floating-point expressions into abstract syntax trees, and use the abstract syntax trees to generate equivalent MPFR high-precision version programs as test oracles to obtain test oracles, i.e., precise values ​​for each test case for error detection.

[0022] The error distribution map construction module is used to perform error detection on the starting interval of a floating-point expression and obtain the error distribution map of the floating-point expression in that interval.

[0023] The image processing module is used to perform image processing on the error distribution map;

[0024] The polynomial function fitting module is used to perform DBSCAN clustering on the processed error distribution map and fit upper and lower bound polynomial functions.

[0025] The high error interval derivation module is used to derive more high error intervals based on polynomial functions.

[0026] The maximum error value acquisition module is used to perform error detection on each derived high error interval to obtain the final maximum error value.

[0027] The judgment module is used to determine the accuracy of weather forecasts based on the maximum error value obtained.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] 1. This invention extracts floating-point expressions from weather forecast applications, transforming the verification of weather forecast accuracy into the problem of error detection of floating-point expressions. Compared with existing methods that analyze the entire floating-point application, this improves the efficiency of weather forecast accuracy verification.

[0030] 2. This invention uses two polynomial functions, one upper and one lower bound, fitted to a floating-point expression to predict high-error intervals. Error detection is performed on each high-error interval to obtain the maximum error value. Here, error detection is only performed on the predicted intervals, rather than the entire interval, which greatly reduces the detection range, reduces the prediction time overhead, and improves detection efficiency.

[0031] 3. This invention treats floating-point expression error detection as an interval reasoning problem rather than a heuristic search problem, thus overcoming the limitations of search algorithms in dynamic detection of floating-point programs. This includes overcoming problems such as high time overhead and missing the maximum error, thereby increasing the accuracy of weather forecasts.

[0032] 4. Once a polynomial function is fitted to the floating-point expression extracted from the weather forecast application, the present invention can reuse the polynomial function for different detection intervals of the same floating-point expression, solving the problem that traditional search strategies can only re-search once the detection interval changes. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart illustrating the weather forecast accuracy detection method in the high error range of the fitting expression according to Embodiment 1 of the present invention.

[0035] Figure 2 This is a structural block diagram of the weather forecast accuracy detection device in the high error range of the fitting expression according to Embodiment 1 of the present invention;

[0036] Figure 3 This is a schematic diagram of the program for generating a high-precision MPFR version according to Embodiment 2 of the present invention;

[0037] Figure 4 This is a schematic diagram of the error distribution diagram of Embodiment 2 of the present invention;

[0038] Figure 5 This is a schematic diagram of the error distribution after image processing in Embodiment 2 of the present invention;

[0039] Figure 6 This is a schematic diagram of DBSCAN clustering of the processed error distribution map according to Embodiment 2 of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Example 1

[0042] like Figure 1 As shown, the weather forecast accuracy detection method for the high error range of the fitted expression in this embodiment includes the following steps:

[0043] Step S11: Extract floating-point expressions from the weather forecast application, parse the floating-point expressions into an abstract syntax tree (AST), and use the abstract syntax tree (AST) to generate an equivalent MPFR high-precision version of the program as a test oracle. The test oracle, i.e., the precise value, can be obtained for each test case for error detection.

[0044] Specifically, parsing a floating-point expression into an abstract syntax tree involves lexical and syntactic analysis to parse the floating-point expression into a syntax tree; generating an equivalent MPFR high-precision version program replaces all variables, arithmetic operations, and function operations in the syntax tree nodes with equivalent MPFR operations.

[0045] Step S12: Perform error detection on the starting interval of the floating-point expression to obtain the error distribution map of the floating-point expression in the interval.

[0046] In this example, a large-scale sampling test is performed on a small initial interval of the detection range, with a sampling scale of m. The test case generation method is uniform sampling, that is, m sampling points are uniformly generated within the initial small interval. An error distribution graph is plotted for the error values ​​of the m test cases. The horizontal axis of the error distribution graph represents the test cases, and the vertical axis represents the error value under that test case. The error value of the test case can be calculated using the minimum precision unit ULP and the calculated and precise values ​​of floating-point expressions under that test case.

[0047] Step S13: Perform image processing on the error distribution map.

[0048] An alternative image processing method is to take the top p points with the largest error values ​​for every n test cases and retain them as error data, while removing the remaining np low error points. The purpose is to remove low error points that are useless for clustering and fitting, thereby reducing the clustering time overhead.

[0049] Step S14: Perform DBSCAN clustering on the processed error distribution map and fit two polynomial functions for upper and lower bounds, which are used to derive the upper and lower bounds of the subsequent high error interval.

[0050] DBSCAN clustering is a noisy density-based spatial clustering method that divides regions with sufficient density into clusters, obtains the maximum error point of each cluster, and fits a polynomial function to these maximum error points.

[0051] Step S15: Derive more high error intervals based on the polynomial function.

[0052] Step S16: Perform error detection on each derived high error interval to obtain the final maximum error value.

[0053] The error detection method for the derived high error interval is the same as the error detection method for the initial small interval in step S12. Large-scale sampling detection is performed on each derived high error interval, and the test case generation method is also uniform sampling, that is, several test cases are generated uniformly within the interval.

[0054] Step S17: Determine the accuracy of the weather forecast based on the maximum error value obtained in step S16.

[0055] The maximum error value is used to judge the accuracy of weather forecasts. The larger the maximum error value, the greater the forecast deviation and the less accurate the forecast. The smaller the maximum error value, the smaller the forecast deviation and the more accurate the forecast.

[0056] Corresponding to the above-mentioned method for detecting the accuracy of weather forecasts in the high error range of a fitted expression, such as... Figure 2 As shown, this embodiment also provides a weather forecast accuracy detection device for high error range of fitted expression, including MPFR high-precision version program generation module, error distribution map construction module, image processing module, polynomial function fitting module, high error range derivation module, maximum error value acquisition module and judgment module.

[0057] The MPFR high-precision version program generation module is used to extract floating-point expressions from weather forecast applications, parse the floating-point expressions into abstract syntax trees, and use the abstract syntax trees to generate equivalent MPFR high-precision version programs as test oracles to obtain test oracles, i.e., precise values ​​for each test case, for error detection.

[0058] The error distribution map construction module is used to perform error detection on the starting interval of a floating-point expression and obtain the error distribution map of the floating-point expression in that interval.

[0059] The image processing module is used to perform image processing on the error distribution map.

[0060] The polynomial function fitting module is used to perform DBSCAN clustering on the processed error distribution map and fit upper and lower bound polynomial functions.

[0061] The high error interval derivation module is used to derive more high error intervals based on polynomial functions.

[0062] The maximum error value acquisition module is used to perform error detection on each derived high error interval to obtain the final maximum error value.

[0063] The judgment module is used to determine the accuracy of weather forecasts based on the maximum error value obtained.

[0064] The following example further explains how the present invention detects the accuracy of weather forecasts, illustrating the specific implementation method and effects of the present invention.

[0065] Example 2

[0066] Suppose we extract a floating-point expression from a weather forecast application. Assume the detection interval is [0, 100000].

[0067] In the field of floating-point arithmetic, ULP error is commonly used as an error metric. For a floating-point number expression z = (-1) s ×d0d1…d n ×2 E And its actual value, the smallest unit of precision ULP, and ULP error. ulp This can be represented as:

[0068]

[0069]

[0070] Where f(x0) represents the calculated value of the floating-point expression f(x) extracted from the weather forecast application at x = x0 (i.e., the floating-point calculation result), o((fx0)) represents the exact value of the floating-point expression f(x) at x = x0 (i.e., the real number calculation result); s(sign) is the sign bit, d0d1...d n The mantissa is the number of significant digits of z, n is the length of the significant digits, which is the precision of the floating-point number, and E (exponent) is the exponent.

[0071] Step S21: Extract floating-point expressions from the weather forecast application, parse the floating-point expressions into an abstract syntax tree, and replace each node of the abstract syntax tree with an MPFR equivalent high-precision version to generate an MPFR high-precision version program, such as... Figure 3 As shown, the MPFR custom precision bit depth is set to 128 bits.

[0072] Step S23: Perform error detection on the initial interval. The interval size is selected as [0, 100], the sampling scale is selected as 500,000, and the test case generation method is selected as uniform sampling, that is, uniformly generate 500,000 sampling points in the interval [0, 100]. Use the MPFR high-precision version program as the test oracle to obtain the test oracle under each test case as the accurate value. Substitute the accurate value and the calculated value into formula (1) and formula (2) to obtain the ULP error under each test case. Plot the error distribution map of the error values ​​of 500,000 test cases, as shown below. Figure 4 As shown, the horizontal axis represents the test cases, and the vertical axis represents the error value under that test case.

[0073] Step S23: Perform image processing on the error distribution map obtained in step S22. The processing method is to take the first p points with the largest error values ​​for every n test cases. Specifically, the parameters can be selected as taking the first point with the largest error value for every 500 points, and finally retaining 1000 error values ​​to form an error dataset. Figure 5 As shown.

[0074] Step S24: The image processing error dataset obtained in step S23 is subjected to DBSCAN clustering and multinomial function fitting. After DBSCAN clustering, it is divided into 3 clusters, such as... Figure 6 As shown, the maximum error points for each cluster are obtained as (4.75933, 1.96755), (16.106, 2.24603), and (64.0682, 2.35632). A polynomial function is fitted to these three points using a geometric series x. i =x1×q i-1 For i≥1, the fitting function first calculates a lower bound x. i -1 and the upper bound x i +1, forming the range [x i -1,x i +1] Covers each x i It's important to note that when x1 < 2, the lower bound is instantiated from x1 itself. This is to avoid the initial term of the geometric series being very close to or equal to 0, thus avoiding violating the definition of a geometric series. Finally, the functions best suited for the lower and upper bounds of each cell are fitted. The lower bound fitting points are (1, 3.75933), (2, 15.106), and (3, 63.0682), and the upper bound fitting points are (1, 5.75933), (2, 17.106), and (3, 65.0682). The two fitted functions are x1 = 3.65 × 4.16. i-1 x² = 4.79 × 3.69 i-1 ,(i≥1).

[0075] Step S25: Extrapolate the two polynomial functions fitted in step S24 to predict the high error intervals that will appear later. The predicted intervals are [240.666, 262.768], [888.057, 1093.12], [3276.93, 4547.36], [12091.9, 18917], [44619, 78694.8].

[0076] Step S26: Perform error detection on all predicted intervals, combine the error values ​​within the [0,100] interval, and obtain the maximum error value. The test case with the maximum error value is x = 66068.6256743999983882, with a maximum error value of 2.45ULP. For floating-point errors less than 0.5ULP, it is generally considered that there is no error. The error result of 2.45ULP indicates that the weather forecast application has an error under the test case x = 66068.6256743999983882, which will reduce the accuracy of weather forecasts. We can analyze and study how to reduce the error under this test case to improve the accuracy of weather forecasts.

[0077] By comparing with three of the most advanced and widely used dynamic error detection tools currently known—Herbie, Atomu, and S3fp—Herbie detects errors through random sampling, Atomu is based on atomic state function search, and S3fp is based on binary heuristic search. In this example, the maximum error value obtained for this floating-point expression in the interval [0, 100000] is 2.45ULP. Since Herbie outputs the error in bits, the formula for converting ULP error to bit error is log2(ULP). 2.45ULP is converted to a bit error of 1.8, while Herbie's result is 1.6, showing better performance than Herbie. The error reported by S3fp is a relative error. For the test case where the maximum error value obtained for this floating-point expression is x = 66068.6256743999983882, the relative error under this test case is calculated to be 3.05E-16, while S3fp reports a result of 2.89E-16, showing better performance than S3fp. Atomu failed to find a maximum error value in this interval, meaning it did not report any error results. Clearly, Atomu's detection performance is superior. Therefore, the weather forecast accuracy detection method based on clustering and fitting expressions for high error intervals can achieve significant detection results, and the accuracy of weather forecasts can be judged based on the maximum error value.

[0078] It should be noted that, in this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0079] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is only used to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for detecting the accuracy of weather forecasts within the high error range of a fitted expression, characterized in that, Includes the following steps: Floating-point expressions are extracted from the weather forecast application, parsed into abstract syntax trees, and used to generate an equivalent MPFR high-precision version of the program as a test oracle. The test oracle, i.e. the precise value, is obtained for each test case and used for error detection. Error detection is performed on the starting interval of the floating-point expression to obtain the error distribution map of the floating-point expression in that interval; Perform image processing on the error distribution map; DBSCAN clustering was performed on the processed error distribution map, and upper and lower bound polynomial functions were fitted. More high error intervals can be derived based on polynomial functions; Error detection is performed on each derived high error interval to obtain the final maximum error value; The accuracy of weather forecasts is determined by the maximum error value obtained.

2. The method for detecting the accuracy of weather forecasts in the high error range of the fitted expression according to claim 1, characterized in that, The process of parsing a floating-point expression into an abstract syntax tree involves lexical and syntactic analysis to parse the floating-point expression into a syntax tree. The equivalent MPFR high-precision version program replaces all variables, arithmetic operations, and function operations in the syntax tree nodes with equivalent MPFR operations.

3. The method for detecting the accuracy of weather forecasts in the high error range of the fitted expression according to claim 1, characterized in that, Error detection is performed on the initial interval of the floating-point expression to obtain the error distribution map of the floating-point expression in the interval. This includes: performing large-scale sampling detection on a certain initial interval of the detection interval, with a sampling scale of m. The test case generation method is uniform sampling, that is, uniformly generating m sampling points in the initial interval. An error distribution map is plotted on the error values ​​of the m test cases. The horizontal axis of the error distribution map represents the test case, and the vertical axis represents the error value under the test case.

4. The method for detecting the accuracy of weather forecasts in the high error range of the fitted expression according to claim 3, characterized in that, The error value of the test case is calculated using the minimum precision unit ULP and the calculated and precise values ​​of floating-point expressions under that test case.

5. The method for detecting the accuracy of weather forecasts in the high error range of the fitted expression according to claim 1, characterized in that, The method for image processing of the error distribution map is to take the first p points with the largest error values ​​for every n test cases and retain them as error data, while removing the remaining np points with low error values.

6. The method for detecting the accuracy of weather forecasts in the high error range of the fitted expression according to claim 1, characterized in that, DBSCAN clustering divides regions with sufficient density into clusters, obtains the maximum error point in each cluster, and fits a polynomial function to these maximum error points.

7. The method for detecting the accuracy of weather forecasts in the high error range of the fitted expression according to claim 1, characterized in that, The maximum error value is used to judge the accuracy of weather forecasts. The larger the maximum error value, the greater the forecast deviation and the less accurate the forecast. The smaller the maximum error value, the smaller the forecast deviation and the more accurate the forecast.

8. A weather forecast accuracy detection device for high error range of fitted expression, characterized in that, include: The MPFR high-precision version program generation module is used to extract floating-point expressions from weather forecast applications, parse the floating-point expressions into abstract syntax trees, and use the abstract syntax trees to generate equivalent MPFR high-precision version programs as test oracles to obtain test oracles, i.e., precise values ​​for each test case for error detection. The error distribution map construction module is used to perform error detection on the starting interval of a floating-point expression and obtain the error distribution map of the floating-point expression in that interval. The image processing module is used to perform image processing on the error distribution map; The polynomial function fitting module is used to perform DBSCAN clustering on the processed error distribution map and fit upper and lower bound polynomial functions. The high error interval derivation module is used to derive more high error intervals based on polynomial functions. The maximum error value acquisition module is used to perform error detection on each derived high error interval to obtain the final maximum error value. The judgment module is used to determine the accuracy of weather forecasts based on the maximum error value obtained.

Images