Abnormal value determination method and device, storage medium and electronic device

Abstract

The invention discloses an abnormal value determination method and device, a storage medium and an electronic device, and the method comprises the steps: determining whether meteorological observation data collected within a preset time period are not detected or not; under the condition that the meteorological observation data is not detected, calling a first detection model to detect the meteorological observation data so as to determine an abnormal value in the meteorological observation data, the first detection model comprises at least one of a second detection model used for detecting the time correlation of the meteorological observation data and a third detection model used for detecting the element correlation of the meteorological observation data, and the element correlation at least comprises the space correlation. The problem that the data quality control level of the collected meteorological data is poor is solved, and the data quality control level of the meteorological data is improved.

Description

Technical Field

[0001] The present application relates to the technical field of meteorological observation, and in particular to a method and device for determining an abnormal value, a storage medium, and an electronic device. Background Art

[0002] Meteorological observation is one of the most important and basic tasks in the field of meteorological research. It is the data basis for weather forecasting and climate analysis, and is also an important standard for weather forecast verification. Therefore, the quality of meteorological observation data directly affects the effect of data application. The quality control of meteorological data refers to the integration and analysis of the observed meteorological observation data, checking the data quality and recording it, so that problems in the data can be discovered in time and solved quickly. In addition, since errors and deviations may occur in a series of links in the collection, transmission, encoding and decoding of meteorological data, the quality control of meteorological data is particularly important. It not only directly affects the credibility of meteorological data, but also has vital significance for meteorological forecasts and meteorological services.

[0003] In the relevant technology, standard data quality control includes missing test, range test, continuity test and comprehensive analysis test, etc. Traditional meteorological data quality control uses simple statistical detection methods such as climate extremes and numerical change rates, which are widely used, but the detection methods are relatively rough, resulting in poor data quality control of the collected meteorological data.

[0004] There is no effective solution to the problem of poor data quality control of collected meteorological data in related technologies. Summary of the invention

[0005] The embodiments of the present application provide a method and device for determining an abnormal value, a storage medium, and an electronic device to at least solve the problem of poor data quality control level of collected meteorological data in the related art.

[0006] According to an embodiment of the embodiments of the present application, a method for determining an outlier is provided, including: determining whether meteorological observation data collected within a preset time length is missing; in the case that the meteorological observation data is missing, calling a first detection model to detect the meteorological observation data to determine the outlier in the meteorological observation data, wherein the first detection model includes at least one of the following: a second detection model for detecting the time correlation of the meteorological observation data, a third detection model for detecting the element correlation of the meteorological observation data, wherein the element correlation includes at least: spatial correlation.

[0007] In an exemplary embodiment, before determining whether the meteorological observation data collected within a preset time period is missing, the method also includes: performing a range check on the meteorological observation data to eliminate data in the meteorological observation data that exceeds a threshold range; and / or performing a peak check on the meteorological observation data to eliminate peak values ​​in the meteorological observation data.

[0008] In an exemplary embodiment, calling the first detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data includes: determining the missing type of the meteorological observation data; when the missing type is the first type, determining to call the second detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data; when the missing type is the second type, determining to call the third detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data; when the missing type includes both the first type and the second type, determining to call the second detection model and the third detection model at the same time to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data.

[0009] In an exemplary embodiment, determining the missing type of the meteorological observation data includes: when a first meteorological element in the meteorological observation data is missing, determining that the missing type is the first type; and when a second meteorological element in the meteorological observation data is missing, determining that the missing type is the second type.

[0010] In an exemplary embodiment, when the meteorological observation data is missing, before calling the first detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data, the method also includes: determining the correlation between the third meteorological element and the fourth meteorological element by a grey correlation method, wherein the meteorological observation data includes: first meteorological observation data and second meteorological observation data, the first meteorological observation data is the meteorological observation data of the target site within a preset spatial range, the second meteorological observation data is the meteorological observation data of other sites except the target site within the preset spatial range, the third meteorological element is any meteorological element in the first meteorological observation data, and the fourth meteorological element includes: the observation values ​​corresponding to the third meteorological element at the previous n moments, where n is a positive integer; constructing the second detection model through the fifth meteorological element, wherein the fifth meteorological element is the meteorological element with the highest correlation with the third meteorological element among the fourth meteorological elements.

[0011] In an exemplary embodiment, when the meteorological observation data is missing, before calling the first detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data, the method also includes: determining the first correlation coefficient between the third meteorological element and the sixth meteorological element by the Spearman correlation coefficient, and determining the second correlation coefficient between the third meteorological element and the seventh meteorological element by the Spearman correlation coefficient; determining the third detection model by the first correlation coefficient and the second correlation coefficient; wherein the meteorological observation data includes: first meteorological observation data and second meteorological observation data, the first meteorological observation data is the meteorological observation data of the target site within a preset spatial range, the second meteorological observation data is the meteorological observation data of other sites within the preset spatial range except the target site, the third meteorological element is any meteorological element in the first meteorological observation data, the sixth meteorological element is other meteorological elements in the first meteorological observation data except the third meteorological element, and the seventh meteorological element is the meteorological element in the other sites that is the same as the third meteorological element.

[0012] According to another embodiment of the embodiments of the present application, a device for determining abnormal values ​​is also provided, including: a first determination module, used to determine whether meteorological observation data collected within a preset time length is missing; a second determination module, used to call a first detection model to detect the meteorological observation data when the meteorological observation data is missing, so as to determine the abnormal values ​​in the meteorological observation data, wherein the first detection model includes at least one of the following: a second detection model for detecting the time correlation of the meteorological observation data, and a third detection model for detecting the element correlation of the meteorological observation data, wherein the element correlation includes at least: spatial correlation.

[0013] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above method when running.

[0014] According to another aspect of the embodiments of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method through the computer program.

[0015] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0016] In an embodiment of the present application, it is determined whether the meteorological observation data collected within a preset time period is missing; in the case of missing meteorological observation data, a first detection model is called to detect the meteorological observation data to determine abnormal values ​​in the meteorological observation data, wherein the first detection model includes at least one of the following: a second detection model for detecting the time correlation of the meteorological observation data, a third detection model for detecting the element correlation of the meteorological observation data, wherein the element correlation includes at least: spatial correlation. Through the above embodiments, the problem of poor data quality control level of the collected meteorological data in the related art is solved, and the control level of the data quality of the meteorological data is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 It is a hardware structure block diagram of a computer terminal of a method for determining an abnormal value in an embodiment of the present application;

[0019] Figure 2 is a flow chart of a method for determining an outlier according to an embodiment of the present application;

[0020] Figure 3 is another flow chart of a method for determining an outlier according to an embodiment of the present application;

[0021] Figure 4 It is a structural block diagram of an abnormal value determination device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices; "plurality" means two or more.

[0024] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal or similar computing device or cloud platform or independent physical server or software platform, wherein the above software platform is run through one or more servers. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal for determining an abnormal value in an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data. In an exemplary embodiment, the computer terminal may also include a transmission device 106 for communication functions and an input and output device 108, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0025] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the methods in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, the above-mentioned methods are implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0026] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0027] In this embodiment, a method for determining an abnormal value is provided, which is applied to the above-mentioned computer terminal. Figure 2 : is a flow chart of a method for determining an outlier according to an embodiment of the present application, the process comprising the following steps:

[0028] Step S202, determining whether the meteorological observation data collected within a preset time period is missing;

[0029] Step S204, when the meteorological observation data is missing, call the first detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data, wherein the first detection model includes at least one of the following: a second detection model for detecting the time correlation of the meteorological observation data, and a third detection model for detecting the element correlation of the meteorological observation data, wherein the element correlation includes at least: spatial correlation.

[0030] Through the above steps, it is determined whether the meteorological observation data collected within the preset time period is missing; in the case of missing meteorological observation data, the first detection model is called to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data, wherein the first detection model includes at least one of the following: a second detection model for detecting the time correlation of the meteorological observation data, a third detection model for detecting the element correlation of the meteorological observation data, wherein the element correlation includes at least: spatial correlation. Through the above embodiments, the problem of poor data quality control level of the collected meteorological data in the related technology is solved, and the control level of the data quality of the meteorological data is improved.

[0031] In an exemplary embodiment, before determining whether the meteorological observation data collected within a preset time period is missing, the method also includes: performing a range check on the meteorological observation data to eliminate data in the meteorological observation data that exceeds a threshold range; and / or performing a peak check on the meteorological observation data to eliminate peak values ​​in the meteorological observation data.

[0032] Optionally, when performing range checking and peak checking at the same time, the order of the range checking and peak checking is not limited in the embodiment of the present application.

[0033] In an exemplary embodiment, calling the first detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data includes: determining the missing type of the meteorological observation data; when the missing type is the first type, determining to call the second detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data; when the missing type is the second type, determining to call the third detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data; when the missing type includes both the first type and the second type, determining to call the second detection model and the third detection model at the same time to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data.

[0034] Optionally, determining the missing type of the meteorological observation data includes: when a first meteorological element in the meteorological observation data is missing, determining that the missing type is the first type; and when a second meteorological element in the meteorological observation data is missing, determining that the missing type is the second type.

[0035] That is, when the target meteorological element is missing for a continuous period of time (equivalent to the preset duration) (equivalent to the second type), the element-related machine learning data quality control model (equivalent to the third detection model) is used to perform outlier detection; when other elements related to the target meteorological element are missing in large numbers at the same time (equivalent to the first type), the time-related deep learning data quality control model (equivalent to the second detection model) is used to perform outlier detection.

[0036] It should be noted that the target meteorological element in the above embodiment is equivalent to the second meteorological element, and other elements related to the target meteorological element are equivalent to the first meteorological element.

[0037] In some embodiments, when the meteorological observation data is missing, before calling the first detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data, the method also includes: determining the correlation between the third meteorological element and the fourth meteorological element by a grey correlation method, wherein the meteorological observation data includes: first meteorological observation data and second meteorological observation data, the first meteorological observation data is the meteorological observation data of the target site within a preset spatial range, the second meteorological observation data is the meteorological observation data of other sites except the target site within the preset spatial range, the third meteorological element is any meteorological element in the first meteorological observation data, and the fourth meteorological element includes: the observation values ​​corresponding to the third meteorological element at the previous n moments, where n is a positive integer; constructing the second detection model through the fifth meteorological element, wherein the fifth meteorological element is the meteorological element with the highest correlation with the third meteorological element among the fourth meteorological elements.

[0038] It should be noted that the third meteorological element is any meteorological element in the first meteorological observation data. Similarly, the target meteorological element is: any meteorological element of the first meteorological observation data in the meteorological observation data collected within a preset time period when the first detection model is put into use.

[0039] Furthermore, other elements related to the target meteorological element include: a fourth meteorological element in the meteorological observation data collected within a preset time period.

[0040] In an exemplary embodiment, when the meteorological observation data is missing, before calling the first detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data, the method also includes: determining the first correlation coefficient between the third meteorological element and the sixth meteorological element by the Spearman correlation coefficient, and determining the second correlation coefficient between the third meteorological element and the seventh meteorological element by the Spearman correlation coefficient; determining the third detection model by the first correlation coefficient and the second correlation coefficient; wherein the meteorological observation data includes: first meteorological observation data and second meteorological observation data, the first meteorological observation data is the meteorological observation data of the target site within a preset spatial range, the second meteorological observation data is the meteorological observation data of other sites within the preset spatial range except the target site, the third meteorological element is any meteorological element in the first meteorological observation data, the sixth meteorological element is other meteorological elements in the first meteorological observation data except the third meteorological element, and the seventh meteorological element is the meteorological element in the other sites that is the same as the third meteorological element.

[0041] Furthermore, other elements related to the target meteorological element include: a sixth meteorological element and a seventh meteorological element.

[0042] In order to better understand the process of the above-mentioned method for determining abnormal values, the above-mentioned method for determining abnormal values ​​is described below in combination with an optional embodiment, but is not used to limit the technical solution of the embodiment of the present application.

[0043] Standard data quality control includes missing test, range test, continuity test and comprehensive analysis test. Traditional meteorological data quality control uses simple statistical detection methods such as climate extremes and numerical change rates, which are widely used, but the detection methods are relatively rough. In related technologies, a large number of experts and scholars have proposed many improvement plans for data quality control, such as making more accurate judgments on the boundary values ​​and time consistency of meteorological observation values ​​based on historical data. However, most of the current improvement plans are relatively superficial in their use of the relationship between meteorological elements and the spatial relationship between stations. The mutual influence between meteorological elements and the spatial relationship between different stations will have a certain impact on the accuracy of quality control. This requires the use of updated algorithm models to explore the relationship between elements and thus improve the level of data quality control.

[0044] At the same time, with the development of artificial intelligence technology, machine learning has gradually been applied to the field of meteorological data quality control in recent years, such as BP neural network, support vector machine, K-means clustering and LightGBM and other integrated learning algorithms. Meteorological data quality control based on machine learning algorithms is suitable for processing large amounts of meteorological data, and can also quickly process multi-dimensional data and deeply explore the multi-factor relationship of meteorological data, so it has gradually become a research hotspot.

[0045] Based on the above content, the present application embodiment proposes a meteorological data quality control method that integrates multiple elements of time and space (equivalent to the method for determining abnormal values ​​in the above embodiment). Figure 3 As shown in the figure, first, outliers are removed based on range test and peak test to improve the quality of meteorological data; secondly, a time-related deep learning data quality control model is constructed, and outlier test is realized by considering the time correlation of meteorological elements; then, an element-related machine learning data quality control model is constructed, and outlier test is realized by considering the spatial correlation of meteorological elements and the correlation with other elements; finally, the quality control of meteorological elements is realized according to the comprehensive meteorological data quality control plan that integrates multiple models.

[0046] Specifically, Figure 3 As shown, the following steps are included:

[0047] Step 1: Data collection and range check: First, collect the meteorological observation data of the meteorological observation system of the target site and adjacent sites (equivalent to other sites); then, perform a range check on the meteorological elements of the target site and remove data that exceeds the threshold range to improve data quality. Optionally, a range check can also be performed on the collected meteorological observation data.

[0048] Step 2: Data peak test: After the meteorological observation data has been range tested, a peak test is performed. The principle of the peak test is that the changes of the observation elements within the space and time range are limited. If a certain observation value is significantly different from the surrounding observation values ​​and a peak appears, it is determined to be an outlier. The calculation formula is shown in formula (1).

[0049] x i -(x i+1 +x i+2 ) / 2|≤H j1 (1)

[0050] Where: H j1 is the peak test parameter, constant.

[0051] Step 3: Construct a time-related deep learning data quality control model. First, the grey correlation method is used to calculate the correlation between the target station meteorological element (equivalent to the third meteorological element) and the element of the station at the previous n moments (equivalent to the fourth meteorological element). Second, the meteorological element at the historical moment with a higher correlation is selected as the model input. Then, a meteorological data quality control model based on BiLSTM (i.e., the second detection model) is constructed. This model can better capture the temporal characteristics of meteorological elements, thereby improving the accuracy of quality control. Finally, the F1 value that comprehensively considers the precision and recall rates is used to select the threshold, and outliers are eliminated according to the threshold. For details, see formulas (2) to (4).

[0052]

[0053] Where: P is the precision rate; R is the recall rate; β is the weight parameter, which can be set according to needs.

[0054]

[0055] Where: TP is the number of true samples; FP is the number of false positive samples.

[0056]

[0057] Where: TP is the number of true samples; FN is the number of false negative samples.

[0058] Step 4: Construct a machine learning data quality control model related to elements. First, the Spearman correlation coefficient is used to calculate the correlation coefficient (i.e., the first correlation coefficient) between the meteorological elements of the target site and other meteorological elements of this site (equivalent to the sixth meteorological element); secondly, the Spearman correlation coefficient is used to calculate the correlation coefficient (i.e., the second correlation coefficient) between the meteorological elements of the target site and the meteorological elements of the neighboring sites (equivalent to the seventh meteorological element); then, the other meteorological elements of the target site (equivalent to the sixth meteorological element) with higher correlation coefficients and the meteorological element data of the neighboring sites are selected as the independent variable data set, and the data set is trained using the integrated learning XGBoost algorithm, and then the meteorological element data fitting model (equivalent to the third detection model) is obtained based on the weighted combination of multiple weak learners; finally, the F1 value is used to select the threshold, and outlier detection is performed after the threshold is confirmed.

[0059] Step 5: A comprehensive solution for meteorological data quality control that integrates multiple models. When the target meteorological element is missing for a period of time, the element-related machine learning data quality control model is used to detect outliers; when other elements related to the target meteorological element are missing in large numbers at the same time, the time-related deep learning data quality control model is used to detect outliers; when both models can be used, the quality control accuracy of the two models is compared, and the method with higher accuracy is selected.

[0060] In summary, the embodiment of the present application proposes a meteorological data quality control method that integrates multiple elements of time and space as a protection point, including a time-related deep learning data quality control model and an element-related machine learning data quality control model. Using the BiLSTM algorithm, a time-related deep learning data quality control model is constructed. The model can learn the temporal characteristics of meteorological elements, thereby effectively improving the accuracy of quality control; at the same time, the threshold for abnormal judgment is selected by comprehensively considering the recall rate and precision rate, which is more targeted and more flexible. Furthermore, using the XGBoost algorithm, an element-related machine learning data quality control model is constructed, which can fully consider the spatial characteristics of meteorological elements, thereby improving the accuracy of quality control; at the same time, a comprehensive solution for meteorological data quality control that integrates multiple models is also considered, which has stronger generalization and higher accuracy.

[0061] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0062] The present application also provides a structural block diagram of a device for determining an abnormal value. Figure 4 is a structural block diagram of an apparatus for determining an abnormal value according to an embodiment of the present application; Figure 4 As shown, including:

[0063] A first determination module 42 is used to determine whether the meteorological observation data collected within a preset time period is missing;

[0064] The second determination module 44 is used to call the first detection model to detect the meteorological observation data in the case that the meteorological observation data is missing, so as to determine the abnormal values ​​in the meteorological observation data, wherein the first detection model includes at least one of the following: a second detection model for detecting the time correlation of the meteorological observation data, and a third detection model for detecting the element correlation of the meteorological observation data, wherein the element correlation includes at least: spatial correlation.

[0065] Through the above device, it is determined whether the meteorological observation data collected within a preset time period is missing; in the case of missing meteorological observation data, the first detection model is called to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data, wherein the first detection model includes at least one of the following: a second detection model for detecting the time correlation of the meteorological observation data, a third detection model for detecting the element correlation of the meteorological observation data, wherein the element correlation includes at least: spatial correlation. Through the above embodiment, the problem of poor data quality control level of the collected meteorological data in the related technology is solved, and the control level of the data quality of the meteorological data is improved.

[0066] In an exemplary embodiment, the device also includes a preprocessing module for determining whether the meteorological observation data collected within a preset time period is missing: performing a range check on the meteorological observation data to eliminate data in the meteorological observation data that exceeds a threshold range; and / or performing a peak check on the meteorological observation data to eliminate spike values ​​in the meteorological observation data.

[0067] In an exemplary embodiment, the second determination module 44 is also used to: determine the missing type of the meteorological observation data; when the missing type is the first type, determine to call the second detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data; when the missing type is the second type, determine to call the third detection model to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data; when the missing type includes both the first type and the second type, determine to call the second detection model and the third detection model at the same time to detect the meteorological observation data to determine the abnormal values ​​in the meteorological observation data.

[0068] In an exemplary embodiment, the second determination module 44 is further used to: when the first meteorological element in the meteorological observation data is missing, determine that the missing type is the first type; when the second meteorological element in the meteorological observation data is missing, determine that the missing type is the second type.

[0069] In an exemplary embodiment, the device also includes a first construction module, which is used to call the first detection model to detect the meteorological observation data when the meteorological observation data is missing, so as to determine the abnormal values ​​in the meteorological observation data: determine the correlation between the third meteorological element and the fourth meteorological element by using the grey correlation method, wherein the meteorological observation data include: first meteorological observation data and second meteorological observation data, the first meteorological observation data is the meteorological observation data of the target site within a preset spatial range, the second meteorological observation data is the meteorological observation data of other sites except the target site within the preset spatial range, the third meteorological element is any meteorological element in the first meteorological observation data, and the fourth meteorological element includes: the observation values ​​corresponding to the third meteorological element at the previous n moments, where n is a positive integer; construct the second detection model through the fifth meteorological element, wherein the fifth meteorological element is the meteorological element with the highest correlation with the third meteorological element among the fourth meteorological elements.

[0070] In an exemplary embodiment, the first construction module is also used to call the first detection model to detect the meteorological observation data when the meteorological observation data is missing, so as to determine the abnormal values ​​in the meteorological observation data: determine the first correlation coefficient between the third meteorological element and the sixth meteorological element by the Spearman correlation coefficient, and determine the second correlation coefficient between the third meteorological element and the seventh meteorological element by the Spearman correlation coefficient; determine the third detection model by the first correlation coefficient and the second correlation coefficient; wherein the meteorological observation data include: first meteorological observation data and second meteorological observation data, the first meteorological observation data is the meteorological observation data of the target site within a preset spatial range, the second meteorological observation data is the meteorological observation data of other sites except the target site within the preset spatial range, the third meteorological element is any meteorological element in the first meteorological observation data, the sixth meteorological element is other meteorological elements in the first meteorological observation data except the third meteorological element, and the seventh meteorological element is the meteorological element in the other sites that is the same as the third meteorological element.

[0071] An embodiment of the present application further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0072] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0073] S1, determining whether the meteorological observation data collected within a preset time period is missing;

[0074] S2. When the meteorological observation data is missing, call the first detection model to detect the meteorological observation data to determine abnormal values ​​in the meteorological observation data, wherein the first detection model includes at least one of the following: a second detection model for detecting the time correlation of the meteorological observation data, and a third detection model for detecting the element correlation of the meteorological observation data, wherein the element correlation includes at least: spatial correlation.

[0075] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0076] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0077] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0078] S1, determining whether the meteorological observation data collected within a preset time period is missing;

[0079] S2. When the meteorological observation data is missing, call the first detection model to detect the meteorological observation data to determine abnormal values ​​in the meteorological observation data, wherein the first detection model includes at least one of the following: a second detection model for detecting the time correlation of the meteorological observation data, and a third detection model for detecting the element correlation of the meteorological observation data, wherein the element correlation includes at least: spatial correlation.

[0080] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0081] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0082] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0083] An embodiment of the present application also provides a computer program, which includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any one of the above method embodiments.

[0084] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0085] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0086] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining an outlier, characterized in that: include: Determine whether the meteorological observation data collected within the preset time period is missing; In the case that the meteorological observation data is missing, a first detection model is called to detect the meteorological observation data to determine abnormal values ​​in the meteorological observation data, wherein the first detection model includes at least one of the following: a second detection model for detecting the time correlation of the meteorological observation data, and a third detection model for detecting the element correlation of the meteorological observation data, wherein the element correlation includes at least: spatial correlation.

2. The method for determining an abnormal value according to claim 1, characterized in that: Before determining whether the meteorological observation data collected within the preset time period is missing, the method further includes: Performing a range check on the meteorological observation data to remove data in the meteorological observation data that exceeds a threshold range; and / or A peak check is performed on the meteorological observation data to remove the peak values ​​in the meteorological observation data.

3. The method for determining an abnormal value according to claim 1, characterized in that: Calling a first detection model to detect the meteorological observation data to determine abnormal values ​​in the meteorological observation data includes: Determining the missing type of the meteorological observation data; In a case where the missing measurement type is of the first type, it is determined to call the second detection model to detect the meteorological observation data so as to determine the abnormal values ​​in the meteorological observation data; in a case where the missing measurement type is of the second type, it is determined to call the third detection model to detect the meteorological observation data so as to determine the abnormal values ​​in the meteorological observation data; in a case where the missing measurement type includes both the first type and the second type, it is determined to call the second detection model and the third detection model at the same time to detect the meteorological observation data so as to determine the abnormal values ​​in the meteorological observation data.

4. The method for determining an abnormal value according to claim 3, characterized in that: Determine the missing type of the meteorological observation data, including: In the case where a first meteorological element in the meteorological observation data is missing, determining that the missing type is the first type; When a second meteorological element in the meteorological observation data is missing, it is determined that the missing type is the second type.

5. The method for determining an abnormal value according to claim 1, characterized in that: In the case where the meteorological observation data is missing, before calling the first detection model to detect the meteorological observation data to determine abnormal values ​​in the meteorological observation data, the method further includes: The correlation between the third meteorological element and the fourth meteorological element is determined by a grey correlation method, wherein the meteorological observation data includes: first meteorological observation data and second meteorological observation data, the first meteorological observation data is the meteorological observation data of a target site within a preset spatial range, the second meteorological observation data is the meteorological observation data of other sites within the preset spatial range except the target site, the third meteorological element is any meteorological element in the first meteorological observation data, and the fourth meteorological element includes: the observation values ​​corresponding to the third meteorological element at the previous n moments, where n is a positive integer; The second detection model is constructed by using a fifth meteorological element, wherein the fifth meteorological element is the meteorological element with the highest correlation with the third meteorological element among the fourth meteorological elements.

6. The method for determining an abnormal value according to claim 1, characterized in that: In the case where the meteorological observation data is missing, before calling the first detection model to detect the meteorological observation data to determine abnormal values ​​in the meteorological observation data, the method further includes: Determine a first correlation coefficient between the third meteorological element and the sixth meteorological element by using the Spearman correlation coefficient, and determine a second correlation coefficient between the third meteorological element and the seventh meteorological element by using the Spearman correlation coefficient; Determine the third detection model by using the first correlation coefficient and the second correlation coefficient; Among them, the meteorological observation data includes: first meteorological observation data and second meteorological observation data, the first meteorological observation data is the meteorological observation data of the target site within a preset spatial range, the second meteorological observation data is the meteorological observation data of other sites within the preset spatial range except the target site, the third meteorological element is any meteorological element in the first meteorological observation data, the sixth meteorological element is other meteorological elements in the first meteorological observation data except the third meteorological element, and the seventh meteorological element is the meteorological element in the other sites that is the same as the third meteorological element.

7. A device for determining an abnormal value, characterized in that: include: The first determination module is used to determine whether the meteorological observation data collected within a preset time period is missing; The second determination module is used to call the first detection model to detect the meteorological observation data when the meteorological observation data is missing, so as to determine the abnormal values ​​in the meteorological observation data, wherein the first detection model includes at least one of the following: a second detection model for detecting the time correlation of the meteorological observation data, and a third detection model for detecting the element correlation of the meteorological observation data, wherein the element correlation includes at least: spatial correlation.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 6 when executed.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 6 through the computer program.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images