Meteorological short-term and imminent forecasting method based on multi-source data fusion and real-time modeling

Abstract

The invention relates to the technical field of meteorological short-term and imminent forecasting, in particular to a meteorological short-term and imminent forecasting method based on multi-source data fusion and real-time modeling, and the meteorological short-term and imminent forecasting method based on multi-source data fusion and real-time modeling performs spatio-temporal interpolation through a time sliding filling method in combination with a DEM-containing trust propagation algorithm. And complementing the missing data and expanding meteorological data of other non-automatic station monitoring areas in the target area at the same time so as to obtain complete high-precision ground three-dimensional meteorological data (10 minutes per 10 minutes, 1 kilometer-level forecast precision) used for representing meteorological data in a three-dimensional space of the target area, and updating the UNet model in real time through an incremental learning strategy so as to realize real-time prediction of the meteorological data in the three-dimensional space of the target area. And secondly, a physical constraint module of an output layer of the UNet model ensures that a forecast result accords with a meteorological dynamics rule, and SIMD instruction optimization enables the fusion efficiency to be improved by 8 times, so that the real-time service requirement is met.

Description

Technical Field

[0001] The present invention relates to the technical field of short-term and imminent weather forecasting, and in particular, to a short-term and imminent weather forecasting method based on multi-source data fusion and real-time modeling. Background Art

[0002] Traditional short-term and imminent weather forecasting relies on numerical models (such as WRF) and radar extrapolation technology, and has defects of low spatio-temporal resolution (hourly level, 10-kilometer level) and insufficient data fusion ability. In the prior art, the collaborative processing efficiency of satellite data and ground observation data is low, and the lack of dynamic modeling ability results in poor forecasting timeliness. In addition, the problem of missing station data is serious, and traditional interpolation methods cannot take into account spatio-temporal consistency. Summary of the Invention

[0003] The purpose of the present invention is to provide a short-term and imminent weather forecasting method based on multi-source data fusion and real-time modeling to improve the above technical problems.

[0004] To achieve the above purpose, the embodiments of the present application provide the following technical solutions: On the one hand, the embodiments of the present application provide a short-term and imminent weather forecasting method based on multi-source data fusion and real-time modeling. The method includes: collecting ground automatic station minute observation data and satellite remote sensing data in real time, aligning the ground automatic station minute observation data to a preset unified timestamp and a preset spatial coordinate system, and then obtaining ground automatic station data; performing climate extreme value test and gradient test on the ground automatic station data in sequence, and removing the outliers detected by the climate extreme value test and / or gradient test; detecting the missing data in the ground automatic station data after removing the outliers, and using the time-sliding filling method combined with the trust propagation algorithm containing DEM for spatio-temporal interpolation to complete the missing data and expand the meteorological data of other non-automatic station monitoring areas within the target area, and then obtaining complete three-dimensional ground meteorological data; performing standardization processing on the three-dimensional ground meteorological data and the satellite remote sensing data, extracting satellite image features in the satellite remote sensing data at the same time, and fusing the three-dimensional ground meteorological data and the satellite image features to obtain spatio-temporal fusion data; initializing model parameters by transfer learning, training a UNet model in combination with the Huber loss function and the ADAM optimizer, and dynamically updating model weights to adapt to real-time spatio-temporal fusion data; inputting the spatio-temporal fusion data into the trained UNet model, and then outputting the meteorological element forecasting results within the next 2 hours in the target space area, where the spatial resolution of the meteorological element forecasting results is 1 kilometer and the time resolution is 10 minutes.

[0005] Optionally, detecting the missing data in the ground automatic station data after removing the outliers includes: Detect whether there is a data gap at a time node in the surface automatic station data at preset time intervals. If there is a data gap, generate a corresponding timestamp identifier, which is used to represent that there is no meteorological data at the corresponding time node in the surface automatic station data; In the case that there is meteorological data at the time node, verify the hash value corresponding to the meteorological data, and then determine whether the meteorological data corresponding to the time node is complete. If it is incomplete, supplement the data.

[0006] Optionally, use the time-sliding filling method combined with the trust propagation algorithm containing DEM for spatio-temporal interpolation to supplement the missing data, including: Time filling: Calculate the exponentially smoothed filling value using the meteorological data of the previous two hours; Spatial filling: Based on the meteorological data feedback from neighboring stations, altitude differences, and the spatial change rate of meteorological elements, iteratively update the missing values through the trust propagation algorithm.

[0007] Optionally, calculate the exponentially smoothed filling value using the meteorological data of the previous two hours, including: ; Among them, is the filling value at the current moment, and are the observed values at the previous t - 1 and previous t - 2 moments respectively, is the sliding coefficient, with a value range of 0.7 - 0.9; Based on the meteorological data feedback from neighboring stations, altitude differences, and the spatial change rate of meteorological elements, iteratively update the missing values through the trust propagation algorithm, including: ; Among them, is the filling value of station i at the (k + 1)-th iteration, is the filling value of station j at the k-th iteration, is the weight of neighboring station j of i, is the altitude difference correction term, is the adjustment factor, with a value range of 0.1 - 0.3; N(i) is the set of stations that are spatially adjacent to station i. j ∈ N(i) represents that station j is in the set of stations that are spatially adjacent to station i. In the iterative calculation, only consider the values of neighboring station j of station i.

[0008] Optionally, extracting the satellite image features in the satellite remote sensing data includes: Calculating the brightness temperature difference BTD1 of the first infrared channel, with a value range of 11.2 - 12.4 µm, the brightness temperature difference BTD2 of the second infrared channel, with a value range of 8.6 - 11.2 µm, and the brightness temperature difference BTD3 of the third infrared channel, with a value range of 6.2 - 11.2 µm; Convert the albedo in the visible light band to grayscale values from 0 to 255, and adjust the brightness in combination with the solar altitude angle.

[0009] Optionally, a physical constraint module is added to the output layer of the UNet model to forcibly adjust the precipitation prediction value to be non - negative and forcibly adjust the wind speed prediction value to satisfy the energy conservation equation.

[0010] Optionally, the dynamic update of the model weights to adapt to real - time spatio - temporal fusion data includes: adopting an incremental learning strategy: fine - tuning the model parameters every 10 minutes based on the latest spatio - temporal fusion data, and retaining the weights of 90% of the historical model parameters and updating the weights of 10% of the parameters each time the model parameters are fine - tuned.

[0011] In a second aspect, an embodiment of the present application provides a short - term and nowcasting meteorological forecasting device based on multi - source data fusion and real - time modeling. The device includes a memory and a processor.

[0012] The memory is used to store a computer program; the processor is used to implement the steps of the above - mentioned short - term and nowcasting meteorological forecasting method based on multi - source data fusion and real - time modeling when executing the computer program.

[0013] In a third aspect, an embodiment of the present application provides a medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above - mentioned short - term and nowcasting meteorological forecasting method based on multi - source data fusion and real - time modeling.

[0014] The beneficial effects of the present invention are as follows: For the short - term and nowcasting meteorological forecasting method based on multi - source data fusion and real - time modeling of the present invention, spatio - temporal interpolation is performed through the time - sliding filling method in combination with the belief propagation algorithm containing DEM, completing the missing data while expanding the meteorological data in other non - automatic weather station monitoring areas within the target area, and thus obtaining complete and high - precision ground three - dimensional meteorological data (forecast accuracy of every 10 minutes and 1 - kilometer level) for characterizing the meteorological data in the three - dimensional space of the target area. At the same time, the UNet model is updated in real time through the incremental learning strategy, improving the response ability to sudden weather.

[0015] Secondly, the physical constraint module in the output layer of the UNet model ensures that the forecast results conform to the meteorological dynamics law, and the SIMD instruction optimization improves the fusion efficiency by 8 times, meeting the real - time service requirements.

[0016] Other features and advantages of the present invention will be described in the subsequent specification, and, in part, will become apparent from the specification or be understood by implementing the embodiments of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the structures specifically pointed out in the written specification, claims, and drawings. Description of the Drawings

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0018] Figure 1 is a schematic flowchart of a short-term and imminent weather forecasting method based on multi-source data fusion and real-time modeling described in the embodiments of the present invention; Figure 2 is a schematic structural diagram of a short-term and imminent weather forecasting device based on multi-source data fusion and real-time modeling described in the embodiments of the present invention. Detailed Embodiments

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and shown in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0020] It should be noted that similar reference numerals or letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present invention, terms such as "first" and "second" are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0021] Embodiment 1:

[0022] As Figure 1 shown, this embodiment provides a short-term and imminent weather forecasting method based on multi-source data fusion and real-time modeling. The method includes step S100, step S200, step S300, and step S400.

[0023] Step S100: Real-time collect the minute observation data of the ground automatic station and satellite remote sensing data, and align the minute observation data of the ground automatic station to a preset unified timestamp and a preset spatial coordinate system, and then obtain the ground automatic station data; Step S200: Conduct climate extreme value tests and gradient tests on the surface automatic station data in sequence, and eliminate the outliers detected by the climate extreme value test (detect whether the detected data exceeds the climatologically reasonable range, such as the temperature exceeding the historical extreme value, and eliminate the obvious outliers to avoid the influence of incorrect data on model training) and / or the gradient test (verify whether the data change rate between adjacent time or space points is reasonable, such as the temperature dropping suddenly by 30°C within 1 hour), and identify the mutation data caused by sensor failures or transmission errors, so as to ensure the physical rationality and spatio-temporal consistency of the data and improve the reliability of the spatio-temporal fusion data; Step S300: Detect the missing data in the surface automatic station data after eliminating the outliers, and perform spatio-temporal interpolation using the time-sliding filling method combined with the trust propagation algorithm containing DEM to complete the missing data and expand the meteorological data in other non-automatic station monitoring areas within the target area, thereby obtaining complete three-dimensional surface meteorological data. Among them, performing spatio-temporal interpolation using the time-sliding filling method combined with the trust propagation algorithm containing DEM includes: Time filling: Calculate the exponentially smoothed filling value using the meteorological data of the previous two hours; ; Among them, is the filling value at the current moment, and are the observed values at the previous t - 1 and previous t - 2 moments respectively, is the sliding coefficient, with a value range of 0.7 - 0.9.

[0024] Spatial filling: Based on the meteorological data, altitude difference, and spatial change rate of meteorological elements feedback from adjacent stations, iteratively update the missing values through the trust propagation algorithm; ; Among them, is the filled value of station i at the (k + 1)-th iteration, is the filled value of station j at the k-th iteration, is the weight of the adjacent station j of i, is the altitude difference correction term, is the adjustment factor, with a value range of 0.1 - 0.3; N(i) is the set of stations adjacent to station i in space, j ∈ N(i), indicating that station j is in the set of stations adjacent to station i in space, and only the values of the adjacent stations j of station i are considered in the iterative calculation.

[0025] Step S400: Standardize the ground three-dimensional meteorological data and satellite remote sensing data. Meanwhile, extract the satellite image features from the satellite remote sensing data (calculate the brightness temperature difference BTD1 of the first infrared channel, with a value range of 11.2 - 12.4 µm, the brightness temperature difference BTD2 of the second infrared channel, with a value range of 8.6 - 11.2 µm, and the brightness temperature difference BTD3 of the third infrared channel, with a value range of 6.2 - 11.2 µm; convert the albedo in the visible light band to a grayscale value of 0 - 255, and adjust the brightness in combination with the solar altitude angle). Then fuse the ground three-dimensional meteorological data and satellite image features to obtain spatio-temporal fusion data. The spatio-temporal fusion data has higher spatio-temporal integrity and multi-dimensional feature expression ability, supporting the model to generate more refined forecast results. For example, if the ground three-dimensional meteorological data does not introduce multi-source features (such as cloud features, infrared brightness temperature difference), while the spatio-temporal fusion data contains features such as RGB grayscale and BTD extracted by satellites, enhancing the correlation of meteorological elements; Step S500: Initialize the model parameters using transfer learning, and train the UNet model in combination with the Huber loss function and the ADAM optimizer, dynamically updating the model weights to adapt to real-time spatio-temporal fusion data. A physical constraint module is added to the output layer of the UNet model to force the precipitation prediction value to be non-negative and force the wind speed prediction value to satisfy the energy conservation equation; The dynamic update of the model weights to adapt to real-time spatio-temporal fusion data includes: adopting an incremental learning strategy: fine-tuning the model parameters based on the latest spatio-temporal fusion data every 10 minutes, and when fine-tuning the model parameters each time, retaining the weights of 90% of the historical model parameters and updating the weights of 10% of the parameters. The spatio-temporal fusion data is used to calculate meteorological data such as U / V wind decomposition, water vapor flux, and pseudo-equivalent potential temperature.

[0026] Step S600: Input the spatio-temporal fusion data into the trained UNet model, and then output the meteorological element forecast results within the next 2 hours in the target spatial region. The spatial resolution of the meteorological element forecast results is 1 km, and the time resolution is 10 minutes.

[0027] Secondly, in step S300, the specific implementation method for detecting and removing missing data in the ground automatic station data after detecting and removing outliers is as follows: Step S310: Detect whether there is a data gap at a time node in the ground automatic station data at a preset time interval. If there is a data gap, generate a corresponding timestamp identifier, which is used to represent that there is no meteorological data at the corresponding time node in the ground automatic station data; Step S320: When there is meteorological data at the time node, verify the hash value corresponding to the meteorological data, and then determine whether the meteorological data corresponding to the time node is complete. If it is incomplete, complete the data.

[0028] The short-term and nowcasting meteorological forecasting method based on multi-source data fusion and real-time modeling described in this embodiment performs spatio-temporal interpolation through the time-sliding filling method combined with the trust propagation algorithm containing DEM, completes the missing data, and expands the meteorological data in other non-automatic station monitoring areas within the target area, so as to obtain complete and high-precision ground three-dimensional meteorological data (forecast accuracy of 10 minutes and 1 km level) for characterizing the meteorological data in the three-dimensional space of the target area. At the same time, the UNet model is updated in real time through the incremental learning strategy to improve the response ability to sudden weather.

[0029] Secondly, the physical constraint module in the output layer of the UNet model ensures that the forecast results conform to the meteorological dynamics law, and the SIMD instruction optimization improves the fusion efficiency by 8 times, meeting the real-time service requirements.

[0030] Embodiment 2:

[0031] Corresponding to the above method embodiment, the present disclosure embodiment also provides a short-term and nowcasting meteorological forecasting device based on multi-source data fusion and real-time modeling. The short-term and nowcasting meteorological forecasting device described below can be correspondingly referred to the short-term and nowcasting meteorological forecasting method described above.

[0032] Figure 2 It is a block diagram of a short-term and nowcasting meteorological forecasting device shown according to an exemplary embodiment. As Figure 2 shown, the electronic device 800 may include: a processor 801, a memory 802. The electronic device 800 may also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0033] Among them, the processor 801 is used to control the overall operation of the electronic device 800 to complete all or part of the steps in the above-mentioned short-term meteorological forecasting method based on multi-source data fusion and real-time modeling. The memory 802 is used to store various types of data to support the operation of the electronic device 800. These data may include, for example, instructions for any application or method operating on the electronic device 800, as well as application-related data, such as contact data, sent and received messages, pictures, audio, video, and so on. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc. The multimedia component 803 may include a screen and an audio component. The screen can be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal can be further stored in the memory 802 or sent through the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, and the above-mentioned other interface modules can be a keyboard, a mouse, buttons, etc. These buttons can be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the electronic device 800 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them. Accordingly, the communication component 805 may include: a Wi-Fi module, a Bluetooth module, and an NFC module.

[0034] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components, and is used to execute the above-mentioned short-term and imminent weather forecasting method based on multi-source data fusion and real-time modeling.

[0035] In another exemplary embodiment, a computer-readable storage medium including program instructions is further provided. When the program instructions are executed by a processor, the steps of the above-mentioned short-term and imminent weather forecasting method based on multi-source data fusion and real-time modeling are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 802 including program instructions, and the above program instructions may be executed by the processor 801 of the electronic device 800 to complete the above-mentioned short-term and imminent weather forecasting method based on multi-source data fusion and real-time modeling.

[0036] Embodiment 3:

[0037] Corresponding to the above method embodiment, the present disclosure embodiment also provides a readable storage medium. A readable storage medium described below and a short-term and imminent weather forecasting method based on multi-source data fusion and real-time modeling described above can be correspondingly referred to each other.

[0038] A readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the short-term and imminent weather forecasting method based on multi-source data fusion and real-time modeling in the above method embodiment are implemented.

[0039] The readable storage medium may specifically be various readable storage media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.

[0040] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A short-term meteorological forecasting method based on multi-source data fusion and real-time modeling, characterized in that: The method comprises: Collect ground automatic station minute observation data and satellite remote sensing data in real time, and align the ground automatic station minute observation data to a preset unified timestamp and a preset spatial coordinate system, thereby obtaining ground automatic station data; Conduct climate extreme value test and gradient test on the ground automatic station data in turn, and remove the abnormal values ​​found in the climate extreme value test and / or gradient test; Detect missing data in the ground automatic station data after removing outliers, and use the time sliding filling method combined with the trust propagation algorithm containing DEM to perform spatiotemporal interpolation to complete the missing data while expanding the meteorological data of other non-automatic station monitoring areas in the target area, thereby obtaining complete ground three-dimensional meteorological data; The ground three-dimensional meteorological data and the satellite remote sensing data are standardized, satellite image features in the satellite remote sensing data are extracted, and the ground three-dimensional meteorological data and the satellite image features are fused to obtain spatiotemporal fusion data; Transfer learning is used to initialize model parameters, and the UNet model is trained in combination with the Huber loss function and ADAM optimizer, and the model weights are dynamically updated to adapt to real-time spatiotemporal fusion data; The spatiotemporal fusion data is input into the trained UNet model, and then the meteorological element forecast results in the target spatial area within the next 2 hours are output. The spatial resolution of the meteorological element forecast results is 1 km and the temporal resolution is 10 minutes.

2. The method for short-term meteorological forecasting based on multi-source data fusion and real-time modeling according to claim 1 is characterized in that: Detect missing data in ground automatic station data after removing outliers, including: Detect whether there is a data gap at a time node in the ground automatic station data at a preset time interval, and if there is a data gap, generate a corresponding timestamp identifier, wherein the timestamp identifier is used to indicate that there is no meteorological data at the corresponding time node in the ground automatic station data; If there is meteorological data at a time node, verify the hash value corresponding to the meteorological data to determine whether the meteorological data corresponding to the time node is complete. If not, complete the data.

3. The method for short-term meteorological forecasting based on multi-source data fusion and real-time modeling according to claim 2 is characterized in that: The time sliding filling method combined with the belief propagation algorithm containing DEM is used for spatiotemporal interpolation to complete the missing data, including: Time completion: use the meteorological data of the previous two hours to calculate the index sliding filling value; Spatial completion: Based on the meteorological data fed back by neighboring stations, altitude differences, and spatial change rates of meteorological elements, missing values ​​are iteratively updated through the trust propagation algorithm.

4. The method for short-term meteorological forecasting based on multi-source data fusion and real-time modeling according to claim 3 is characterized in that: The index sliding fill value is calculated using the meteorological data of the previous two hours, including: ； in, Fill in the value for the current moment, and are the observation values ​​before t-1 and t-2 respectively, is the sliding coefficient, the value is 0.7-0.9; Based on the meteorological data, altitude differences and spatial change rates of meteorological elements fed back by neighboring stations, the missing values ​​are iteratively updated through the trust propagation algorithm, including: ； in, is the padding value of the k+1th iteration of site i, is the completion value of the kth iteration of site j, is the weight of i’s neighboring site j, is the altitude difference correction term, is the adjustment factor, with a value of 0.1-0.3; N(i) is the set of sites spatially adjacent to site i, j∈N(i), representing the site j in the set of sites spatially adjacent to site i. In the iterative calculation, only the values ​​of the neighboring sites j of site i are considered.

5. The method for short-term meteorological forecasting based on multi-source data fusion and real-time modeling according to claim 4 is characterized in that: The extracting of satellite image features from satellite remote sensing data comprises: Calculate the brightness temperature difference BTD1 of the first infrared channel, with a value of 11.2-12.4µm, the brightness temperature difference BTD2 of the second infrared channel, with a value of 8.6-11.2µm, and the brightness temperature difference BTD3 of the third infrared channel, with a value of 6.2-11.2µm; Convert the visible light band albedo to a 0-255 grayscale value and adjust the brightness based on the sun altitude angle.

6. The method for short-term meteorological forecasting based on multi-source data fusion and real-time modeling according to claim 5 is characterized in that: A physical constraint module is added to the output layer of the UNet model to force the precipitation prediction value to be adjusted to a non-negative value and to force the wind speed prediction value to satisfy the energy conservation equation.

7. The method for short-term meteorological forecasting based on multi-source data fusion and real-time modeling according to claim 6 is characterized in that: The dynamic updating of model weights to adapt to real-time spatiotemporal fusion data includes: adopting an incremental learning strategy: fine-tuning model parameters based on the latest spatiotemporal fusion data every 10 minutes, and retaining 90% of the parameter weights of the historical model and updating 10% of the parameter weights each time the model parameters are fine-tuned.

Images