A method for short-term meteorological forecasting based on multi-source data fusion and real-time modeling

Abstract

The present invention relates to the technical field of short-term meteorological forecasting, and specifically to a method for short-term meteorological forecasting based on multi-source data fusion and real-time modeling. The method for short-term meteorological forecasting based on multi-source data fusion and real-time modeling described in the present invention performs spatiotemporal interpolation by combining a time sliding filling method with a trust propagation algorithm containing a DEM, thereby completing the missing data while expanding the meteorological data of other non-automatic station monitoring areas within the target area, thereby obtaining complete and high-precision ground three-dimensional meteorological data (10-minute, 1-kilometer forecast accuracy) for characterizing meteorological data in the three-dimensional space of the target area. At the same time, an incremental learning strategy is used to update a UNet model in real time, thereby improving the response capability to sudden weather events. Secondly, a physical constraint module of the output layer of the UNet model ensures that the forecast results conform to the laws of meteorological dynamics, and SIMD instruction optimization increases the fusion efficiency by 8 times, meeting real-time business needs.

Description

Technical Field

[0001] The present invention relates to the technical field of short-term and impending weather forecasting, and in particular to a short-term and impending weather forecasting method based on multi-source data fusion and real-time modeling. Background Art

[0002] Traditional short-term weather forecasts rely on numerical models (such as WRF) and radar extrapolation techniques, which suffer from low temporal and spatial resolution (hourly and 10-kilometer levels) and insufficient data fusion capabilities. Existing technologies inefficiently process satellite and ground-based data and lack dynamic modeling capabilities, resulting in poor forecast timeliness. Furthermore, due to significant missing site data, traditional interpolation methods cannot guarantee both temporal and spatial consistency. Summary of the Invention

[0003] The purpose of the present invention is to provide a short-term weather forecast method based on multi-source data fusion and real-time modeling to improve the problems of the above-mentioned technology.

[0004] In order to achieve the above objectives, the embodiments of the present application provide the following technical solutions:

[0005] On the one hand, an embodiment of the present application provides a short-term weather forecasting method based on multi-source data fusion and real-time modeling, the method comprising: real-time collection of ground automatic station minute observation data and satellite remote sensing data, and alignment of the ground automatic station minute observation data to a preset unified timestamp and a preset spatial coordinate system, thereby obtaining ground automatic station data; performing climate extreme value test and gradient test on the ground automatic station data in turn, and removing outliers from the climate extreme value test and / or gradient test; detecting missing data in the ground automatic station data after removing outliers, and performing spatiotemporal interpolation using a time sliding filling method combined with a trust propagation algorithm containing a DEM, while completing the missing data and expanding other non-automatic station monitoring areas within the target area. meteorological data, and then obtain complete ground three-dimensional meteorological data; standardize the ground three-dimensional meteorological data and satellite remote sensing data, and extract satellite image features in the satellite remote sensing data, and fuse the ground three-dimensional meteorological data and satellite image features to obtain spatiotemporal fusion data; use transfer learning to initialize model parameters, combine the Huber loss function and ADAM optimizer to train the UNet model, and dynamically update the model weights to adapt to real-time spatiotemporal fusion data; input the spatiotemporal fusion data into the trained UNet model, and then output the meteorological element forecast results in the target spatial area within the next 2 hours, the spatial resolution of the meteorological element forecast results is 1 km, and the temporal resolution is 10 minutes.

[0006] Optionally, detecting missing data in the ground automatic station data after removing outliers includes:

[0007] Detecting whether there is a data gap at a time node in the ground automatic station data at a preset time interval, and if there is a data gap, generating a corresponding timestamp identifier, wherein the timestamp identifier is used to indicate that there is no meteorological data at the corresponding time node in the ground automatic station data;

[0008] If there is meteorological data at a time node, the hash value corresponding to the meteorological data is verified to determine whether the meteorological data corresponding to the time node is complete. If not, the data is completed.

[0009] Optionally, a time sliding filling method combined with a belief propagation algorithm containing a DEM is used to perform spatiotemporal interpolation to complete the missing data, including:

[0010] Time completion: Use the meteorological data of the previous two hours to calculate the index sliding filling value;

[0011] Spatial completion: Based on the meteorological data fed back by neighboring stations, altitude differences, and spatial change rates of meteorological elements, missing values ​​are iteratively updated through the trust propagation algorithm.

[0012] Optionally, the index sliding fill value is calculated using the meteorological data of the previous two hours, including:

[0013] ;

[0014] in, Fill in the value for the current moment, and are the observation values ​​before t-1 and t-2 respectively, is the sliding coefficient, with a value of 0.7-0.9;

[0015] Based on meteorological data fed back from neighboring stations, altitude differences, and spatial change rates of meteorological elements, missing values ​​are iteratively updated using a trust propagation algorithm, including:

[0016] ;

[0017] in, is the padding value of the k+1th iteration of site i, is the padding value of the k-th iteration of site j, is the weight of i’s neighboring site j, is the altitude difference correction term, is the adjustment factor, with a value of 0.1-0.3; N(i) is the set of sites spatially adjacent to site i, j∈N(i), representing the site j in the set of sites spatially adjacent to site i. In the iterative calculation, only the values ​​of the neighboring sites j of site i are considered.

[0018] Optionally, extracting satellite image features from satellite remote sensing data includes:

[0019] Calculate the brightness temperature difference (BTD1) of the first infrared channel, with a value of 11.2-12.4µm, the brightness temperature difference (BTD2) of the second infrared channel, with a value of 8.6-11.2µm, and the brightness temperature difference (BTD3) of the third infrared channel, with a value of 6.2-11.2µm;

[0020] Convert the visible light band albedo to a 0-255 grayscale value and adjust the brightness based on the sun's altitude angle.

[0021] Optionally, a physical constraint module is added to the output layer of the UNet model to force the precipitation prediction value to be adjusted to a non-negative value and to force the wind speed prediction value to satisfy the energy conservation equation.

[0022] Optionally, the dynamic updating of model weights to adapt to real-time spatiotemporal fusion data includes: adopting an incremental learning strategy: fine-tuning the model parameters based on the latest spatiotemporal fusion data every 10 minutes, and retaining the weights of 90% of the parameters of the historical model each time the model parameters are fine-tuned, and updating the weights of 10% of the parameters.

[0023] In a second aspect, an embodiment of the present application provides a short-term weather forecasting device based on multi-source data fusion and real-time modeling, wherein the device includes a memory and a processor.

[0024] The memory is used to store computer programs; the processor is used to implement the steps of the above-mentioned weather short-term forecast method based on multi-source data fusion and real-time modeling when executing the computer program.

[0025] In a third aspect, an embodiment of the present application provides a medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method for short-term weather forecasting based on multi-source data fusion and real-time modeling.

[0026] The beneficial effects of the present invention are:

[0027] The short-term meteorological forecast method based on multi-source data fusion and real-time modeling described in the present invention performs spatiotemporal interpolation through a time sliding filling method combined with a trust propagation algorithm containing a DEM, thereby completing the missing data while expanding the meteorological data of other non-automatic station monitoring areas in the target area, thereby obtaining complete and high-precision ground three-dimensional meteorological data (10-minute, 1-kilometer forecast accuracy) for characterizing the meteorological data in the three-dimensional space of the target area. At the same time, an incremental learning strategy is used to update the UNet model in real time, thereby improving the response capability to sudden weather events.

[0028] Secondly, the physical constraint module of the UNet model output layer ensures that the forecast results conform to the laws of meteorological dynamics, and SIMD instruction optimization increases the fusion efficiency by 8 times to meet real-time business needs.

[0029] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or understood by practicing the embodiments of the present invention. The purposes and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 This is a flow chart of a method for short-term and impending weather forecasting based on multi-source data fusion and real-time modeling according to an embodiment of the present invention;

[0032] Figure 2 This is a schematic structural diagram of a short-term weather forecast device based on multi-source data fusion and real-time modeling described in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0034] It should be noted that similar reference numerals or letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0035] Example 1:

[0036] like Figure 1As shown, this embodiment provides a method for short-term weather forecasting based on multi-source data fusion and real-time modeling, and the method includes step S100, step S200, step S300 and step S400.

[0037] Step S100: collecting ground automatic station minute observation data and satellite remote sensing data in real time, and aligning the ground automatic station minute observation data to a preset unified timestamp and a preset spatial coordinate system, thereby obtaining ground automatic station data;

[0038] Step S200: Perform climate extreme value and gradient tests on the ground automatic station data in sequence, and remove outliers from the climate extreme value test (detecting whether the data exceeds the climatologically reasonable range, such as temperature exceeding the historical extreme value, removing obvious outliers to prevent erroneous data from affecting model training) and / or gradient test (verifying whether the data change rate of adjacent time or space points is reasonable, such as a sudden drop of 30°C in temperature within 1 hour, and identifying sudden changes caused by sensor failure or transmission errors) to ensure the physical rationality and spatiotemporal consistency of the data and improve the reliability of spatiotemporal fusion data.

[0039] Step S300: Detect missing data in the ground automatic station data after removing outliers, and perform spatiotemporal interpolation using a time sliding filling method combined with a trust propagation algorithm containing a DEM, thereby completing the missing data and expanding the meteorological data of other non-automatic station monitoring areas within the target area, thereby obtaining complete ground three-dimensional meteorological data. The spatiotemporal interpolation using the time sliding filling method combined with a trust propagation algorithm containing a DEM includes:

[0040] Time completion: Use the meteorological data of the previous two hours to calculate the index sliding filling value;

[0041] ;

[0042] in, Fill in the value for the current moment, and are the observation values ​​before t-1 and t-2 respectively, is the sliding coefficient, and its value is 0.7-0.9.

[0043] Spatial filling: Based on meteorological data fed back by neighboring stations, altitude differences, and spatial change rates of meteorological elements, missing values ​​are iteratively updated using the trust propagation algorithm;

[0044] ;

[0045] in, is the padding value of the k+1th iteration of site i, is the padding value of the k-th iteration of site j, is the weight of i’s neighboring site j, is the altitude difference correction term, is the adjustment factor, with a value of 0.1-0.3; N(i) is the set of sites spatially adjacent to site i, j∈N(i), representing the site j in the set of sites spatially adjacent to site i. In the iterative calculation, only the values ​​of the neighboring sites j of site i are considered.

[0046] Step S400: Standardize the ground-based three-dimensional meteorological data and the satellite remote sensing data, extract satellite image features from the satellite remote sensing data (calculate the brightness temperature difference (BTD1) of the first infrared channel, with a value of 11.2-12.4 µm, the brightness temperature difference (BTD2) of the second infrared channel, with a value of 8.6-11.2 µm, and the brightness temperature difference (BTD3) of the third infrared channel, with a value of 6.2-11.2 µm; convert the albedo of the visible light band into a grayscale value of 0-255, and adjust the brightness based on the solar altitude angle). Then, fuse the ground-based three-dimensional meteorological data and the satellite image features to obtain spatiotemporal fusion data. The spatiotemporal fusion data has higher spatiotemporal integrity and multidimensional feature expression capabilities, supporting the model to generate more refined forecast results. For example, the ground-based three-dimensional meteorological data does not introduce multi-source features (such as cloud features and infrared brightness temperature difference), while the spatiotemporal fusion data includes features such as RGB grayscale and BTD extracted by satellites, thereby enhancing the correlation of meteorological elements.

[0047] Step S500, using transfer learning to initialize model parameters, combining the Huber loss function and the ADAM optimizer to train the UNet model, dynamically updating the model weights to adapt to real-time spatiotemporal fusion data, adding a physical constraint module to the output layer of the UNet model to force the precipitation prediction value to be adjusted to a non-negative value, and forcing the wind speed prediction value to be adjusted to satisfy the energy conservation equation; the dynamic updating of the model weights to adapt to real-time spatiotemporal fusion data includes: adopting an incremental learning strategy: fine-tuning the model parameters based on the latest spatiotemporal fusion data every 10 minutes, and retaining the weights of 90% of the parameters of the historical model each time the model parameters are fine-tuned, and updating the weights of 10% of the parameters. The spatiotemporal fusion data is used to calculate meteorological data such as U / V wind decomposition, water vapor flux and pseudo equivalent potential temperature.

[0048] Step S600: Input the spatiotemporal fusion data into the trained UNet model, and then output the meteorological element forecast results in the target spatial area within the next 2 hours. The spatial resolution of the meteorological element forecast results is 1 km and the temporal resolution is 10 minutes.

[0049] Secondly, in step S300, the specific implementation method of detecting missing data in the ground automatic station data after removing outliers is as follows:

[0050] Step S310: detecting whether there is a data gap at a time node in the ground automatic station data at a preset time interval; if there is a data gap, generating a corresponding timestamp identifier, wherein the timestamp identifier is used to indicate that there is no meteorological data at the corresponding time node in the ground automatic station data;

[0051] Step S320: If there is meteorological data at the time node, verify the hash value corresponding to the meteorological data, and then determine whether the meteorological data corresponding to the time node is complete. If not, complete the data.

[0052] The short-term meteorological forecast method based on multi-source data fusion and real-time modeling described in this embodiment performs spatiotemporal interpolation through a time sliding filling method combined with a trust propagation algorithm containing a DEM, thereby completing the missing data while expanding the meteorological data of other non-automatic station monitoring areas in the target area, thereby obtaining complete and high-precision ground three-dimensional meteorological data (10-minute, 1-kilometer forecast accuracy) for characterizing the meteorological data in the three-dimensional space of the target area. At the same time, an incremental learning strategy is used to update the UNet model in real time, thereby improving the response capability to sudden weather events.

[0053] Secondly, the physical constraint module of the UNet model output layer ensures that the forecast results conform to the laws of meteorological dynamics, and SIMD instruction optimization increases the fusion efficiency by 8 times to meet real-time business needs.

[0054] Example 2:

[0055] Corresponding to the above method embodiments, the embodiments of the present disclosure also provide a weather short-term forecast device based on multi-source data fusion and real-time modeling. The weather short-term forecast device based on multi-source data fusion and real-time modeling described below and the weather short-term forecast method based on multi-source data fusion and real-time modeling described above can be referred to each other.

[0056] Figure 2 FIG1 is a block diagram of a weather forecast device based on multi-source data fusion and real-time modeling according to an exemplary embodiment. Figure 2 As shown, the electronic device 800 may include: a processor 801 , a memory 802 , and may further include one or more of a multimedia component 803 , an I / O interface 804 , and a communication component 805 .

[0057] The processor 801 is used to control the overall operation of the electronic device 800 to complete all or part of the steps in the aforementioned method for short-term and impending weather forecasting based on multi-source data fusion and real-time modeling. The memory 802 is used to store various types of data to support the operation of the electronic device 800. This data may include, for example, instructions for any application or method operating on the electronic device 800, as well as application-related data such as contact information, sent and received messages, images, audio, video, and the like. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 802 or transmitted via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, which may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the electronic device 800 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more thereof, may include a Wi-Fi module, a Bluetooth module, or an NFC module.

[0058] In an exemplary embodiment, the electronic device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the above-mentioned short-term weather forecasting method based on multi-source data fusion and real-time modeling.

[0059] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, the program instructions implement the steps of the aforementioned method for short-term and impending weather forecasting based on multi-source data fusion and real-time modeling. For example, the computer-readable storage medium may be the aforementioned memory 802 including the program instructions. The program instructions may be executed by the processor 801 of the electronic device 800 to implement the aforementioned method for short-term and impending weather forecasting based on multi-source data fusion and real-time modeling.

[0060] Example 3:

[0061] Corresponding to the above method embodiment, the embodiment of the present disclosure also provides a readable storage medium. The readable storage medium described below and the weather short-term forecast method based on multi-source data fusion and real-time modeling described above can refer to each other.

[0062] A readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the weather short-term forecast method based on multi-source data fusion and real-time modeling of the above method embodiment.

[0063] The readable storage medium may specifically be any readable storage medium that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0064] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for short-term weather forecasting based on multi-source data fusion and real-time modeling, characterized in that: The method comprises: Collect ground automatic station minute observation data and satellite remote sensing data in real time, and align the ground automatic station minute observation data to a preset unified timestamp and a preset spatial coordinate system to obtain ground automatic station data; Perform climate extreme value test and gradient test on the ground automatic station data in turn, and remove the abnormal values ​​found in the climate extreme value test and / or gradient test; Detect missing data in the ground automatic station data after removing outliers, and use the time sliding filling method combined with the belief propagation algorithm containing DEM to perform spatiotemporal interpolation to fill the missing data while expanding the meteorological data of other non-automatic station monitoring areas in the target area, thereby obtaining complete ground three-dimensional meteorological data; Standardizing the ground three-dimensional meteorological data and the satellite remote sensing data, extracting satellite image features from the satellite remote sensing data, and fusing the ground three-dimensional meteorological data and the satellite image features to obtain spatiotemporal fusion data; Transfer learning is used to initialize model parameters, and the UNet model is trained in combination with the Huber loss function and ADAM optimizer, and the model weights are dynamically updated to adapt to real-time spatiotemporal fusion data; The spatiotemporal fusion data is input into the trained UNet model, which then outputs the meteorological element forecast results for the target spatial area within the next 2 hours. The spatial resolution of the meteorological element forecast results is 1 km and the temporal resolution is 10 minutes. Secondly, detect missing data in the ground automatic station data after removing outliers, including: Detecting whether there is a data gap at a time node in the ground automatic station data at a preset time interval, and if there is a data gap, generating a corresponding timestamp identifier, wherein the timestamp identifier is used to indicate that there is no meteorological data at the corresponding time node in the ground automatic station data; If there is meteorological data at a time node, verify the hash value corresponding to the meteorological data to determine whether the meteorological data corresponding to the time node is complete. If not, complete the data. The extracting of satellite image features from satellite remote sensing data includes: Calculate the brightness temperature difference (BTD1) of the first infrared channel, with a value of 11.2-12.4µm, the brightness temperature difference (BTD2) of the second infrared channel, with a value of 8.6-11.2µm, and the brightness temperature difference (BTD3) of the third infrared channel, with a value of 6.2-11.2µm; Convert the visible light band albedo to a grayscale value of 0-255 and adjust the brightness based on the sun's altitude angle; Secondly, a physical constraint module is added to the output layer of the UNet model to force the precipitation prediction value to be non-negative and the wind speed prediction value to satisfy the energy conservation equation. Secondly, the dynamic updating of model weights to adapt to real-time spatiotemporal fusion data includes: adopting an incremental learning strategy: fine-tuning model parameters every 10 minutes based on the latest spatiotemporal fusion data, and each time the model parameters are fine-tuned, retaining the weights of 90% of the parameters of the historical model and updating the weights of 10% of the parameters; Secondly, the time sliding filling method combined with the belief propagation algorithm with DEM is used to perform spatiotemporal interpolation to fill in the missing data, including: Time completion: Use the meteorological data of the previous two hours to calculate the index sliding filling value; Spatial filling: Based on meteorological data fed back by neighboring stations, altitude differences, and spatial change rates of meteorological elements, missing values ​​are iteratively updated using the trust propagation algorithm; Secondly, the index sliding fill value is calculated using the meteorological data of the previous two hours, including: ； in, Fill in the value for the current moment, and are the observation values ​​before t-1 and t-2 respectively, is the sliding coefficient, with a value of 0.7-0.9; Based on meteorological data fed back from neighboring stations, altitude differences, and spatial change rates of meteorological elements, missing values ​​are iteratively updated using a trust propagation algorithm, including: ； in, is the padding value of the k+1th iteration of site i, is the padding value of the k-th iteration of site j, is the weight of i’s neighboring site j, is the altitude difference correction term, is the adjustment factor, with a value of 0.1-0.3; N(i) is the set of sites spatially adjacent to site i, j∈N(i), representing the site j in the set of sites spatially adjacent to site i. In the iterative calculation, only the values ​​of the neighboring sites j of site i are considered.

Images