A system and method for real-time regional precipitation forecasting based on an extension of cycle generative adversarial networks

Abstract

The regional precipitation real-time forecast system based on the extension of the cycle generative adversarial network includes an input unit that receives an input synthetic hybrid surface rainfall (HSR) image including precipitation information of a first time in a region of interest, a cycle generative adversarial network (Cycle-GAN) that generates a resultant synthetic HSR image including precipitation information of a second time, which is a time in the future from the first time, based on the input synthetic HSR image using a first cycle generative adversarial network (Cycle-GAN) and a second cycle generative adversarial network complementary to the first cycle generative adversarial network, and an output unit that outputs the resultant synthetic HSR image as a nowcasting image of the region of interest. The regional precipitation real-time forecast system and method based on the extension of the cycle generative adversarial network according to the present invention can ensure robustness against temporal causality by applying pixel loss to the cycle generative adversarial network.

Description

[Technical field]

[0001] The present invention relates to a system and method for real-time precipitation forecasting, and more particularly to a system and method for real-time regional precipitation forecasting based on an extension of cycle generative adversarial networks. [Background technology]

[0002] Rain has had a close impact on human life since ancient times. Therefore, predicting precipitation has always been a concern for people. There have been many attempts to predict precipitation from a meteorological perspective. One example is the numerical weather prediction (NWP) model. The NWP model supports good quantitative precipitation prediction over a wide range of time. However, since the NWP model develops convective-scale structures in the early stages of prediction, it cannot guarantee accuracy for the first few hours. This limitation is called the spin-up problem. To solve this problem, the importance of nowcasting, which focuses on short-term prediction, has increased.

[0003] In meteorology, a real-time forecast is a model or prediction that accurately predicts the weather, typically within the next two hours. In particular, a real-time precipitation forecast is usually based on weather radar observations. Although there are various quantitative precipitation forecast (QPF) models, perfect precipitation forecasting is always a difficult problem.

[0004] In addition, conventional precipitation forecasts are limited in that they cover a wide area and cannot forecast the current weather conditions of a local area of ​​interest to a user.

[0005] Therefore, there has been a problem in that it is not possible to effectively prepare for localized heavy rainfall based on the conventional precipitation actual situation forecast.

[0006] There are increasing efforts to use artificial intelligence in real-time precipitation forecasting. However, there is a need for reliable real-time precipitation forecasting systems that can forecast localized heavy rainfall over a small area.

[0007] There is a need for a system for obtaining real-time weather information for a user-desired local area at a desired time period. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Korean Patent Publication 10-2457470 (2022.10.18) Summary of the Invention [Problem to be solved by the invention]

[0009] The present invention provides a system and method for reliable real-time regional precipitation forecasting by extending a cycle-consistent adversarial network (Cycle-GAN) in the time-series domain.

[0010] The objects of the present invention are not limited to those mentioned above, and other objects not mentioned herein will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0011] A regional precipitation forecast system based on an extension of a cycle generative adversarial network according to an embodiment of the present invention is as follows: an input for receiving an input composite hybrid surface rainfall (HSR) image including first-hour precipitation information for a region of interest; a cycle generative adversarial network (Cycle-GAN) for generating a resultant synthetic HSR image based on the input synthetic HSR image, the resultant synthetic HSR image including precipitation information for a second time, the second time being a future time relative to the first time, in the region of interest; and an output unit that outputs the resulting composite HSR image as a nowcasting image of the area of ​​interest.

[0012] Preferably, The Cycle Generative Adversarial Network (Cycle-GAN) is A first cycle generative adversarial network (Cycle-GAN) and a second cycle generative adversarial network complementary to the first cycle generative adversarial network; The first cycle generative adversarial network and the second cycle generative adversarial network perform forward image mapping and backward image mapping, respectively, which proceed forward in time and backward in time.

[0013] Preferably, The first cycle generative adversarial network (Cycle-GAN) a forward generator that learns a predictive mapping of a first predicted synthesized HSR image at the second time based on the input synthesized HSR image at the first time; a backward generator that learns a predictive mapping of a first cycle predicted synthesized HSR image at the first time based on the first predicted synthesized HSR image at the second time; a forward discriminator that evaluates the accuracy of the predictive mapping of the forward generator and discriminates between the input synthesized HSR image and the first predicted synthesized HSR image; and a backward discriminator for evaluating accuracy of the prediction mapping of the backward generator and discriminating between the first predicted synthesized HSR image and the first cycle predicted synthesized HSR image.

[0014] Preferably, The second cycle generative adversarial network (Cycle-GAN) learning a prediction mapping of a complementary input synthesized HSR image corresponding to the input synthesized HSR image at the first time based on a complementary first predicted synthesized HSR image obtained by applying a first pixel loss function to the first predicted synthesized HSR image at the second time using the inverse generator, wherein the input synthesized HSR image and the complementary input synthesized HSR image have a complementary relationship to which a second pixel loss function is applied;

[0015] learning a predictive mapping of a second cycle predicted synthesized HSR image at a second time based on the complementary input synthesized HSR image at the first time using the forward generator; evaluating the accuracy of the predictive mapping of the forward generator using the forward discriminator to discriminate between the complementary input composite HSR image and the second cycle predicted composite HSR image; The inverse discriminator is configured to assess accuracy of a predictive mapping of the inverse generator using the inverse discriminator to discriminate between the complementary first predicted synthesized HSR image and the complementary input synthesized HSR image.

[0016] Preferably, The first cycle generative adversarial network (Cycle-GAN) is trained to match the first cycle predicted synthetic HSR image to the input synthetic HSR image by applying a first cycle consistency loss function (Cycle-Consistency Loss); The forward discriminator is trained to increase a probability of misidentifying the first predicted synthesized HSR image as the input synthesized HSR image by applying a forward adversarial loss function, The backward classifier is trained to increase the probability of misidentifying the first cycle predicted synthesized HSR image as the first predicted synthesized HSR image by applying a backward adversarial loss function.

[0017] Preferably, The second cycle generative adversarial network (Cycle-GAN) is trained to match the second cycle predicted synthetic HSR image to the complementary first predicted synthetic HSR image by applying a second cycle consistency loss function (Cycle-Consistency Loss); The forward discriminator is trained to increase a probability of misidentifying the second cycle predicted synthesized HSR image as the complementary input synthesized HSR image by applying the forward adversarial loss function; The backward discriminator is trained to increase the probability of misidentifying the complementary input synthesized HSR image as the complementary first predicted synthesized HSR image by applying a backward adversarial loss function.

[0018] Preferably, The cycle generative adversarial network (Cycle-GAN) outputs the first predicted synthesized HSR image or the complementary first predicted synthesized HSR image or the second cycle predicted synthesized HSR image as the resulting synthesized HSR image when the first pixel loss function and the second pixel loss function are trained to converge to a predetermined reference pixel loss function value and the first cycle consistency loss function and the second cycle consistency loss function are trained to converge to a predetermined cycle consistency loss function value.

[0019] Preferably, The forward generator and the backward generator are an encoder for reducing a size of the input composite HSR image and extracting a plurality of feature maps from the input composite HSR image; A number of SE-residual blocks for recalibrating the feature maps; a decoder for restoring the size of the input composite HSR image that was reduced by the encoder.

[0020] Preferably, The forward classifier and the backward classifier use a patch generative adversarial network (Patch-GAN) architecture that performs classification on a patch-by-patch basis.

[0021] According to another embodiment of the present invention, a method for forecasting precipitation amount by region based on an extension of a cycle generative adversarial network is as follows: an input data receiving stage for receiving an input composite hybrid height surface rainfall (HSR) image including first hourly precipitation information for a region of interest; performing a cyclic generative adversarial network to generate a resultant synthetic HSR image based on the input synthetic HSR image, the resultant synthetic HSR image including precipitation information for a second time, the second time being a future time relative to the first time, in the region of interest; and outputting the resulting composite HSR image as a nowcasting image of the area of ​​interest.

[0022] Preferably, The cycle generative adversarial network is A first cycle generative adversarial network (Cycle-GAN) and a second cycle generative adversarial network complementary to the first cycle generative adversarial network; The first cycle generative adversarial network and the second cycle generative adversarial network perform forward image mapping and backward image mapping, respectively, which proceed forward in time and backward in time.

[0023] Preferably, The first cycle generative adversarial network (Cycle-GAN) a forward image mapping step of learning a predictive mapping of a first predicted synthesized HSR image at the second time based on the input synthesized HSR image at the first time using a forward generator; a backward image mapping step of learning a predictive mapping of a first cycle predicted synthesized HSR image at the first time based on the first predicted synthesized HSR image at the second time using a backward generator; a forward discrimination step of evaluating the accuracy of the predictive mapping of the forward generator using a forward discriminator to discriminate between the input synthesized HSR image and the first predicted synthesized HSR image; and A reverse discrimination step is performed in which an accuracy of the predictive mapping of the reverse generator is evaluated using a reverse discriminator to discriminate between the first predicted synthesized HSR image and the first cycle predicted synthesized HSR image.

[0024] Preferably, The second cycle generative adversarial network (Cycle-GAN) learning a prediction mapping of a complementary input synthesized HSR image corresponding to the input synthesized HSR image at the first time based on a complementary first predicted synthesized HSR image obtained by applying a first pixel loss function to the first predicted synthesized HSR image at the second time using the inverse generator, and the input synthesized HSR image and the complementary input synthesized HSR image maintain a complementary relationship to which a second pixel loss function is applied;

[0025] learning a predictive mapping of a second cycle predicted synthesized HSR image at a second time based on the complementary input synthesized HSR image at the first time using the forward generator; evaluating the accuracy of the predictive mapping of the forward generator using the forward discriminator to discriminate between the complementary input composite HSR image and the second cycle predicted composite HSR image; The inverse discriminator is configured to assess accuracy of a predictive mapping of the inverse generator using the inverse discriminator to discriminate between the complementary first predicted synthesized HSR image and the complementary input synthesized HSR image.

[0026] Preferably, The first cycle generative adversarial network (Cycle-GAN) is trained to match the first cycle predicted synthetic HSR image to the input synthetic HSR image by applying a first cycle consistency loss function (Cycle-Consistency Loss); The forward discriminator is trained to increase a probability of misidentifying the first predicted synthesized HSR image as the input synthesized HSR image by applying a forward adversarial loss function, The backward classifier is trained to increase the probability of misidentifying the first cycle predicted synthesized HSR image as the first predicted synthesized HSR image by applying a backward adversarial loss function.

[0027] Preferably, The second cycle generative adversarial network (Cycle-GAN) is trained to match the second cycle predicted synthetic HSR image to the complementary first predicted synthetic HSR image by applying a second cycle consistency loss function (Cycle-Consistency Loss); The forward discriminator is trained to increase a probability of misidentifying the second cycle predicted synthesized HSR image as the complementary input synthesized HSR image by applying the forward adversarial loss function; The backward discriminator is trained to increase the probability of misidentifying the complementary input synthesized HSR image as the complementary first predicted synthesized HSR image by applying a backward adversarial loss function.

[0028] Preferably, The cycle generative adversarial network (Cycle-GAN) outputs the first predicted synthesized HSR image or the complementary first predicted synthesized HSR image or the second cycle predicted synthesized HSR image as the resulting synthesized HSR image when the first pixel loss function and the second pixel loss function are trained to converge to a predetermined reference pixel loss function value and the first cycle consistency loss function and the second cycle consistency loss function are trained to converge to a predetermined cycle consistency loss function value.

[0029] Preferably, The forward generator and the backward generator are an encoding step of reducing a size of the input composite HSR image using an encoder and extracting a plurality of feature maps from the input composite HSR image; a recalibration step of recalibrating the feature map using a plurality of SE-residual blocks; and performing a decoding step of restoring the size of the input composite HSR image, which was reduced by the encoder, using a decoder.

[0030] Preferably, The forward classifier and the backward classifier perform classification on a patch-by-patch basis using a patch-generative adversarial network (Patch-GAN) architecture.

[0031] Other specific details of the embodiments are included in the detailed description and drawings. Effect of the Invention

[0032] The real-time regional precipitation forecast system and method based on the extension of cycle generative adversarial networks according to the present invention enables improved real-time precipitation forecasts in both qualitative and quantitative assessments.

[0033] The system and method for regional precipitation forecasting based on an extension of the cycle generative adversarial network according to the present invention can ensure robustness against temporal causality by applying pixel loss to the cycle generative adversarial network.

[0034] However, the effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. [Brief description of the drawings]

[0035] [Figure 1] FIG. 1 is a diagram illustrating a schematic diagram of a regional precipitation real-time forecast system based on an extension of a cycle generative adversarial network according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram illustrating a schematic diagram for training a cycle generative adversarial network of the present invention. [Diagram 3] FIG. 1 is a schematic diagram illustrating the generation of synthetic hybrid elevated surface rainfall (HSR) data of the present invention. [Figure 4] 1 is a diagram illustrating a network architecture of a forward generator and a backward generator of the present invention. [Diagram 5] 1 is a diagram illustrating a network architecture of a forward discriminator and a backward generator according to the present invention. [Figure 6] 1 is a diagram illustrating a schematic diagram of a method for forecasting actual precipitation amounts by region based on an extension of a cycle generative adversarial network according to an embodiment of the present invention. [Figure 7] 1 is a diagram showing the qualitative evaluation results of a regional precipitation real-time forecast system and method based on an extension of the cycle generative adversarial network of the present invention. [Figure 8]1 is a table showing the qualitative evaluation results of the regional precipitation real-time forecast system and method based on the extension of the cycle generative adversarial network of the present invention. [Figure 9] 1 illustrates an example computing device capable of implementing devices and / or systems according to various embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0036] The advantages and features of the present invention, and the method of achieving them, will become clear from the following detailed description of the embodiments in conjunction with the accompanying drawings. The present invention is not limited to the embodiments disclosed below, but may be embodied in various different forms, and the embodiments are provided to complete the disclosure of the present invention and to fully inform those skilled in the art of the present invention of the scope of the invention, and the present invention is defined only by the scope of the claims. The same reference numerals refer to the same elements throughout the specification.

[0037] The embodiments described herein will be described with reference to cross-sectional views and / or plan views which are ideal exemplary views of the present invention. In the drawings, the thickness of the structures is exaggerated for the effective explanation of the technical contents. Therefore, the structures illustrated in the drawings have schematic attributes, and the shapes of the structures illustrated in the drawings are intended to illustrate specific forms of the structures, and are not intended to limit the scope of the invention. In various embodiments of the present specification, terms such as first, second, and third are used to describe various components, but these components are not limited by such terms. These terms are merely used to distinguish a certain component from another component. The embodiments described and illustrated herein also include their complementary embodiments.

[0038] The terms used in the present specification are for the purpose of describing the embodiments and are not intended to limit the present invention. In the present specification, the singular form includes the plural form unless otherwise specified in the context. The terms "comprises" and / or "comprising" used in the specification do not exclude the presence or addition of one or more other components, steps, operations and / or elements to the referenced components, steps, operations and / or elements.

[0039] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in the sense commonly understood by those having ordinary skill in the art to which the present invention belongs. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless specifically and explicitly defined.

[0040] The concept of the present invention and embodiments according to the present invention will be described in detail below with reference to the drawings.

[0041] A system and method for regional precipitation forecasting based on an extension of a cycle generative adversarial network according to one embodiment of the present invention uses a cycle generative adversarial network (CycleGAN), which assumes that the relationship between the two domains is bijection. In particular, by extending and applying the cycle generative adversarial network in the time-series domain, it is possible to predict an image containing precipitation information in the future based on an image containing current precipitation information.

[0042] Cycle generative adversarial networks enable unpaired image-to-image translation using cycle consistency loss.

[0043] FIG. 1 is a diagram illustrating a schematic diagram of a regional precipitation real-time forecast system based on an extension of a cycle generative adversarial network according to an embodiment of the present invention.

[0044] A regional precipitation real-time forecast system 100 based on an extension of a cycle generative adversarial network according to one embodiment of the present invention includes an input unit 110, a cycle generative adversarial network (Cycle-GAN) 120, and an output unit 130.

[0045] An input 110 receives an input composite hybrid surface rainfall (HSR) image containing precipitation information for a first hour for a region of interest.

[0046] Cycle-Generative Adversarial Network (Cycle-GAN) 120 includes a first cycle generative adversarial network (Cycle-GAN) and a second cycle generative adversarial network complementary to the first cycle generative adversarial network.

[0047] A first cycle generative adversarial network (Cycle-GAN) and a second cycle generative adversarial network are utilized to generate a resultant synthetic HSR image containing precipitation information for a second time period in the region of interest based on the input synthetic HSR image.

[0048] The second time is a time in the future from the first time, and can be up to two hours in intervals of 10 minutes, for example.

[0049] The output unit 130 outputs the resulting composite HSR image as a nowcasting image of the area of ​​interest.

[0050] The first-cycle generative adversarial network and the second-cycle generative adversarial network can make temporal causality more robust by performing forward image mapping and backward image mapping, respectively, which go forward in time and backward in time.

[0051] FIG. 2 is a diagram illustrating a schematic diagram for training a cycle generative adversarial network of the present invention.

[0052] For a clear and concise explanation, we use the forward generator forward ) 223-1, 223-2, Reverse Generator (Generator backward )224-1, 224-2, forward discriminator (D i scr i m i nator(t i+step ))225, Reverse Discriminator(t i ))226 to G f , G b , D(t i+step ), D(t i ) is used. Two generators (G f and G b ) and two discriminators (D(t i+step ) and D(t i ) is learned through two cycles 221, 222. G f and G b The synthetic hybrid height surface precipitation (HSR) of the next time step and the previous time step are generated from the input HSR, respectively.

[0053] D(t i+step ) is G f Similarly, we distinguish the fake HSR generated by D(t i ) is G b Distinguish between fake HSRs generated by and genuine HSRs.

[0054] Cycle 1 221 is a genuine HSR (t i ) in one-step fake future HSR(t i+step ) fake Generate the fake future HSR(t i+step ) fake Then cycled again from HSR(t i ) cycled Generate.

[0055] TIFF2024546212000002.tif13166

[0056] Cycle 2 222 is a genuine HSR (ti+step ) from the previous HSR (t i ) fake Generate the fake past HSR(t i ) fake Then, cycle HSR(t i+step ) Cycled Generate.

[0057] TIFF2024546212000003.tif13166

[0058] The ultimate goal of the learning model illustrated in Figure 2 is to make the real, fake and cycle HSRs identical.

[0059] TIFF2024546212000004.tif20166

[0060] The first cycle generative adversarial network (Cycle-GAN) 221 includes a forward generator 223-1, a backward generator 224-1, a forward discriminator 225, and a backward discriminator 226.

[0061] A forward generator 223-1 learns a predictive mapping of a first predicted synthesized HSR image 252 at a second time based on an input synthesized HSR image 250 at a first time.

[0062] The second time is a time further in the future than the first time.

[0063] A backward generator 224-1 learns a predictive mapping of a first cycle predicted synthesized HSR image 254 at a first time based on a first predicted synthesized HSR image 252 at a second time.

[0064] A forward discriminator 225 evaluates the accuracy of the predictive mapping of the forward generator 223-1 and discriminates between the input synthesized HSR image 250 and the first predicted synthesized HSR image 252.

[0065] A backward discriminator 226 evaluates the accuracy of the predicted mapping of the backward generator 224 - 1 and discriminates between the first predicted synthesized HSR image 252 and the first cycle predicted synthesized HSR image 254 .

[0066] The second cycle generative adversarial network (Cycle-GAN) 222 includes a backward generator 224-2, a forward generator 223-2, a forward discriminator 225 and a backward discriminator 226.

[0067] The inverse generator 224-2 learns a predictive mapping of a complementary input synthesized HSR image 258 corresponding to the input synthesized HSR image 250 at the first time based on a complementary first predicted synthesized HSR image 256 obtained by applying a first pixel loss function (pixel loss forward) to a first predicted synthesized HSR image 252 at the second time. The input synthesized HSR image 250 and the complementary input synthesized HSR image 258 have a complementary relationship to which a second pixel loss function (pixel loss backward) is applied.

[0068] The forward generator 223-2 learns a predictive mapping of a second cycle predicted synthesized HSR image 260 at a second time based on a complementary input synthesized HSR image 258 at a first time.

[0069] The forward discriminator 225 evaluates the accuracy of the predicted mapping of the forward generator 223 - 2 and discriminates between the complementary input composite HSR image 258 and the second cycle predicted composite HSR image 260 .

[0070] The inverse discriminator 226 evaluates the accuracy of the predictive mapping of the inverse generator 224-2 and discriminates between a complementary first predicted combined HSR image 256 and a complementary input combined HSR image 258.

[0071] The forward generator 223-1 of the first cycle generative adversarial network (Cycle-GAN) 221 and the forward generator 223-2 of the second cycle generative adversarial network (Cycle-GAN) 222 of the present invention have the same network architecture.

[0072] The reverse generator 224-1 of the first cycle generative adversarial network (Cycle-GAN) 221 and the reverse generator 224-2 of the second cycle generative adversarial network (Cycle-GAN) 222 of the present invention have the same network architecture.

[0073] In one embodiment, the forward and backward generators of the present invention have the same network architecture.

[0074] The forward classifier 225 and the backward classifier 226 of the first cycle generative adversarial network (Cycle-GAN) 221 of the present invention and the forward classifier 225 and the backward classifier 226 of the second cycle generative adversarial network (Cycle-GAN) 222 of the present invention have the same network architecture.

[0075] In one embodiment, the forward and backward classifiers of the present invention have the same network architecture.

[0076] The first-cycle generative adversarial network (Cycle-GAN) 221 of the present invention is trained to match the first-cycle predicted synthesized HSR image 254 to the input synthesized HSR image 250 by applying a first-cycle consistency loss function.

[0077] The forward discriminator 225 is trained to increase the probability of misidentifying the first predicted synthesized HSR image 252 as the input synthesized HSR image 250 by applying a forward adversarial loss function.

[0078] The backward discriminator 226 is trained to increase the probability of misidentifying the first cycle predicted synthesized HSR image 254 as the first predicted synthesized HSR image 252 by applying a backward adversarial loss function.

[0079] A second-cycle generative adversarial network (Cycle-GAN) 222 is trained to match the second-cycle predicted synthetic HSR image 260 to the complementary first predicted synthetic HSR image 256 by applying a second-cycle consistency loss function (Cycle-Consistency Loss).

[0080] The forward discriminator 225 is trained to increase the probability of misidentifying the second cycle predicted synthesized HSR image 260 as the complementary input synthesized HSR image 258 by applying a forward adversarial loss function.

[0081] The backward discriminator 226 is trained to increase the probability of misidentifying the complementary input synthesized HSR image 258 as the complementary first predicted synthesized HSR image 256 by applying a backward adversarial loss function.

[0082] The cycle generative adversarial network (Cycle-GAN) 120 outputs a first predicted synthesized HSR image or a complementary first predicted synthesized HSR image or a second cycle predicted synthesized HSR image as a result synthesized HSR image when the first pixel loss function and the second pixel loss function are trained to converge to a predetermined reference pixel loss function value and the first cycle consistency loss function and the second cycle consistency loss function are trained to converge to a predetermined cycle consistency loss function value.

[0083] The output unit 130 outputs the resulting composite HSR image as a nowcasting image of the area of ​​interest.

[0084] As shown in Fig. 2, the learning model for real-time precipitation forecasting is composed of two cycles (Cycle 1 and Cycle 2) and uses synthetic hybrid height surface precipitation (HSR) as input data. As shown in Fig. 2, in the real-time regional precipitation forecasting system based on the extension of the cycle generative adversarial network of the present invention, a total of four different networks are trained in two cycles.

[0085] Generator forward ) and the reverse generator (Generator backward ) learns image mapping from the present to the future and from the future to the present, respectively. By performing image mapping, we predict images containing future precipitation information.

[0086] Forward discriminator (t i+step ) and the inverse discriminator (t i ) is a forward generator forward ) and the reverse generator (Generator backward ) to evaluate the mapping accuracy.

[0087] Input data HSR used as input for the model i is expressed as equation (1).

[0088]

number

[0089] TIFF2024546212000006.tif32166

[0090]

number

[0091] Here, the subscripts fake and cycled refer to data generated from real and fake, respectively. f and G b is the forward generator (Generator forward ) and the reverse generator (Generator backward ) is shown.

[0092] Cycle 2, where we predict the present with the known future and then predict the future again with the predicted present, can be written as equation (3):

[0093]

number

[0094] The variables in equation (3) are the same as those in equation (2). In addition, Cycle discriminator (t i+step ) and the reverse discriminator (t i ) are D(t i+step ) and D(t i ) is displayed.

[0095] TIFF2024546212000009.tif20166

[0096]

number

[0097] TIFF2024546212000011.tif20166

[0098]

number

[0099] The cycle generative adversarial network of the present invention is HSR i as input and convert it to HSR(t i ) real and HSR(t i+step ) real are distributed to cycle 1 and cycle 2, respectively. Through the complementary optimization of cycle 1 and cycle 2, the learning model can acquire strong temporal causality.

[0100] Our cycle generative adversarial network can be optimized by applying three types of loss functions: adversarial loss, cycle-consistency loss, and pixel loss.

[0101] The cycle generative adversarial network of the present invention is HSR(t i+step ) real and HSR(t i+step ) fake We apply an adversarial loss to match the distribution of .

[0102] Similarly, HSR(t i ) real and HSR(t i ) fake We also apply an adversarial loss to match the distribution of .

[0103] We adopt the LSGAN adversarial loss and extend it with two complementary cycles.

[0104] That is, the adversarial loss of the cycle generative adversarial network of the present invention is expressed as Equation (12) to Equation (15):

[0105]

number

[0106] Here, p(t i+step ) and p(t i ) are HSR(t i+step ) real and HSR(t i ) real The distribution of is shown.

[0107] The cycle generative adversarial network of the present invention applies the cycle consistency loss to the time series domain to make the temporal causal relationship robust. The cycle consistency loss is used to satisfy Equation (16) and Equation (17).

[0108]

number

[0109] Therefore, the cycle consistency loss function can be expressed as:

[0110]

number

[0111] Our cycle generative adversarial network applies pixel loss together with the adversarial loss to satisfy the following condition:

[0112]

number

[0113] TIFF2024546212000017.tif20166

[0114]

number

[0115] Here, p(t i ) and p(t i+step ) are HSR(t i ) real and HSR(t i+step ) real The distribution of is shown.

[0116] To improve performance, we apply identity loss, which is additionally adopted in conventional cycle generative adversarial networks (CycleGANs). However, identity loss cannot be applied in the present invention because it breaks temporal causality. For example, the prior generator (G f ) must always be a mapping from the present to the future, but if identity loss is added, it can also indicate a mapping from the present to the present. We add pixel loss instead of identity loss to obtain the forward generator (G f ) and the inverse generator (G b ) helps capture changes over time.

[0117] From equations (12)-(15), (18), (21), and (22), the overall objective function is:

[0118]

number

[0119] TIFF2024546212000020.tif26166

[0120]

number

[0121] The goal of equations (27) and (28) is to ensure temporal causality of cycle 1 and cycle 2.

[0122] FIG. 3 is a schematic diagram illustrating the generation of synthetic hybrid surface rainfall (HSR) data of the present invention.

[0123] As shown in Figure 3, the synthetic hybrid height surface precipitation (HSR) data is selected from the weather radar (310) at each horizontal distance to select the reflectivity of the beam that is closest to the ground and is less affected by non-weather echoes and beam blockages. Non-weather echoes include bird echoes, ground echoes, chaff echoes, etc., which are the main obstacles to precipitation estimation.

[0124] The collected reflectance data is converted into HSR data through a reflectance-rainfall equation that takes into account regional characteristics. The HSR data is currently used for weather forecasting in Korea.

[0125] FIG. 4 is a diagram illustrating the network architecture of a forward generator and a backward generator of the present invention.

[0126] In one embodiment, the forward and backward generators of the present invention have the same network architecture.

[0127] In one embodiment, the forward and backward classifiers of the present invention have the same network architecture.

[0128] A generator network 410 including a forward generator 223 and a backward generator 224 may be configured as shown in FIG.

[0129] The generator network 410 includes an encoder 421 , a number of SE-residual blocks 423 and a decoder 424 .

[0130] The encoder 421 reduces the size of the input composite HSR image and extracts a number of feature maps from the input composite HSR image.

[0131] The SE-residual block 423 may be composed of multiple blocks, for example, it may include 16 residual blocks. The SE-residual block 423 recalibrates the feature map. The SE-residual block is the core of the generator network, and is a combination of a residual block and an SE-block. For robustness, a dropout layer is added to the residual block. Since the present invention uses a small number of residual blocks, it does not use the bottleneck block due to the original ResNet design policy.

[0132] The decoder 424 restores the size of the input synthesized HSR image that was reduced by the encoder 421. The convolution layer just before the output is for fine resizing, and tanh is used as the final activation function.

[0133] FIG. 5 is a diagram illustrating the network architecture of a forward discriminator and a backward generator according to the present invention.

[0134] The forward and backward classifiers use a patch generative adversarial network (Patch-GAN) architecture that performs classification on a patch-by-patch basis.

[0135] The classifier network classifies the output data patch as real or fake. A patch-based classifier is lighter than a whole data classifier. In one embodiment, the classifier network of the present invention uses a patch size of 31 x 31. It is obvious to one skilled in the art that the patch size can be flexibly changed according to the size of the input data.

[0136] FIG. 6 is a diagram illustrating a schematic diagram of a method for forecasting actual precipitation amounts by region based on an extension of a cycle generative adversarial network according to an embodiment of the present invention.

[0137] Another embodiment of the present invention relates to a method for forecasting actual regional precipitation based on an extension of a cycle generative adversarial network (600), which includes an input data receiving step (S610) of receiving an input synthetic hybrid height surface precipitation (HSR) image including precipitation information for a first hour in a region of interest.

[0138] The method includes a cyclic generative adversarial network execution step (S620) for generating a resultant synthetic HSR image including precipitation information for a second time, which is a time in the future from the first time, in the region of interest based on the input synthetic HSR image.

[0139] The method includes an output step (S630) of outputting the resultant composite HSR image as a nowcasting image of the area of ​​interest.

[0140] The cycle generative adversarial network of the present invention includes a first cycle generative adversarial network (Cycle-GAN) and a second cycle generative adversarial network complementary to the first cycle generative adversarial network.

[0141] The first cycle GAN and the second cycle GAN of the GAN perform forward image mapping and backward image mapping, respectively.

[0142] First cycle generative adversarial network (Cycle-GAN) a forward image mapping step of learning a predictive mapping of a first predicted synthesized HSR image at a second time based on an input synthesized HSR image at a first time using a forward generator; a backward image mapping step of learning a predictive mapping of a first cycle predicted synthetic HSR image at a first time based on a first predicted synthetic HSR image at a second time using a backward generator; a forward discrimination step of evaluating the accuracy of the predictive mapping of the forward generator using a forward discriminator to discriminate between the input synthetic HSR image and the first predicted synthetic HSR image; and A reverse discrimination step is performed in which the reverse discriminator is used to evaluate the accuracy of the predicted mapping of the reverse generator and to discriminate between the first predicted synthetic HSR image and the first cycle predicted synthetic HSR image.

[0143] Cycle-GAN is a second-cycle generative adversarial network. A prediction mapping of a complementary input synthetic HSR image corresponding to an input synthetic HSR image at a first time is learned based on a complementary first predicted synthetic HSR image in which a first pixel loss function is applied to a first predicted synthetic HSR image at a second time using a reverse generator, and the input synthetic HSR image and the complementary input synthetic HSR image maintain a complementary relationship in which a second pixel loss function is applied;

[0144] learning a predictive mapping of a second cycle predicted synthesized HSR image at a second time based on the complementary input synthesized HSR image at a first time using a forward generator; evaluating the accuracy of the predicted mapping of the forward generator using a forward discriminator to discriminate between the complementary input composite HSR image and the second cycle predicted composite HSR image; The inverse discriminator is configured to assess the accuracy of the predictive mapping of the inverse generator using the inverse discriminator to discriminate between the complementary first predicted synthesized HSR image and the complementary input synthesized HSR image.

[0145] A first-cycle generative adversarial network (Cycle-GAN) is trained to match the first-cycle predicted synthetic HSR image to the input synthetic HSR image by applying a first-cycle consistency loss function.

[0146] The forward discriminator is trained to increase the probability of misidentifying the first predicted synthesized HSR image as the input synthesized HSR image by applying a forward adversarial loss function.

[0147] The backward discriminator is trained to increase the probability of misidentifying the first-cycle predicted synthetic HSR image as the first predicted synthetic HSR image by applying a backward adversarial loss function.

[0148] A 2-cycle generative adversarial network (Cycle-GAN) is trained to match the 2-cycle predicted synthetic HSR image to the complementary 1-cycle predicted synthetic HSR image by applying a 2-cycle consistency loss function (Cycle-Consistency Loss).

[0149] The forward discriminator is trained to increase the probability of misidentifying the second-cycle predicted synthetic HSR image as the complementary input synthetic HSR image by applying a forward adversarial loss function.

[0150] The backward discriminator is trained to increase the probability of misidentifying the complementary input synthesized HSR image as the complementary first predicted synthesized HSR image by applying a backward adversarial loss function.

[0151] The cycle generative adversarial network (Cycle-GAN) outputs a first predicted synthesized HSR image or a complementary first predicted synthesized HSR image or a second cycle predicted synthesized HSR image as a result synthesized HSR image when the first pixel loss function and the second pixel loss function are trained to converge to a predetermined reference pixel loss function value and the first cycle consistency loss function and the second cycle consistency loss function are trained to converge to a predetermined cycle consistency loss function value.

[0152] The forward generator and the backward generator of the Cycle Generative Adversarial Network (Cycle-GAN) of the present invention are: an encoding step of reducing a size of the input composite HSR image using an encoder and extracting a plurality of feature maps from the input composite HSR image; A recalibration phase in which multiple SE-residual blocks are used to recalibrate the feature maps; The decoder is configured to perform a decoding step of restoring the size of the input composite HSR image that was reduced by the encoder using a decoder.

[0153] The forward classifier and backward classifier of the Cycle Generative Adversarial Network (Cycle-GAN) of the present invention are characterized by performing classification on a patch-by-patch basis using the architecture of the Patch Generative Adversarial Network (Patch-GAN).

[0154] FIG. 7 is a diagram showing the qualitative evaluation results of the regional precipitation real-time forecast system and method based on the extension of the cycle generative adversarial network of the present invention.

[0155] Figure 7 compares our method with two existing models for qualitative evaluation. The first is MAPLE, a QPF model. The second is ConvLSTM, which is based on RNN.

[0156] Both models are currently used in weather radar operation systems and are well-tested methods in real life.

[0157] We use CSI as a metric for qualitative evaluation.

[0158] As shown in FIG. 7, in the experimental results of the CSI-based qualitative evaluation, the regional precipitation real-time forecast method based on the extension of the cycle generative adversarial network of the present invention secured the highest score at all time points.

[0159] It can be seen that the CSI of the method of the present invention remained relatively uniform over a 2-hour lead time compared to the CSI of the two existing models.

[0160] In particular, the regional precipitation real-time forecast method based on the extension of the cycle generative adversarial network of the present invention predicted the disappearance of radar echo better than MAPLE and predicted the details of radar echo better than ConvLSTM.

[0161] FIG. 8 is a table showing the qualitative evaluation results of the regional precipitation real-time forecast system and method based on the extension of the cycle generative adversarial network of the present invention.

[0162] Looking at the table in Figure 8, we use PSNR and SSIM as metrics for quantitative evaluation.

[0163] As shown in Table 1, the proposed method achieved the best scores in all areas of PSNR and SSIM except for the first 10 minutes of prediction.

[0164] 7 and 8, it can be seen that the regional precipitation real-time forecast system and method based on the extension of the cycle generative adversarial network of the present invention has higher CSI, PSNR and SSIM than the conventional method.

[0165] FIG. 9 illustrates an example computing device capable of implementing devices and / or systems according to various embodiments of the present invention.

[0166] With reference to FIG. 9, an example computing device 900 capable of implementing apparatus according to some embodiments of the present disclosure will now be more particularly described.

[0167] The computing device 900 may include one or more processors 910, a bus 950, a communication interface 970, a memory 930 for loading computer programs 991 executed by the processor 910, and a storage 990 for storing the computer programs 991. However, only components relevant to the embodiments of the present disclosure are shown in FIG.

[0168] Therefore, a person of ordinary skill in the art to which the present disclosure pertains would understand that other general-purpose components may be included in addition to the components illustrated in FIG.

[0169] The processor 910 controls the overall operation of each component of the computing device 900. The processor 910 may include a central processing unit (CPU), a micro processor unit (MPU), a micro controller unit (MCU), a graphic processing unit (GPU), or any other type of processor commonly known in the technical field of the present disclosure. The processor 910 may also perform calculations for at least one application or program for executing a method according to an embodiment of the present disclosure. The computing device 900 may include one or more processors. The computing device 900 may indicate artificial intelligence (AI).

[0170] The memory 930 stores various data, instructions, and / or information. The memory 930 can load one or more programs 991 from the storage 990 to execute a method according to an embodiment of the present disclosure. The memory 930 can be embodied as a volatile memory such as a RAM, although the technical scope of the present disclosure is not limited in this respect.

[0171] The bus 950 provides a communication function between the components of the computing device 900. The bus 950 may be implemented with various types of buses such as an address bus, a data bus, and a control bus.

[0172] The communication interface 970 supports wired / wireless Internet communication of the computing device 900. The communication interface 970 may also support various communication methods other than Internet communication. To this end, the communication interface 970 may be configured to include a communication module that is widely known in the technical field of the present disclosure.

[0173] According to some embodiments, the communications interface 970 may be omitted.

[0174] The storage 990 can non-temporarily store one or more programs 991 and various data.

[0175] Storage 990 may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, etc., a hard disk, a removable disk, or any form of computer-readable recording medium commonly known in the technical field to which this disclosure pertains.

[0176] The computer program 991, when loaded into the memory 930, may include one or more instructions that cause the processor 910 to perform methods / operations according to various embodiments of the present disclosure. That is, the processor 910 may execute the one or more instructions to perform methods / operations according to various embodiments of the present disclosure.

[0177] Although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and it goes without saying that various modifications can be made by those having ordinary skill in the art to which the invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical ideas and perspectives of the present invention.

Claims

1. an input for receiving an input composite hybrid surface rainfall (HSR) image including precipitation information for a first hour for a region of interest; a cycle generative adversarial network (Cycle-GAN) for generating a resultant synthetic HSR image based on the input synthetic HSR image, the resultant synthetic HSR image including precipitation information for a second time, the second time being a future time relative to the first time, in the area of ​​interest; and an output unit for outputting the resulting composite HSR image as a nowcasting image of the area of ​​interest. A regional precipitation real-time forecast system based on an extension of cycle generative adversarial networks.

2. The cycle generative adversarial network (Cycle-GAN) is A first cycle generative adversarial network (Cycle-GAN) and a second cycle generative adversarial network complementary to the first cycle generative adversarial network; The first cycle generative adversarial network and the second cycle generative adversarial network perform forward image mapping and backward image mapping, respectively. A regional precipitation real-time forecast system based on the cycle generative adversarial network extension of claim 1.

3. The first cycle generative adversarial network (Cycle-GAN) a forward generator for learning a predictive mapping of a first predicted synthesized HSR image at the second time based on the input synthesized HSR image at the first time; a backward generator that learns a predictive mapping of the first cycle predicted synthesized HSR image at the first time based on the first predicted synthesized HSR image at the second time; a forward discriminator that evaluates the accuracy of the predictive mapping of the forward generator and discriminates between the input synthesized HSR image and the first predicted synthesized HSR image; and a backward discriminator for evaluating the accuracy of the prediction mapping of the backward generator and discriminating between the first predicted synthesized HSR image and the first cycle predicted synthesized HSR image; A regional precipitation real-time forecast system based on the cycle generative adversarial network extension of claim 2.

4. The second cycle generative adversarial network (Cycle-GAN) is A prediction mapping of a complementary input synthesized HSR image corresponding to the input synthesized HSR image at the first time is learned based on a complementary first predicted synthesized HSR image obtained by applying a first pixel loss function to the first predicted synthesized HSR image at the second time using the inverse generator, and the input synthesized HSR image and the complementary input synthesized HSR image have a complementary relationship to which a second pixel loss function is applied; learning a predictive mapping of a second cycle predicted synthesized HSR image at a second time based on the complementary input synthesized HSR image at the first time using the forward generator; evaluating an accuracy of a predictive mapping of the forward generator using the forward discriminator to discriminate between the complementary input synthesized HSR image and the second cycle predicted synthesized HSR image; and a second input combined HSR image generating unit configured to generate a combined HSR image based on the first combined HSR image and a second input combined HSR image based on the first combined HSR image. A regional precipitation real-time forecast system based on the cycle generative adversarial network extension of claim 3.

5. The first-cycle generative adversarial network (Cycle-GAN) is trained to match the first-cycle predicted synthesized HSR image to the input synthesized HSR image by applying a first-cycle consistency loss function (Cycle-Consistency Loss); The forward discriminator is trained to increase a probability of misidentifying the first predicted synthesized HSR image as the input synthesized HSR image by applying a forward adversarial loss function; The backward classifier is trained to increase the probability of misidentifying the first cycle predicted synthetic HSR image as the first predicted synthetic HSR image by applying a backward adversarial loss function. A regional precipitation real-time forecast system based on the cycle generative adversarial network extension of claim 4.

6. The second cycle generative adversarial network (Cycle-GAN) is trained to match the second cycle predicted synthesized HSR image to the complementary first predicted synthesized HSR image by applying a second cycle consistency loss function (Cycle-Consistency Loss); The forward discriminator is trained to increase a probability of misidentifying the second cycle predicted synthesized HSR image as the complementary input synthesized HSR image by applying the forward adverse loss function; The backward classifier is trained to increase the probability of misidentifying the complementary input synthesized HSR image as the complementary first predicted synthesized HSR image by applying a backward adversarial loss function. A regional precipitation real-time forecast system based on the cycle generative adversarial network extension of claim 5.

7. The cycle generative adversarial network (Cycle-GAN) outputs the first predicted synthesized HSR image or the complementary first predicted synthesized HSR image or the second cycle predicted synthesized HSR image as the result synthesized HSR image when the first pixel loss function and the second pixel loss function are trained to converge to a predetermined reference pixel loss function value and the first cycle consistency loss function and the second cycle consistency loss function are trained to converge to a predetermined cycle consistency loss function value. A regional precipitation real-time forecast system based on the cycle generative adversarial network extension of claim 6.

8. The forward generator and the backward generator are an encoder for reducing a size of the input composite HSR image and extracting a plurality of feature maps from the input composite HSR image; A number of SE-residual blocks for recalibrating the feature maps; a decoder for restoring the size of the input composite HSR image that was reduced by the encoder; A regional precipitation real-time forecast system based on the cycle generative adversarial network extension of claim 3.

9. The forward classifier and the backward classifier use a patch-generative adversarial network (Patch-GAN) architecture that performs classification on a patch-by-patch basis. A regional precipitation real-time forecast system based on the cycle generative adversarial network extension of claim 8.

10. receiving an input data receiving step of receiving an input synthetic hybrid surface rainfall (HSR) image including precipitation information for a first time period for a region of interest; performing a cyclic generative adversarial network to generate a resultant synthetic HSR image based on the input synthetic HSR image, the resultant synthetic HSR image including precipitation information for a second time, the second time being a time in the future of the first time, in the area of ​​interest; and outputting the resulting composite HSR image as a nowcasting image of the area of ​​interest. A method for forecasting regional precipitation amounts based on an extension of a cycle generative adversarial network, characterized by:

11. The cycle generative adversarial network is a first cycle generative adversarial network (Cycle-GAN) and a second cycle generative adversarial network complementary to the first cycle generative adversarial network; The first cycle generative adversarial network and the second cycle generative adversarial network perform forward image mapping and backward image mapping, respectively. A method for real-time regional precipitation forecast based on the cycle generative adversarial network extension of claim 10.

12. The first cycle generative adversarial network (Cycle-GAN) a forward image mapping step of learning a predictive mapping of a first predicted synthesized HSR image at a second time based on the input synthesized HSR image at the first time using a forward generator; a backward image mapping step of learning a predictive mapping of the first cycle predicted synthesized HSR image at the first time based on the first predicted synthesized HSR image at the second time using a backward generator; a forward discrimination step of evaluating the accuracy of the prediction mapping of the forward generator using a forward discriminator to discriminate between the input synthesized HSR image and the first predicted synthesized HSR image; and a reverse discrimination step of evaluating the accuracy of the prediction mapping of the reverse generator using a reverse discriminator to discriminate between the first predicted synthetic HSR image and the first cycle predicted synthetic HSR image; A method for real-time regional precipitation forecast based on the cycle generative adversarial network extension of claim 11.

13. The second cycle generative adversarial network (Cycle-GAN) is A prediction mapping of a complementary input synthesized HSR image corresponding to the input synthesized HSR image at the first time is learned based on a complementary first predicted synthesized HSR image obtained by applying a first pixel loss function to the first predicted synthesized HSR image at the second time using the inverse generator, and the input synthesized HSR image and the complementary input synthesized HSR image maintain a complementary relationship to which a second pixel loss function is applied; learning a predictive mapping for a second cycle predicted synthesized HSR image at a second time based on the complementary input synthesized HSR image at the first time using the forward generator; evaluating an accuracy of a predictive mapping of the forward generator using the forward discriminator to discriminate between the complementary input synthesized HSR image and the second cycle predicted synthesized HSR image; and a second input combined HSR image generating unit configured to generate a combined HSR image based on the first combined HSR image and a second input combined HSR image based on the first combined HSR image. A method for real-time regional precipitation forecasting based on the cycle generative adversarial network extension of claim 12.

14. The first-cycle generative adversarial network (Cycle-GAN) is trained to match the first-cycle predicted synthesized HSR image to the input synthesized HSR image by applying a first-cycle consistency loss function (Cycle-Consistency Loss); The forward discriminator is trained to increase a probability of misidentifying the first predicted synthesized HSR image as the input synthesized HSR image by applying a forward adversarial loss function; The backward classifier is trained to increase the probability of misidentifying the first cycle predicted synthetic HSR image as the first predicted synthetic HSR image by applying a backward adversarial loss function. A method for real-time regional precipitation forecast based on the cycle generative adversarial network extension of claim 13.

15. The second cycle generative adversarial network (Cycle-GAN) is trained to match the second cycle predicted synthesized HSR image to the complementary first predicted synthesized HSR image by applying a second cycle consistency loss function (Cycle-Consistency Loss); The forward discriminator is trained to increase a probability of misidentifying the second cycle predicted synthesized HSR image as the complementary input synthesized HSR image by applying the forward adverse loss function; The backward classifier is trained to increase the probability of misidentifying the complementary input synthesized HSR image as the complementary first predicted synthesized HSR image by applying a backward adversarial loss function. A method for real-time regional precipitation forecast based on the cycle generative adversarial network extension of claim 14.

16. The cycle generative adversarial network (Cycle-GAN) outputs the first predicted synthesized HSR image or the complementary first predicted synthesized HSR image or the second cycle predicted synthesized HSR image as the result synthesized HSR image when the first pixel loss function and the second pixel loss function are trained to converge to a predetermined reference pixel loss function value and the first cycle consistency loss function and the second cycle consistency loss function are trained to converge to a predetermined cycle consistency loss function value. A method for real-time regional precipitation forecasting based on the cycle generative adversarial network extension of claim 15.

17. The forward generator and the backward generator are an encoding step of reducing a size of the input composite HSR image using an encoder and extracting a plurality of feature maps from the input composite HSR image; a recalibration step of recalibrating the feature map using a plurality of SE-residual blocks; a decoding step of restoring the size of the input composite HSR image reduced by the encoder using a decoder; A method for real-time regional precipitation forecasting based on the cycle generative adversarial network extension of claim 12.

18. The forward classifier and the backward classifier perform classification on a patch-by-patch basis using a patch-generative adversarial network (Patch-GAN) architecture. A method for real-time regional precipitation forecast based on the cycle generative adversarial network extension of claim 17.

Images