A method and system for generating virtual cloud maps based on interaction assumptions and deep learning

Abstract

The application discloses a method and system for generating virtual cloud images based on interactive assumption and deep learning, and the method comprises the following steps: constructing a training data set according to historical weather phenomenon observation data; constructing a virtual cloud image generation model based on a deep generative adversarial network according to the training data set and training the virtual cloud image generation model to obtain a trained virtual cloud image generation model; designing an assumed weather scene based on a human-computer interaction mode; taking a customized cloud amount product with a weather phenomenon identifier as an input of the trained virtual cloud image generation model, reconstructing a virtual satellite cloud image under the assumed weather scene and taking the virtual satellite cloud image as a result output. The application realizes the purpose of generating virtual cloud images based on interactive assumption and deep learning.

Description

Technical Field

[0001] This invention belongs to the field of meteorological technology, and in particular relates to a method and system for generating virtual cloud maps based on interactive scenarios and deep learning. Background Technology

[0002] With the rapid development of satellite remote sensing technology, satellite cloud images are playing an increasingly important role in weather analysis and forecasting. Satellite cloud images allow for the analysis of cloud structure and activity patterns at different scales. Furthermore, cloud image data is highly intuitive and easy to understand, making it crucial data for weather forecasters analyzing weather phenomena. In the simulation of weather processes by meteorological professionals and users of meteorological information, it is necessary to generate corresponding virtual meteorological information based on pre-determined meteorological scenarios. Previously, due to a lack of technical means, pre-determined virtual meteorological information could only be labeled with abstract symbols. This abstractly labeled meteorological information lacks intuitiveness and cannot be directly used for training weather forecasters or as input data for meteorological information systems. Therefore, obtaining highly realistic meteorological data based on pre-determined meteorological scenarios through human-computer interaction is a work of great practical significance.

[0003] In the process of weather process simulation by meteorological professionals and users of meteorological information, it is necessary to generate corresponding virtual meteorological information based on pre-determined meteorological scenarios. Due to a lack of technical means, the pre-determined virtual meteorological information can only be marked with abstract symbols. This abstractly marked meteorological information lacks intuitiveness and cannot be directly used for training meteorologists or as input data for meteorological information systems. Summary of the Invention

[0004] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a method and system for generating virtual cloud maps based on interactive scenarios and deep learning, aiming to overcome the deficiencies of existing weather scenario projection and annotation methods.

[0005] To address the aforementioned technical problems, this invention discloses a method for generating virtual cloud maps based on interactive scenario design and deep learning, comprising:

[0006] A training dataset was constructed based on historical weather phenomenon observation data;

[0007] Based on the training dataset, a virtual cloud map generation model based on a deep generative adversarial network is constructed and trained to obtain the trained virtual cloud map generation model.

[0008] Based on human-computer interaction, a hypothetical weather scenario is set up;

[0009] Customized cloud cover products with weather phenomenon labels are used as input to the trained virtual cloud image generation model to reconstruct virtual satellite cloud images under a hypothetical weather scenario and output the result.

[0010] In the aforementioned method for generating virtual cloud maps based on interactive scenarios and deep learning, a training dataset is constructed based on historical weather phenomenon observation data, including:

[0011] Acquire historical weather phenomenon observation data;

[0012] Historical weather phenomenon observation data are organized and analyzed to determine the type, location, and extent of historical weather phenomena, and these phenomena are marked with color blocks or weather phenomenon examples on the corresponding cloud cover products; at the same time, real satellite cloud images of the corresponding time and region are collected and organized.

[0013] A training dataset is constructed based on the identification results of the types, locations, and extents of historical weather phenomena on the corresponding cloud cover products and the collected and organized real satellite cloud images of the corresponding times and regions.

[0014] In the above method for generating virtual cloud maps based on interactive scenarios and deep learning, a hypothetical weather scenario is set based on human-computer interaction, including:

[0015] Custom modifications are made to the cloud volume products corresponding to the basic satellite cloud image through human-computer interaction, and color blocks or weather phenomenon examples are used to mark the corresponding cloud volume products to complete the setting of the desired weather scenario.

[0016] In the above-mentioned method for generating virtual cloud maps based on interactive scenarios and deep learning, cloud quantity products refer to the distribution of clouds in a specific area, reflecting the amount of clouds; the types of weather phenomena include: typhoons, thunderstorms, heavy precipitation, hail and fog.

[0017] In the above method for generating virtual cloud maps based on interactive scenarios and deep learning, the color of the color block is used to represent the intensity characteristics of the weather phenomenon, and the position of the color block is used to represent the influence range of the weather phenomenon; the contour map of the weather phenomenon example is used to represent the intensity characteristics and influence range of the weather phenomenon.

[0018] In the above-mentioned method for generating virtual cloud maps based on interactive scenario and deep learning, the satellite cloud map is a combination of specific visible light channel data, water vapor channel data, infrared channel data, or multiple channel data from meteorological satellites within the same area as the cloud cover product.

[0019] In the above-mentioned method for generating virtual cloud maps based on interactive scenarios and deep learning, the virtual cloud map generation model based on deep generative adversarial networks includes: a generator network G and a discriminator network D; wherein, the generator network G is composed of an encoder-decoder network as the backbone network, and the discriminator network D is composed of a binary classification network composed of multiple convolutional neural network layers as the backbone network.

[0020] In the above method for generating virtual cloud maps based on interactive scenarios and deep learning, when training the virtual cloud map generation model based on a deep generative adversarial network, we have:

[0021] Based on the training dataset, determine the identification results of the types, locations and extents of historical weather phenomena on the corresponding cloud cover products and the corresponding real satellite cloud images of the time and region;

[0022] The identification results of the type, location and extent of historical weather phenomena on the corresponding cloud cover products and the corresponding real satellite cloud images of time and region are used as inputs to the generator network G;

[0023] The identification network D takes the historical weather phenomenon type, location and extent as the labeling results on the corresponding cloud cover product, the corresponding time and region real satellite cloud image, and the virtual satellite cloud image generated by the generation network G as inputs;

[0024] The virtual cloud map generation model is obtained by iterative training.

[0025] In the above method for generating virtual cloud maps based on interactive scenarios and deep learning, the iterative training process is as follows:

[0026] Input the historical weather phenomenon type, location and extent identification results on the corresponding cloud cover product, the corresponding time and region real satellite cloud image and the virtual satellite cloud image generated by the generator network G into the discrimination network D in batches; and update the parameters of the discrimination network D by backpropagation using the discrimination network loss function L_D.

[0027] Freeze the parameters of the discrimination network D;

[0028] Input the historical weather phenomenon type, location and range identification results on the corresponding cloud cover product and the corresponding time and region real satellite cloud image into the generator network G in batches, and update the parameters of the generator network G through backpropagation using the generator network loss function L_G;

[0029] Repeat the above process until the abilities of the generator network G and the discriminator network D reach a balance, obtain the parameters of the virtual cloud map generation model at runtime, and then obtain the trained virtual cloud map generation model.

[0030] Accordingly, the present invention also discloses a system for generating virtual cloud maps based on interactive scenarios and deep learning, comprising:

[0031] The training dataset building module is used to build a training dataset based on historical weather phenomenon observation data.

[0032] The model building module is used to build and train a virtual cloud map generation model based on a deep generative adversarial network according to the training dataset, so as to obtain the trained virtual cloud map generation model.

[0033] The design module is used to design hypothetical weather scenarios based on human-computer interaction.

[0034] The generation module is used to take a customized cloud cover product with weather phenomenon labels as input to the trained virtual cloud image generation model, reconstruct a virtual satellite cloud image under a hypothetical weather scenario, and output the result.

[0035] The present invention has the following advantages:

[0036] This invention generates highly realistic virtual satellite cloud image products through interactive scenario planning and deep learning, thereby supporting cloud image recognition training for professional weather forecasters, pre-setting and extrapolating meteorological scenarios based on cloud images, and inputting simulated cloud image data into meteorological information systems. It overcomes the shortcomings of existing weather scenario planning and extrapolation labeling methods and can serve as a foundation and prerequisite method for generating various virtual meteorological data. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating the steps of a method for generating virtual cloud maps based on interactive scenario and deep learning in an embodiment of the present invention.

[0038] Figure 2 This is a schematic diagram of the VCloudGAN architecture, a generative adversarial network model for generating virtual satellite cloud images based on interactive scenarios and deep learning, in an embodiment of the present invention.

[0039] Figure 3 This is a schematic diagram comparing the effects of a visible light simulated satellite cloud image and an actual cloud image product in an embodiment of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments disclosed in the present invention will be described in further detail below with reference to the accompanying drawings.

[0041] To simulate natural environment scenarios and train weather forecasters, it is necessary to interactively generate virtual meteorological data according to a predetermined scenario. Satellite cloud imagery is an important type of meteorological data, and related research shows that satellite cloud imagery, combined with other data, can be used to generate virtual weather radar images and other virtual meteorological data.

[0042] Among existing technical methods, generative adversarial network (GAN) models in the field of deep learning have been a research direction in the field of artificial intelligence in recent years. By adversarial between the generator and the discriminator, the parameters of the generator and the discriminator are trained and optimized, thereby discovering the data mapping relationship from domain A to domain B and realizing the transformation from domain A to domain B. It has a wide range of applications in the field of image generation and can generate images with high simulation accuracy.

[0043] Extensive research has revealed that by constructing a training dataset consisting of cloud cover products, weather phenomenon identifiers, and corresponding time-region satellite cloud images, and then establishing and training a deep learning model for virtual cloud images, the data mapping relationship between cloud cover, weather phenomenon identifiers, and corresponding satellite cloud images can be identified. In actual meteorological scenario simulations, cloud cover products and weather phenomenon identifiers are pre-edited by a human-computer interaction according to the hypothetical meteorological scenario. Based on the deep learning model for virtual cloud images, virtual satellite cloud image products corresponding to the hypothetical meteorological scenario can be generated. This theory and method may be applied to the generation of virtual satellite cloud images based on scenarios.

[0044] like Figure 1 This invention proposes a method for generating virtual cloud maps based on interactive scenario and deep learning, which includes the following steps:

[0045] Step 101: Construct a training dataset based on historical weather phenomenon observation data.

[0046] In this embodiment, firstly, historical weather phenomenon observation data is acquired; then, the historical weather phenomenon observation data is organized and analyzed to determine the type, location, and extent of historical weather phenomena, and these phenomena are marked using color blocks or weather phenomenon examples on the corresponding cloud cover products; simultaneously, real satellite cloud images of the corresponding time and region are collected and organized; finally, a training dataset is constructed based on the marking results of the type, location, and extent of historical weather phenomena on the corresponding cloud cover products and the collected and organized real satellite cloud images of the corresponding time and region.

[0047] It should be noted that cloud cover products are meteorological products describing the amount of cloud cover in a specific area. They indicate the distribution of clouds in a particular region and reflect the amount of clouds. Cloud cover can be obtained through observation, satellite radar retrieval, numerical model products, etc. Typically, the cloud cover value for a single grid point is represented using integers from 0 to 10 or decimals from 0.0 to 1.0. Weather phenomena are acquired and published by meteorological agencies through their operational observation work. Commonly used weather phenomena in meteorology include, but are not limited to, typhoons, thunderstorms, heavy precipitation, hail, and fog, among other observable weather phenomena. Data published by various agencies typically includes parameters such as the location, affected area, and intensity level of specific weather phenomena. When using color blocks to identify weather phenomena on corresponding cloud cover products, the color of the color block can be used to characterize the intensity of the weather phenomenon, and the position of the color block can be used to characterize the affected area. When using weather phenomenon examples to identify weather phenomena on corresponding cloud cover products, the outline of the weather phenomenon example can be used to characterize the intensity and affected area of ​​the weather phenomenon. Satellite cloud images are data from meteorological satellites in the same region as cloud cover products, including specific visible light channel data, water vapor channel data, infrared channel data, or a combination of multiple channels.

[0048] In addition, the collected satellite cloud image data, cloud cover products, and weather phenomenon observation data may be misaligned in terms of time and region. In order to realize the virtual satellite cloud image product generated by cloud cover products and weather phenomenon label products during the model building process and to train the discrimination network, it is necessary to collect cloud cover products, weather phenomenon labels, and satellite cloud image data of the same time and region.

[0049] Step 102: Based on the training dataset, construct and train a virtual cloud map generation model based on a deep generative adversarial network to obtain the trained virtual cloud map generation model.

[0050] In this embodiment, the virtual satellite cloud image generation model based on deep generative adversarial networks mainly includes a generator network G and a discriminator network D. The generator network G consists of an encoder-decoder network and other network modules as its backbone, while the discriminator network D consists of a binary classification network composed of multiple convolutional neural network layers as its backbone. The encoder-decoder network can use the U-Net network structure, which is an encoder-decoder network with added skip connections and is widely used in image segmentation. The discriminator network D is a multi-layer convolutional classification network that calculates the probability that the input image is a real satellite cloud image and distinguishes between a real satellite cloud image and a generated virtual satellite cloud image. During training, the discriminator network D attempts to correctly distinguish between real and virtual satellite cloud images, while the generator network G attempts to generate cloud images that are as realistic as possible so that the discriminator network D cannot distinguish between them. Figure 2 As shown, the virtual cloud map generation model can transform the numerical model product on the left into a simulated cloud map product for a specific channel on the right and output it; the generation network G corresponds to... Figure 2 The generator in the middle identifies the network D corresponding to Figure 2 The discriminator in the middle.

[0051] Preferably, when training the virtual cloud image generation model based on a deep generative adversarial network, the following steps are taken: Based on the training dataset, determine the identification results of the types, locations, and extents of historical weather phenomena on the corresponding cloud cover products and the corresponding real satellite cloud images for the time and region; use the identification results of the types, locations, and extents of historical weather phenomena on the corresponding cloud cover products and the corresponding real satellite cloud images for the time and region as input to the generator network G; use the identification results of the types, locations, and extents of historical weather phenomena on the corresponding cloud cover products, the corresponding real satellite cloud images for the time and region, and the virtual satellite cloud image generated by the generator network G as input to the discriminator network D, so that the cloud cover products and weather phenomenon identifications participate in the calculation process of the discriminator network; perform iterative training to obtain the trained virtual cloud image generation model.

[0052] Furthermore, the iterative training process is as follows: Input the historical weather phenomenon type, location, and extent identification results on the corresponding cloud cover product, the corresponding time and region real satellite cloud image, and the virtual satellite cloud image generated by the generator network G into the discrimination network D in batches. Update the parameters of the discrimination network D using the discrimination network loss function L_D through backpropagation; freeze the parameters of the discrimination network D; input the historical weather phenomenon type, location, and extent identification results on the corresponding cloud cover product, and the corresponding time and region real satellite cloud image into the generator network G in batches. Update the parameters of the generator network G using the generator network loss function L_G through backpropagation; repeat the above process until the capabilities of the generator network G and the discrimination network D reach a balance (i.e., there is no significant optimization in either the generator network or the discrimination network during further iterations), obtain the parameters of the virtual cloud image generation model during runtime, and then obtain the trained virtual cloud image generation model.

[0053] Furthermore, binary cross-entropy can be used as a loss metric.

[0054] For the discriminant network D, the loss function (discriminant network loss function L_D) is as follows:

[0055] L_D=L bce (D(x,y),1)+L bce (D(x,G(x,z)),0)…(1)

[0056] Where x represents the combined data of cloud cover products and weather phenomenon identifiers, y represents the corresponding real satellite cloud image, G(x,z) represents the virtual satellite cloud image generated by the generator network G from x and random noise z, and D(x,y) represents the probability of the discriminator network D identifying real and fake satellite cloud images.

[0057] For the generator network G, the loss function (generator network loss function c, L_G) is as follows:

[0058] L_G=λ1L bce (D(x,G(x,z)),1)+λ2|yG(x,z)|…(2)

[0059] Where λ1 and λ2 represent two proportional parameters, respectively.

[0060] in:

[0061]

[0062] …(3)

[0063] Where N is the number of samples in a batch of model input; a∈{0,1} represents the label of the input data, 0 represents simulated satellite cloud image; 1 represents real satellite cloud image; It is the discrimination value of the output of the discrimination network D. If the value is close to 0, the discrimination network has a high probability that the input is a virtual satellite cloud image. If the value is close to 1, the network has a high probability that the input is a real satellite cloud image.

[0064] In addition, since the resolutions of cloud cover products, weather phenomenon labels, and meteorological satellite cloud image products are not necessarily consistent, an upsampling module needs to be added to the generative adversarial network model to unify the resolution of the input data. The model is then trained adversarially on the training dataset according to the common training mode of generative adversarial networks to obtain model parameters for extracting relevant information between cloud cover products, weather phenomenon labels, and satellite cloud image products.

[0065] Step 103: Based on human-computer interaction, set up a hypothetical weather scenario.

[0066] In this embodiment, cloud cover products corresponding to basic satellite cloud images can be customized and modified using a graphical editing tool in a human-computer interactive manner. Color blocks or weather phenomenon examples can be used to mark these cloud cover products (using the same weather phenomenon marking method as in step 101 with the training dataset) to complete the setting of the desired weather scenario. For example, modifying cloud cover products can involve converting them into image format, and using existing or self-developed image editing tools to perform operations such as deleting, copying, pasting, translating, rotating, scaling, dodging, and enhancing the content in the images corresponding to the cloud cover products. Weather phenomenon markers can be placed as needed, moved to the required positions, and scaled to the desired weather phenomenon influence range.

[0067] Step 104: Use the customized cloud cover product with weather phenomenon labels as input to the trained virtual cloud map generation model, reconstruct the virtual satellite cloud map under the hypothetical weather scenario, and output the result.

[0068] In summary, this invention, based on the concept of deep generative adversarial science, addresses the problem that existing methods cannot generate highly realistic meteorological products based on hypothetical meteorological scenarios. It proposes a method for generating virtual satellite cloud images based on interactive hypothetical scenarios and deep learning. Examples of generating virtual Fengyun 4A geostationary satellite products through human-computer interaction by editing cloud cover products and typhoon weather phenomenon identifiers are provided, illustrating the specific method and its effectiveness. The method for generating virtual satellite cloud images based on interactive hypothetical scenarios and deep learning described in this invention can serve as a foundation and prerequisite for generating various virtual meteorological data.

[0069] Based on the above embodiments, a specific example will be used for illustration below.

[0070] The specific implementation process of the method for generating virtual satellite cloud images based on interactive scenario and deep learning is as follows:

[0071] (1) Collect and organize historical weather phenomenon observation data, mark the corresponding cloud cover products with color blocks or historical weather phenomenon examples according to the type, location and range of historical weather phenomena, and collect and organize satellite cloud images of the corresponding time and region to jointly construct a training dataset.

[0072] The weather phenomenon selected here is typhoon. Satellite cloud cover products corresponding to the occurrence times of typhoons within the past three years were collected. The selected products cover the latitude range of 20°N to 50°N and longitude range of 100°E to 130°E. Historical weather phenomenon examples are used for identification. The grayscale images of typhoon examples are stored in example files, and satellite cloud images of the corresponding time areas are collected.

[0073] This embodiment uses the 4km resolution product of the Fengyun-4A (FY-4A) geostationary meteorological satellite. The FY-4A geostationary meteorological satellite is the first of my country's second-generation geostationary meteorological satellites, carrying multiple payloads including a multi-channel scanning imager, an interferometric atmospheric vertical sounder, a lightning imager, and a space environment monitoring instrument package. This work selects the multi-channel scanning imager product carried by FY-4A as the simulation object. The multi-channel scanning imager is one of the main payloads of FY-4A, and its main task is to perform high-frequency, high-precision, multispectral quantitative remote sensing of the physical state parameters of the Earth's surface and clouds, directly serving weather analysis and forecasting, short-term climate prediction, and environmental and disaster monitoring. The observation bands cover visible light, near-infrared, shortwave infrared, mid-wave infrared, and longwave infrared, enabling observation of both the overall picture of large-scale weather systems and the rapid evolution of medium- and small-scale weather systems. The multi-channel scanning imager is equipped with 14 channels, including 7 visible-near-infrared channels and 7 infrared channels. Of the 14 channels, there is one 500-meter ground resolution channel, two 1km channels, four 2km channels, and seven 4km channels. The full-disk observation time is 15 minutes. In this embodiment, visible light channels 1, 2, and 3 are selected as the target products. To ensure consistent product resolution and maintain general operational consistency, 4km resolution products are selected, and hourly observations are chosen.

[0074] Finally, the constructed training dataset is {C(t)+W(t)→FY4A(t)}, where C(t) is the cloud cover product selected in the example at time t, W(t) is the typhoon identification map, and FY4A(t) is the FY-4A geostationary meteorological satellite channel combination product at time t.

[0075] (2) Based on the training dataset, construct a virtual cloud map generation model based on deep generative adversarial network and train the model.

[0076] Build as Figure 3The simulated satellite cloud image generative adversarial network model VCloudGAN shown is trained on the training set {C(t)+W(t)→FY4A(t)} according to the loss functions set by formulas (1) to (3). Based on the above training set and the simulated satellite cloud image generative adversarial network model VCloudGAN, training is performed in an iterative manner:

[0077] Select a batch of data {C(t)+W(t)}8 from {C(t)+W(t)} as the input to the model.

[0078] The discriminator D is trained, and then fed into the VCloudGAN generator G to generate a simulated satellite cloud image product. The discriminator D is fed with the original numerical model product, real satellite cloud imagery, and simulated satellite cloud imagery {(C(t)+W(t), FY4A(t))}8 generated by the generator G. The parameters of the discriminator D are updated using formula (1) through backpropagation. Then, the parameters of the discriminator D are frozen. Numerical model products ECWMF(t) and corresponding real satellite cloud images {(C(t)+W(t), FY4A(t))}8 are input into the generator G in batches, and the parameters of G are updated using formula (2) through backpropagation. In this embodiment, the parameter optimization method selected is the ADAM method. After each batch of training is completed, the next batch of data is taken and the above process is repeated until the capabilities of the generator and discriminator reach a balance. After 500 iterations (each iteration includes one processing of all batches), the determined model parameters are obtained.

[0079] During training, the optimization method used was minibatch-based stochastic gradient descent (minibatchSGD) with a learning rate of 0.0002 and optimizer momentum parameters β1 = 0.5 and β2 = 0.999.

[0080] (3) Customize and modify the cloud volume product corresponding to the basic cloud map in a human-computer interaction manner, and use color blocks or examples to mark weather phenomena in the same way as the training set, so as to set up the desired weather scenario.

[0081] Select a time period as the base time period, use the corresponding cloud cover product as the base cloud cover product, convert it to image format, and use existing image editing tools or a self-developed image editing tool to perform image operations such as deletion, copying, pasting, panning, rotating, scaling, dotting, and enhancing on the content of the image corresponding to the cloud cover product; place weather phenomenon icons as needed, move them to the required positions, and scale them to the desired range of weather phenomenon influence. For example... Figure 3 The image shown is an example of cloud map transformation based on 11:00 on July 10, 2018, with two customized scenarios set up respectively. Figure 3 Column a represents basic cloud map related products. Column b represents customized scenario 1, where the lower left corner is transformed into thin cirrus clouds. The human-computer interaction scenario is edited as follows: Sub-map b.1 is based on the basic cloud data product, with the lower left corner copied and pasted as thin cirrus clouds. Sub-map b.2 keeps the typhoon weather phenomenon marker in its original position. Column c represents customized scenario 2, where the typhoon is moved to the left and transformed into thin cirrus clouds in its original position. The human-computer interaction scenario is edited as follows: Sub-map c.1 is based on the basic cloud data product, with the original typhoon area's cloud data moved to the left and copied and pasted as thin cirrus clouds in the original typhoon position. Sub-map c.2 moves the typhoon weather phenomenon marker to the location of the typhoon cloud data in sub-map c.1.

[0082] (4) Based on the virtual satellite cloud image generation model, a customized cloud volume product with weather phenomenon labels is used as the model input to reconstruct the virtual satellite cloud image under the hypothetical weather scenario and output it as the result.

[0083] Using the customized cloud cover product and typhoon identifier set in step (3) as input, the trained virtual cloud image generation model is run to reconstruct the corresponding simulated satellite cloud image data product as output. The reconstruction effect is as follows: Figure 3 As shown, subgraph b.3 is the virtual cloud map generated under customized scenario 1, and subgraph c.3 is the virtual cloud map generated under customized scenario 2. Intuitively, it can be seen that the method proposed in this invention can effectively simulate virtual cloud map products under the desired interactive scenario.

[0084] At the same time, the inventors also conducted experiments on generating virtual satellite cloud images using infrared channel data and water vapor channel data. The method proposed in this invention can effectively simulate infrared channel products and water vapor channel products of satellite cloud images.

[0085] Based on the above embodiments, the present invention also discloses a system for generating virtual cloud images based on interactive scenarios and deep learning, comprising: a training dataset construction module for constructing a training dataset based on historical weather phenomenon observation data; a model construction module for constructing and training a virtual cloud image generation model based on a deep generative adversarial network based on the training dataset to obtain a trained virtual cloud image generation model; a setting module for setting a hypothetical weather scenario based on human-computer interaction; and a generation module for taking a customized cloud volume product with weather phenomenon identifiers as input to the trained virtual cloud image generation model, reconstructing a virtual satellite cloud image under the hypothetical weather scenario, and outputting the result.

[0086] As the system implementation is in contrast to the method implementation, it is described in a simpler way. For relevant details, please refer to the description in the method implementation section.

[0087] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

[0088] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A method for generating virtual cloud maps based on interactive scenario planning and deep learning, characterized in that, include: A training dataset is constructed based on historical weather phenomenon observation data. This includes: acquiring historical weather phenomenon observation data; organizing and analyzing the historical weather phenomenon observation data to determine the type, location, and extent of historical weather phenomena, and marking them on corresponding cloud cover products using color blocks or weather phenomenon examples; simultaneously, collecting and organizing real satellite cloud images for corresponding times and regions; based on the marking results of the type, location, and extent of historical weather phenomena on corresponding cloud cover products and the collected and organized real satellite cloud images for corresponding times and regions, a training dataset is constructed; when using color blocks to mark weather phenomena on corresponding cloud cover products, the color of the color block is used to represent the intensity characteristics of the weather phenomenon, and the position of the color block is used to represent the impact range of the weather phenomenon; when using weather phenomenon examples to mark weather phenomena on corresponding cloud cover products, the outline of the weather phenomenon example is used to represent the intensity characteristics and impact range of the weather phenomenon. Based on the training dataset, a virtual cloud map generation model based on deep generative adversarial networks (GANs) is constructed and trained to obtain the trained virtual cloud map generation model. The virtual cloud map generation model based on GANs includes a generator network G and a discriminator network D. The generator network G consists of an encoder-decoder network as its backbone, using the U-Net network structure, which is an encoder-decoder network with added skip connections. The discriminator network D consists of a binary classification network composed of multiple convolutional neural network layers as its backbone. By calculating the probability that the input image is a real satellite cloud map, it distinguishes whether the input image is a real satellite cloud map or a generated virtual cloud map. When training the virtual cloud map generation model based on a deep generative adversarial network, the following steps are taken: Based on the training dataset, determine the identification results of the types, locations, and extents of historical weather phenomena on the corresponding cloud cover products and the corresponding real satellite cloud images for the time and region; use the identification results of the types, locations, and extents of historical weather phenomena on the corresponding cloud cover products and the corresponding real satellite cloud images for the time and region as input to the generator network G; use the identification results of the types, locations, and extents of historical weather phenomena on the corresponding cloud cover products, the corresponding real satellite cloud images for the time and region, and the virtual satellite cloud images generated by the generator network G as input to the discriminator network D; perform iterative training to obtain the trained virtual cloud map generation model. The iterative training process is as follows: Input the historical weather phenomenon type, location, and extent identification results on the corresponding cloud cover product, the corresponding time and region real satellite cloud image, and the virtual satellite cloud image generated by the generator network G into the discrimination network D in batches. Update the parameters of the discrimination network D using the discrimination network loss function L_D through backpropagation; freeze the parameters of the discrimination network D; input the historical weather phenomenon type, location, and extent identification results on the corresponding cloud cover product, and the corresponding time and region real satellite cloud image into the generator network G in batches. Update the parameters of the generator network G using the generator network loss function L_G through backpropagation; repeat the above process until the capabilities of the generator network G and the discrimination network D reach a balance, obtain the parameters of the virtual cloud image generation model during runtime, and then obtain the trained virtual cloud image generation model; Use binary cross-entropy as the loss metric: For the discriminant network D, the loss function L_D is as follows: L_D=L bce (D(x,y),1)+L bce (D(x,G(x,z)),0)···(1) Where x represents the combined data of cloud cover products and weather phenomenon identifiers, y represents the corresponding real satellite cloud image, G(x,z) represents the virtual satellite cloud image generated by the generator network G from x and random noise z, and D(x,y) represents the probability of the discriminator network D identifying real and fake satellite cloud images; For the generator network G, the loss function L_G is as follows: L_G=λ1L bce (D(x,G(x,z)),1)+λ2|y-G(x,z)|···(2) Where λ1 and λ2 represent two proportional parameters respectively; in: ···(3) Where N is the number of samples in a batch of model input; a∈{0,1} represents the label of the input data, 0 represents simulated satellite cloud image; 1 represents real satellite cloud image; It is the discrimination value output by the discrimination network D. If the value is close to 0, the discrimination network has a high probability that the input is a virtual satellite cloud image, and if it is close to 1, the network has a high probability that the input is a real satellite cloud image. Based on human-computer interaction, a hypothetical weather scenario is set up; including: customizing and modifying the cloud cover product corresponding to the basic satellite cloud image through human-computer interaction, and marking the corresponding cloud cover product with color blocks or weather phenomenon examples to complete the setting up of the hypothetical weather scenario; Customized cloud cover products with weather phenomenon labels are used as input to the trained virtual cloud image generation model to reconstruct virtual satellite cloud images under a hypothetical weather scenario and output the result.

2. The method for generating virtual cloud maps based on interactive scenario and deep learning according to claim 1, characterized in that, Cloud quantity products refer to the distribution of clouds in a specific area, reflecting the amount of clouds; Types of weather phenomena include: typhoons, thunderstorms, heavy precipitation, hail, and dense fog.

3. The method for generating virtual cloud maps based on interactive scenario and deep learning according to claim 1, characterized in that, Satellite cloud images are data from meteorological satellites in the same region as cloud cover products, including specific visible light channel data, water vapor channel data, infrared channel data, or a combination of multiple channels.

4. A system for generating virtual cloud maps based on interactive scenario and deep learning to implement the method as described in claim 1, characterized in that, include: The training dataset building module is used to build a training dataset based on historical weather phenomenon observation data. The model building module is used to build and train a virtual cloud map generation model based on a deep generative adversarial network according to the training dataset, so as to obtain the trained virtual cloud map generation model. The design module is used to design hypothetical weather scenarios based on human-computer interaction. The generation module is used to take a customized cloud cover product with weather phenomenon labels as input to the trained virtual cloud image generation model, reconstruct a virtual satellite cloud image under a hypothetical weather scenario, and output the result.

Images