Cloud picture enhancement method and device based on style migration, equipment, medium and product

Abstract

The invention discloses a cloud picture enhancement method and device based on style migration, equipment, a medium and a product, and relates to the field of cloud picture enhancement, and the method comprises the steps: carrying out the preliminary enhancement of multi-source cloud product data in satellite cloud picture data, and determining a multi-source cloud feature graph; based on a style migration algorithm, taking an infrared cloud image as a content cloud image, extracting content features of the content cloud image, and taking the multi-source cloud feature image as a style cloud image, and extracting style features of the style cloud image; fusing the content features and the style features to generate a fused cloud picture; according to the photovoltaic power station power prediction method and device, the cloud picture data quality is improved, it is ensured that the fused cloud picture maintains the basic morphological structure of the cloud picture, and rich color distribution information is added, so that the accuracy of photovoltaic power station power prediction is higher.

Description

Technical Field

[0001] The present application relates to the field of cloud image enhancement, and in particular to a cloud image enhancement method, device, equipment, medium and product based on style transfer. Background Art

[0002] Solar energy, as a clean and endless energy source, is highly respected and strongly supported by governments around the world for its huge development potential, sustainability and emission reduction effects. However, with the continuous increase in photovoltaic installed capacity, it also brings more severe challenges to power system operators. In order to help grid operators better plan and manage power resources, and adjust power supply and demand in a timely manner, it is necessary to comprehensively consider the multiple meteorological factors that affect photovoltaic power output and grid operation requirements, and further explore the development of high-precision prediction models.

[0003] The direct prediction method directly predicts the power of photovoltaic power stations. It relies on sufficient historical cloud images, meteorological and power data. By correlating historical cloud image data with historical photovoltaic power output data, it predicts the photovoltaic power generation power in a short period of time in the future, without the need to predict the future state of the cloud image separately. However, historical cloud image data cannot reflect multi-source cloud product data such as cloud type and cloud top temperature in satellite cloud images, resulting in low quality of historical cloud image data. Therefore, the prediction results of photovoltaic power generation based on low-quality historical cloud image data are of low accuracy. Summary of the invention

[0004] The purpose of this application is to provide a cloud image enhancement method, device, equipment, medium and product based on style transfer to solve the problem of low accuracy of photovoltaic power generation prediction results.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a cloud image enhancement method based on style transfer, comprising:

[0007] Preliminarily enhance the multi-source cloud product data in the satellite cloud image data to determine the multi-source cloud feature map; the multi-source cloud product data includes cloud type, cloud top height, cloud top temperature and cloud optical thickness;

[0008] Based on the style transfer algorithm, the infrared cloud image is used as the content cloud image to extract the content features of the content cloud image, and the multi-source cloud feature image is used as the style cloud image to extract the style features of the style cloud image; the content features include cloud phase, cloud structure and texture framework; the style features include color distribution information;

[0009] Fusion of the content features and the style features to generate a fusion cloud map;

[0010] The power of the photovoltaic power station is predicted according to the fused cloud map.

[0011] In a second aspect, the present application provides a cloud image enhancement device based on style transfer, comprising:

[0012] A multi-source cloud feature map determination module is used to preliminarily enhance the multi-source cloud product data in the satellite cloud image data to determine the multi-source cloud feature map; the multi-source cloud product data includes cloud type, cloud top height, cloud top temperature and cloud optical thickness;

[0013] A feature extraction module is used to extract content features of the content cloud map using the infrared cloud map as the content cloud map based on the style migration algorithm, and to extract style features of the style cloud map using the multi-source cloud feature map as the style cloud map; the content features include cloud phase, cloud structure and texture framework; the style features include color distribution information;

[0014] A fusion cloud map generation module, used to fuse the content features and the style features to generate a fusion cloud map;

[0015] The photovoltaic power station power prediction module is used to predict the photovoltaic power station power according to the fused cloud map.

[0016] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-mentioned cloud image enhancement methods based on style transfer.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned cloud image enhancement methods based on style transfer.

[0018] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned cloud image enhancement methods based on style transfer.

[0019] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0020] This application preliminarily enhances the multi-source cloud product data in the satellite cloud image data to obtain a multi-source cloud feature map, thereby fully considering the multi-source cloud product data such as cloud type and cloud top temperature in the satellite cloud image, and improving the quality of the cloud image data; then, based on the style migration algorithm, the infrared cloud image is used as the content cloud image, and the multi-source cloud feature map is used as the style cloud image, and the content features and style features are extracted and fused to generate a fused cloud image to predict the power of the photovoltaic power station. In the fusion process, the content features characterizing the structure and the style features characterizing the color distribution information are effectively integrated to ensure that the fused cloud image not only maintains the basic morphological structure of the cloud image, but also adds rich color distribution information, thereby making the predicted photovoltaic power station power more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A schematic diagram of a flow chart of a cloud image enhancement method based on style transfer provided in one embodiment of the present application;

[0023] Figure 2 A flowchart of an AdaIN implementation process provided in an embodiment of the present application;

[0024] Figure 3 A comparison chart of the enhanced fusion results of the infrared band cloud map and four cloud feature maps provided in one embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0027] The embodiment of the present application provides a cloud image enhancement method based on style transfer, which is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, Figure 1As shown, the method includes the following steps.

[0028] S1: Preliminary enhancement of multi-source cloud product data in satellite cloud image data to determine a multi-source cloud feature map; the multi-source cloud product data includes cloud type, cloud top height, cloud top temperature and cloud optical thickness.

[0029] S2: Based on the style transfer algorithm, the infrared cloud image is used as the content cloud image to extract the content features of the content cloud image, and the multi-source cloud feature image is used as the style cloud image to extract the style features of the style cloud image; the content features include cloud phase, cloud structure and texture framework; the style features include color distribution information.

[0030] S3: Fusing the content features and the style features to generate a fusion cloud map.

[0031] S4: Predicting the power of the photovoltaic power station according to the fused cloud map.

[0032] In an exemplary embodiment, S2 may be replaced by the following steps.

[0033] Encode the content cloud map and the style cloud map by using the convolution layer of the VGG network, and extract feature images of the content cloud map and the style cloud map;

[0034] The content features and the style features are extracted based on the feature images of the content cloud map and the style cloud map.

[0035] In an exemplary embodiment, S3 may be replaced by the following steps.

[0036] The content features and the style features are input into the Adaptive Instance Normalization (AdaIN) layer of the Visual Geometry Group (VGG) network for style transfer, and the mean and standard deviation of the content features are aligned with the mean and standard deviation of the style features to generate a fusion cloud map.

[0037] In an exemplary embodiment, Figure 2 As shown, the content features and the style features are input into the AdaIN layer of the VGG network for style transfer, specifically including: using the formula The content features and the style features are style-transferred; wherein AdaIN(x, y) is a style transfer function, x is a content cloud map, y is a style cloud map; μ(*) and σ(*) are the mean and standard deviation of *, respectively, and * is x or y.

[0038] AdaIN(x,y) is an improvement on the normalization method called Instance-Normalization (IN). It uses the encoder and decoder structure and can achieve arbitrary style transfer of real-time images without learning any affine parameters.

[0039] In an exemplary embodiment, aligning the mean and standard deviation of the content feature with the mean and standard deviation of the style feature to generate a fusion cloud map may be replaced by the following steps.

[0040] S31: Determine a content difference by subtracting the mean of the content features from the calculated mean of the content features.

[0041] S32: Determine a de-stylized content cloud map by dividing the content difference by the standard deviation of the content feature.

[0042] S33: multiplying the calculated mean of the de-stylized content cloud map by the standard deviation of the style feature, and adding the calculated mean of the style feature to generate a combined cloud map after style transfer.

[0043] S34: decoding the style-transferred combined cloud image through a decoder to map out the fused cloud image.

[0044] like Figure 2 As shown in the figure, x and y represent the content cloud map and style cloud map respectively, t is the generated target feature image, g(*) is the decoder, and T is the stylized fusion cloud map.

[0045] The mean and standard deviation of the input content feature cloud map x are calculated, and the content feature x is subtracted from its own mean and then divided by the standard deviation to achieve de-stylization. The style cloud map y is then multiplied by the mean and standard deviation to achieve stylization, and the fused cloud map T is mapped out through the decoder.

[0046] In an exemplary embodiment, the fusion cloud map is: out =D(AdaIN(E(I c ),D(I s ))); among which, I out is a fusion cloud map; D(AdaIN(E(I c ),D(I s ))) is the combined cloud image after decoding style transfer; AdaIN(E(I c ),D(I s )) is the combined cloud map after style transfer; E(I c ) is the content feature I of the content cloud graph c Encoding; D(I s ) is the style feature I s to decode.

[0047] In practical applications, E and D represent the encoder and decoder. Let: x = E(I c ),y=E(I s ), x∈R c×w×h ,y∈R c×w×h ,AdaIN(x,y)∈R c×w×h , R c×w×h is a three-dimensional space, where c is the length, w is the width, and h is the height.

[0048] In practice, the decoder is trained to invert the style-transferred composite cloud map output by AdaIN to the image space, and the same VGG encoder is used to calculate the content and style losses.

[0049] In another embodiment, the present application applies the AdaIN algorithm to the field of cloud image enhancement. The input of the algorithm is an infrared band cloud image (i.e., infrared cloud image) and a cloud type feature map (i.e., a multi-source cloud feature map). The algorithm outputs a combined cloud image after style transfer. The convolution layer of a fixed VGG network is used to encode the content cloud image and the style cloud image, extract the feature images of the content cloud image and the style cloud image, and then input the feature image in the feature space into the AdaIN layer for style transfer. The mean and standard deviation of the content feature are aligned with the mean and standard deviation of the style feature to obtain a fused cloud image. The advantages thereof are as follows:

[0050] It effectively integrates the color feature distribution of cloud texture structure and cloud type. Compared with other style transfer algorithms, AdaIN achieves real-time performance in the image generation process, without the need to train a large number of network models in advance. It can better transfer the features of the style image to the content image, achieve a vivid and clear style transfer effect, and control the amount of transfer of content image and style image information by modifying some parameters, which is simple and effective.

[0051] The data for enhanced fusion in this application are cloud types, cloud top heights, cloud optical thickness, and cloud top temperature data in the Himawari-8 satellite infrared band cloud image and cloud product data.

[0052] The purpose of satellite image fusion is to make the fused image contain as much information as possible from the two source images. Therefore, the evaluation indicators based on the fused image and the source image are selected, namely, the mutual information (MI) and the structural similarity index (SSIM):

[0053] The mutual information calculation method is as follows:

[0054]

[0055] Among them, P F (f) and P X (x) represent the edge distribution histograms of the fused image F and the source image X, respectively, F,X (f,x) represents the joint distribution histogram of the fused image F and the source image X. Then the total mutual information is:

[0056] MI=MI F,A +MI F,B

[0057] Among them, MI F,A and MI F,B They respectively represent the information transferred from source image A and source image B to fused image F. The larger the MI value of the fused image, the more information the fused image inherits from the source image, and the better the fusion effect.

[0058] The structural similarity index is as follows, and the total structural similarity is calculated as follows:

[0059]

[0060] Based on the above evaluation indicators, this application selects weighted fusion and Laplace pyramid fusion methods for comparison. The comparison results are shown in Table 1. The cloud map fusion example is as follows: Figure 3 shown.

[0061] Table 1 Enhanced fusion evaluation results

[0062]

[0063] It can be seen from Table 1 that the MI index of the weighted fusion method is low, indicating that it does not effectively transfer the information in the source image to the fused image, while the MI index of the Laplace method is high, but its SSIM index is low, indicating that this method cannot transfer the texture structure and features of the cloud to the fused cloud image. The MI and SSIM indexes of this application are significantly higher than those of the two methods, and it not only contains the cloud image structure information but also integrates information such as cloud type.

[0064] from Figure 3 It can be clearly seen that although the weighted fusion method inherits the cloud texture in the infrared cloud map, the features contained in the feature map such as cloud type are not inherited, so its MI index is low. The Laplace pyramid fusion method can be clearly seen to inherit the color distribution of feature maps such as cloud type, but its cloud texture structure is fuzzy, so its SSIM index is low. The present application effectively integrates the cloud texture structure and the color feature distribution of cloud type at the same time.

[0065] Based on the same inventive concept, the embodiment of the present application also provides a cloud image enhancement device based on style transfer for implementing the cloud image enhancement method based on style transfer involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the cloud image enhancement device based on style transfer provided below can refer to the limitations of the cloud image enhancement method based on style transfer above, and will not be repeated here.

[0066] In an exemplary embodiment, a cloud image enhancement device based on style transfer is provided, comprising:

[0067] The multi-source cloud feature map determination module is used to preliminarily enhance the multi-source cloud product data in the satellite cloud image data and determine the multi-source cloud feature map; the multi-source cloud product data includes cloud type, cloud top height, cloud top temperature and cloud optical thickness.

[0068] The feature extraction module is used to extract the content features of the content cloud map based on the style migration algorithm, using the infrared cloud map as the content cloud map, and extract the style features of the style cloud map using the multi-source cloud feature map as the style cloud map; the content features include cloud phase, cloud structure and texture framework; the style features include color distribution information.

[0069] The fusion cloud map generation module is used to fuse the content features and the style features to generate a fusion cloud map.

[0070] The photovoltaic power station power prediction module is used to predict the photovoltaic power station power according to the fused cloud map.

[0071] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store cloud image enhancement data based on style transfer. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a cloud image enhancement method based on style transfer is implemented.

[0072] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above method is implemented when the processor executes the computer program.

[0073] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above method when executed by a processor.

[0074] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the above method when executed by a processor.

[0075] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric random access memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0076] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0077] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0078] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A cloud image enhancement method based on style transfer, characterized in that: The cloud image enhancement method based on style transfer includes: Preliminarily enhance the multi-source cloud product data in the satellite cloud image data to determine the multi-source cloud feature map; the multi-source cloud product data includes cloud type, cloud top height, cloud top temperature and cloud optical thickness; Based on the style transfer algorithm, the infrared cloud image is used as the content cloud image to extract the content features of the content cloud image, and the multi-source cloud feature image is used as the style cloud image to extract the style features of the style cloud image; the content features include cloud phase, cloud structure and texture framework; the style features include color distribution information; Fusion of the content features and the style features to generate a fusion cloud map; The power of the photovoltaic power station is predicted according to the fused cloud map.

2. The cloud image enhancement method based on style transfer according to claim 1, characterized in that: Based on the style migration algorithm, the infrared cloud image is used as the content cloud image to extract the content features of the content cloud image, and the multi-source cloud feature image is used as the style cloud image to extract the style features of the style cloud image, which specifically includes: Encode the content cloud map and the style cloud map by using the convolution layer of the VGG network, and extract feature images of the content cloud map and the style cloud map; The content features and the style features are extracted based on the feature images of the content cloud map and the style cloud map.

3. The cloud image enhancement method based on style transfer according to claim 1, characterized in that: The content features and the style features are integrated to generate a fusion cloud map, which specifically includes: The content features and the style features are input into the AdaIN layer of the VGG network for style transfer, and the mean and standard deviation of the content features are aligned with the mean and standard deviation of the style features to generate a fusion cloud map.

4. The cloud image enhancement method based on style transfer according to claim 3, characterized in that: Inputting the content features and the style features into the AdaIN layer of the VGG network for style transfer, specifically including: Using the formula The content features and the style features are style-transferred; wherein AdaIN(x, y) is a style transfer function, x is a content cloud map, y is a style cloud map; μ(*) and σ(*) are the mean and standard deviation of *, respectively, and * is x or y.

5. The cloud image enhancement method based on style transfer according to claim 3, characterized in that: Aligning the mean and standard deviation of the content feature with the mean and standard deviation of the style feature to generate a fusion cloud map, specifically including: Determining a content difference by subtracting a mean of the content feature from a calculated mean of the content feature; Determine a de-stylized content cloud map by dividing the content difference by the standard deviation of the content feature; Multiplying the calculated mean of the de-stylized content cloud map by the standard deviation of the style feature and adding the mean of the style feature to generate a combined cloud map after style transfer; The combined cloud image after style transfer is decoded by a decoder to map out the fused cloud image.

6. The cloud image enhancement method based on style transfer according to claim 5, characterized in that: The fusion cloud map is: I out =D(AdaIN(E(I c ),D(I s ))); Among them, I out is a fusion cloud map; D(AdaIN(E(I c ),D(I s ))) is the combined cloud image after decoding style transfer; AdaIN(E(I c ),D(I s )) is the combined cloud map after style transfer; E(I c ) is the content feature I of the content cloud graph c Encoding; D(I s ) is the style feature I s to decode.

7. A cloud image enhancement device based on style transfer, characterized in that: The cloud image enhancement device comprises: A multi-source cloud feature map determination module is used to preliminarily enhance the multi-source cloud product data in the satellite cloud image data to determine the multi-source cloud feature map; the multi-source cloud product data includes cloud type, cloud top height, cloud top temperature and cloud optical thickness; A feature extraction module is used to extract content features of the content cloud map using the infrared cloud map as the content cloud map based on the style migration algorithm, and to extract style features of the style cloud map using the multi-source cloud feature map as the style cloud map; the content features include cloud phase, cloud structure and texture framework; the style features include color distribution information; A fusion cloud map generation module, used to fuse the content features and the style features to generate a fusion cloud map; The photovoltaic power station power prediction module is used to predict the photovoltaic power station power according to the fused cloud map.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the cloud image enhancement method based on style transfer according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the cloud image enhancement method based on style transfer described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the cloud image enhancement method based on style transfer described in any one of claims 1 to 6 is implemented.

Images