Foundation cloud picture fine-grained segmentation method based on convolution and attention
Through deep learning methods combining convolution and attention mechanisms in fine-grained segmentation of foundation cloud maps, the problems of insufficient feature extraction capabilities and high computing costs in the existing technology are solved, and high-precision cloud layer recognition and photovoltaic power prediction are achieved. In the fields of photovoltaic power station management and energy dispatch.
Patent Information
- Application Number
- CN202411248260.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-06
- Publication Date
- 2024-11-26
AI Technical Summary
The existing deep learning methods have problems such as insufficient local to global feature extraction capabilities, high computing costs and model complexity in fine-grained segmentation of foundation cloud maps, making it difficult to effectively capture multi-scale semantic information of high-resolution cloud maps.
Using a fine-grained segmentation method of foundation cloud maps based on convolution and attention, combined with the Keras deep learning framework, using hybrid convolution and Transformer networks, Swin Transformer Block, Fg-FFM module and GloRe global map module are introduced, and the multi-scale attention module is used. The graph convolution realizes effective interaction between global and local features, reducing the amount of parameters and calculations.
It improves the accuracy of cloud recognition and the accuracy of photovoltaic power prediction, reduces prediction errors, is suitable for photovoltaic power station management, energy scheduling and power market optimization, and has good scalability.
Smart Images

Figure CN119027669A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image analysis, and in particular to a ground-based cloud image fine-grained segmentation method based on convolution and attention. Background Art
[0002] Clouds are visible aggregates floating in the air, composed of small water droplets or small ice crystals formed by the condensation of water vapor in the atmosphere. Cloud amount, cloud height, and cloud distribution affect the evolution of climate to a certain extent, especially in terms of shielding solar irradiance. Cloud observation has always been a challenging task. Traditional cloud observation mainly relies on manual observation and recording with the naked eye. However, with the update and iteration of modern hardware equipment and the widespread application of artificial intelligence technology in remote sensing, energy, meteorology and other fields, cloud observation and recording based on automated equipment has become mainstream.
[0003] Currently, the methods of automatically collecting and observing clouds are mainly divided into two categories: satellite cloud observation and ground-based cloud observation. However, in ultra-short-term photovoltaic power forecasting, satellite remote sensing images have low temporal resolution and too wide an observation range, making them unsuitable for forecasting a single photovoltaic power station. In contrast, ground-based cloud images have higher temporal resolution and only focus on areas near photovoltaic power stations, which gives them a significant advantage in ultra-short-term forecasting.
[0004] Ground-based cloud image observation technology combined with artificial intelligence methods such as deep learning plays a key role in photovoltaic power forecasting. Through high-precision camera equipment to continuously observe the local sky, the cloud change image data obtained can capture extremely subtle weather changes. Based on this data, the deep learning model can train accurate cloud change predictions, thereby greatly improving the accuracy of photovoltaic power station power output predictions. The power output of photovoltaic power stations directly depends on the intensity of solar radiation, and cloud changes are one of the main factors affecting solar radiation. Traditional photovoltaic power prediction methods are usually based on meteorological station data and cannot accurately capture the dynamic changes of local clouds. Therefore, the prediction results often have large errors. Through ground-based cloud image observation technology, photovoltaic power station managers can obtain detailed change information of local clouds in real time and input this information into the deep learning model to obtain more accurate power predictions. This accurate prediction can not only optimize the operation strategy of the power station, but also reduce the economic losses caused by prediction errors.
[0005] In the wider renewable energy sector, the accuracy of photovoltaic power forecasting is crucial to energy dispatching and distribution. As the proportion of renewable energy in the global energy structure continues to increase, the power grid has higher and higher requirements for the stability of renewable energy output. Accurate photovoltaic power forecasting helps the grid dispatching department to better arrange power generation plans and ensure the stability of power supply, thereby reducing the imbalance between supply and demand caused by power fluctuations. This not only improves the utilization efficiency of renewable energy, but also enhances the stability and reliability of the power system.
[0006] In the future, with the continuous advancement of ground-based cloud image observation technology and deep learning models, photovoltaic power prediction will become more accurate, providing stronger support for the efficient use of renewable energy. The widespread application of this technology will help the world achieve sustainable energy development goals, promote energy transformation, and make important contributions to reducing carbon emissions and addressing climate change. In recent years, under the background of cross-integration of different disciplines and artificial intelligence, the ground-based cloud image segmentation method based on deep learning has become a research hotspot due to its superior segmentation performance, and has achieved rapid and substantial development. However, there are the following problems in using deep learning methods to segment ground-based cloud images:
[0007] 1. In the task of fine-grained segmentation of ground-based cloud images, the mainstream deep learning segmentation method is mainly based on CNN. However, due to the weak ability of CNN to extract global features of cloud images, it can only achieve image extraction from local to global by continuously stacking convolutional layers, which lacks robustness.
[0008] 2. The ground-based cloud image fine-grained segmentation method based on visual Transformer always performs sampling at the same scale, requiring the calculation of self-attention between each patch and all other patches, resulting in extremely high computational cost when processing high-resolution images.
[0009] 3. Most traditional CNNs can only extract global semantic information of images from a single scale. Although multi-scale fusion can extract more effective semantic information from multiple scales, it will increase the complexity of the model and the amount of calculation. Summary of the invention
[0010] In order to solve the problem of low efficiency in using cloud image semantic information for photovoltaic power prediction, the present invention discloses a ground-based cloud image fine-grained segmentation method based on convolution and attention.
[0011] The present invention provides the following technical methods:
[0012] A convolution and attention-based ground-based cloud image fine-grained segmentation method, the flow chart is as follows Figure 1 As shown, the method includes:
[0013] Obtain cloud map fine-grained data set data to be processed, pre-process the cloud map fine-grained data set data to be processed, and obtain cloud map fine-grained data set data.
[0014] The cloud map fine-grained dataset data is input into the preset ground-based cloud map fine-grained segmentation network to obtain the cloud map fine-grained segmentation result.
[0015] Among them, the preset ground-based cloud map fine-grained segmentation network is a network built using the Keras deep learning framework. The construction of the preset ground-based cloud map fine-grained segmentation network adopts a ground-based cloud map fine-grained segmentation network of mixed convolution and Transformer, and uses Swin Transformer Block as a basic module to build a segmentation network. The network is a typical encoding-decoding structure. The Fg-FFM module is added to the encoding structure to fuse the multi-scale information extracted by different Transformer layers. The GloRe global graph module is used to replace the jump connection operation. The GloRe unit can realize effective interaction between global and local features. Its lightweight design improves the global modeling efficiency of the model while maintaining low parameters and computational complexity; Among them, the Fg-FFM module is added to the encoding structure to fuse the multi-scale information extracted by different Transformer layers. Its specific structure is as follows Figure 2 As shown, specifically including:
[0016] The Fg-FFM module is used to cascade between the Swin Transformer Block layers in the encoder; the feature fusion module Fg-FFM can be represented by the following mathematical model:
[0017]
[0018] Among them, X1 is a low-level feature map, and Y1 is a high-level semantic feature map. X1 and Y1 are taken as input features, Z1 is the output of attention feature fusion, and η represents the feature weighting operation implemented by the multi-scale attention module;
[0019] Among them, the structure of the GloRe unit contains five main convolutional layers. First, there are two convolutional layers for reducing and expanding the input feature X3 and the output feature Y3 to increase the dimension and expressiveness of the features. Next, there is a top convolutional layer for generating a double projection B between the coordinate and potential interaction space, thereby realizing the mapping from the coordinate space to the interaction space. Then, there are two middle convolutional layers for global reasoning in the interaction space, in which regional features are encoded as graph nodes, and the parameters of the graph convolution are used to capture global relationships. Finally, there is a convolutional layer for projecting the features of the interaction space back to the coordination space to obtain the final output feature Y3.
[0020] Among them, the preset ground-based cloud image fine-grained segmentation network training data is 300 images selected from the comprehensive all-sky cloud image dataset of Kiel University in Germany. These images cover various types of clouds. Under the guidance of meteorological experts, these clouds are divided into five categories and labeled using professional software.
[0021] The cloud map fine-grained data set data to be processed is data from a cloud map analysis of a designated photovoltaic power station that has been put into use.
[0022] Among them, the designed Fg-FFM module adopts the MS-AM multi-scale attention module;
[0023] The MS-AM module adopts a dual-branch structure, in which the spatial branch takes the feature map X2 as input and obtains the correlation distribution in the spatial dimension through a compression excitation operation. The compression excitation operation consists of a convolution layer, a batch normalization layer, and a nonlinear activation function. The convolution layer uses a depth-separable convolution to reduce the number of parameters and calculations. The channel branch first performs a global adaptive pooling operation on the input feature map X2, and then performs a compression excitation operation to obtain the correlation distribution in the channel dimension. The specific size change can be expressed as the initial size C×H×W feature map is obtained through the depth-separable convolution layer and the BN layer. The feature map is then processed by the ReLu activation function and the depth-separable convolution layer to restore the feature map size to C×H×W and C×1×1. After the BN layer, the two feature maps are weighted concatenated to obtain a feature map of size C×H×W. Finally, the Sigmoid activation function is used to multiply the original feature map to obtain the final output feature map.
[0024] Among them, Swin Transformer Block is taken as the basic module, and an encoding and decoding structure is adopted. The input image is patch segmented and linearly embedded, and then input into the Swin Transformer Block for feature extraction. At the same time, patch expansion is used for upsampling and then continued to be input into the Swin Transformer Block for semantic information recovery and reconstruction.
[0025] During the training process, the number of network iteration training was initially set to 1000. Based on the downward trend of the test loss curve, the experiment finally set the epoch to 100 to prevent the model from having the theoretically optimal generalization performance under overfitting conditions.
[0026] Compared with the prior art, the above technical method has the following advantages:
[0027] The present invention utilizes the local feature reading capability of the convolutional neural network and the global semantic information association capability of the Transformer model, achieving the effect of focusing on both local contextual information and global semantic information extraction, thereby ensuring the accuracy of the segmentation results. The patent of this invention focuses on fine-grained ground-based cloud image segmentation, aiming to analyze the proportion of different cloud types in the current sky to support data processing systems or methods for administrative, commercial, financial, management, supervisory or predictive purposes. Since the impact of clouds on photovoltaic power forecasting is crucial, accurately understanding the composition and distribution of clouds becomes the key to improving forecast accuracy. To this end, the present invention introduces an advanced deep learning model to perform fine segmentation of ground-based cloud images, thereby achieving accurate identification and proportional analysis of different types of clouds.
[0028] Through this fine-grained cloud image segmentation method, the present invention not only improves the accuracy of cloud recognition, but also enables photovoltaic power forecasting to more accurately consider the impact of dynamic changes in clouds on solar energy resources. This method shows broad application prospects in the fields of photovoltaic power station management, energy scheduling, and power market optimization. In addition, the technology of the present invention has good scalability and is suitable for other scenarios that require refined meteorological analysis, such as wind power generation forecasting and agricultural meteorological services.
[0029] By applying this advanced segmentation technology to actual energy management systems, the present invention not only improves the accuracy of photovoltaic power prediction, but also provides technical support for the efficient use of renewable energy, which helps to promote the realization of energy conservation and environmental protection goals. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the embodiments of the present invention or the technical methods in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0031] Figure 1 It is a flow chart of a method for fine-grained segmentation of ground-based cloud images based on convolution and attention according to an embodiment of the present invention;
[0032] Figure 2 It is a specific structural diagram of a ground-based cloud image fine-grained segmentation method based on convolution and attention according to an embodiment of the present invention;
[0033] Figure 3 It is a structural diagram of two consecutive Swin Transformer blocks;
[0034] Figure 4 It is a schematic diagram of the structure of the Fg-FFM feature fusion module;
[0035] Figure 5 Schematic diagram of the MS-AM multi-scale attention module structure;
[0036] Figure 6 This is a schematic diagram of the GloRe global graph reasoning module structure;
[0037] Figure 7 The segmentation effect diagram of the ground-based cloud image fine-grained segmentation method based on convolution and attention. DETAILED DESCRIPTION
[0038] As mentioned in the background technology section, how to extract and utilize the high-order semantic information of ground-based cloud images to improve the accuracy of photovoltaic power prediction is an urgent problem to be solved by researchers in this field.
[0039] The core idea of the present invention is to utilize the local feature reading ability of the convolutional neural network and the global semantic information association ability of the Transformer model, so as to achieve the effect of paying attention to both local contextual information and global semantic information extraction. At the same time, the Fg-FFM and GloRe modules can further enhance the recovery and extraction of semantic information while keeping the model lightweight, thus ensuring the accuracy of the segmentation results.
[0040] A convolution and attention-based ground-based cloud image fine-grained segmentation method, the flow chart is as follows Figure 1 As shown, the method includes:
[0041] Obtain cloud map fine-grained data set data to be processed, pre-process the cloud map fine-grained data set data to be processed, and obtain cloud map fine-grained data set data.
[0042] The cloud map fine-grained dataset data is input into the preset ground-based cloud map fine-grained segmentation network to obtain the cloud map fine-grained segmentation result.
[0043] Among them, the preset ground-based cloud map fine-grained segmentation network is a network built using the Keras deep learning framework. The construction of the preset ground-based cloud map fine-grained segmentation network adopts a ground-based cloud map fine-grained segmentation network of mixed convolution and Transformer, and uses Swin Transformer Block as a basic module to build a segmentation network. The network is a typical encoding-decoding structure. The Fg-FFM module is added to the encoding structure to fuse the multi-scale information extracted by different Transformer layers. The GloRe global graph module is used to replace the jump connection operation. The GloRe unit can realize effective interaction between global and local features. Its lightweight design improves the global modeling efficiency of the model while maintaining low parameters and computational complexity; Among them, the Fg-FFM module is added to the encoding structure to fuse the multi-scale information extracted by different Transformer layers. Its specific structure is as follows Figure 2 As shown, specifically including:
[0044] The Fg-FFM module is used to cascade between the Swin Transformer Block layers in the encoder; the feature fusion module Fg-FFM can be represented by the following mathematical model:
[0045]
[0046] Among them, X1 is a low-level feature map, and Y1 is a high-level semantic feature map. X1 and Y1 are taken as input features, Z1 is the output of attention feature fusion, and η represents the feature weighting operation implemented by the multi-scale attention module.
[0047] Among them, the structure of the GloRe unit contains five main convolutional layers. First, there are two convolutional layers for reducing and expanding the input feature X3 and the output feature Y3 to increase the dimension and expressiveness of the features. Next, there is a top convolutional layer for generating a double projection B between the coordinate and potential interaction space, thereby realizing the mapping from the coordinate space to the interaction space. Then, there are two middle convolutional layers for global reasoning in the interaction space, in which regional features are encoded as graph nodes, and the parameters of the graph convolution are used to capture global relationships. Finally, there is a convolutional layer for projecting the features of the interaction space back to the coordination space to obtain the final output feature Y3.
[0048] The preset ground-based cloud image fine-grained segmentation network training data is 300 images selected from the comprehensive all-sky cloud image dataset of Kiel University in Germany. These images cover various types of clouds. Under the guidance of meteorological experts, these clouds are divided into five categories and labeled using professional software.
[0049] The cloud image fine-grained data set data to be processed is data from a cloud image analysis of a designated photovoltaic power station that has been put into use.
[0050] The following is a detailed introduction to the training process and application of the ground-based cloud image fine-grained segmentation network based on convolution and attention:
[0051] Step 1: Computer reads training data; uses computer to read cloud image fine-grained data set data, and pre-processes cloud images to facilitate direct data input into the network; the cloud images used are 300 images selected from the comprehensive all-day cloud image data set of Kiel University in Germany, which cover various types of clouds. Under the guidance of meteorological experts, these clouds are divided into five categories and annotated using professional software, laying the foundation for fine-grained segmentation of ground cloud images and photovoltaic power prediction.
[0052] Step 2: Build a ground-based cloud image fine-grained segmentation network using hybrid convolution and Transformer; Figure 2 The overall description of the proposed method is shown. First, a 512×512×3 RGB ground cloud map is used as the input of the network. Then, the width and height of the input image are changed to 1 / 4 of the original through block partitioning, and the feature map dimension is changed to 16 to make it more suitable for the network. Subsequently, the network extracts ground cloud map features of different resolutions through Swin Transformer Block and Patch Merging. The role of Patch Merging is downsampling. Finally, the size of the encoder output result is 16×16×128. The network decoder of the invention consists of four Stages, convolutional layers and activation function layers. Each stage includes a PatchExpanding and multiple Swin Transformer Blocks, such as Figure 3 The figure shows two consecutive Swin TransformerBolcks. The number of Swin Transformer Blocks is an even number, consisting of alternating MSA and W-MSA. Patch Expanding gradually upsamples the ground-based cloud features of different resolutions to the same resolution as the input image. Finally, the output result is obtained through the convolution layer and the Sigmoid activation function layer.
[0053] Among them, the Swin Transformer block introduces the window-based multi-head self-attention W-MSA and the shifted window-based multi-head self-attention SW-MSA mechanism. When calculating the self-attention mechanism in W-MSA and SW-MSA, the calculation expression for each head is:
[0054]
[0055] in, are query matrix, key matrix and value matrix respectively, K T is the transposed matrix of K; m is the window size, d is the sequence feature dimension; represents a learnable relative position encoding matrix whose values are in the parameterized bias matrix , SoftMax represents the SoftMax function.
[0056] The feature calculation formula between two consecutive Swin Transformer blocks is as follows:
[0057]
[0058] z l-1 Represents the output features of the first W-MSA module and MLP module; and zl They represent the output features of the W-MSA module and the MLP module of the second block respectively; With z l+1 They represent the output features of the W-MSA module and the MLP module of the third block respectively; LN represents layer normalization; MLP represents processing through MLP operation; W-MSA and SW-MSA represent processing through W-MSA and SW-MSA respectively.
[0059] The invention designs a feature fusion module Fg-FFM in the encoder structure to fully extract multi-scale features and fuse context information. Its specific structure is as follows: Figure 4 As shown, specifically including:
[0060] In order to change the size of spatial pooling and realize channel attention at multiple scales, and to avoid the introduction of too many parameters, the local context is added to the global context of the attention module, and 1×1 convolution is selected as the local channel context aggregator. This module generates a weight similar to that used for attention operation. Based on this weight, the Fg-FFM feature fusion module is designed and applied to the encoder.
[0061] The feature fusion module Fg-FFM can be characterized by the following mathematical model:
[0062]
[0063] Among them, X1 is the low-level feature map, and Y1 is the high-level semantic feature map. X1 and Y1 are taken as input features, and Z1 is the output of attention feature fusion. η represents the feature weighting operation implemented by the multi-scale attention module.
[0064] The multi-scale attention (MS-AM) module is an important component of the Fg-FFM module. Its specific structure is as follows: Figure 5 The introduction of this module effectively solves the problem of semantic and scale inconsistency in X1 when fusing features of different scales. This module adjusts the size of the spatial pool at multiple levels by using channel attention. This module focuses on the input features from two dimensions: space and channel. Correlation analysis is performed to extract the weight distribution between feature maps, where C, H, and W represent the number of channels, feature map height, and width, respectively. The output feature is then obtained by feature weighting. Depend on Figure 5It can be seen that the MS-AM module adopts a dual-branch structure, in which the spatial branch takes the feature map X2 as input and obtains the correlation distribution in the spatial dimension through a compression excitation operation. The compression excitation operation consists of a convolution layer, a batch normalization layer, and a nonlinear activation function. The convolution layer uses a deep separable convolution to reduce the number of parameters and calculations; the channel branch first performs a global adaptive pooling operation on the input feature map X2, and then performs a compression excitation operation to obtain the correlation distribution in the channel dimension. The specific size change can be expressed as the initial size C×H×W feature map is obtained through a deep separable convolution layer and a BN layer. The feature map is then processed by the Relu activation function and the depth-separable convolution layer to restore the feature map size to C×H×W and C×1×1. After the BN layer, the two feature maps are weighted concatenated to obtain a feature map of size C×H×W. Finally, the Sigmoid activation function is used to multiply the original feature map to obtain the final output feature map.
[0065] By adopting different spatial pooling sizes, the MS-AM module can effectively capture multi-scale features and perform channel attention operations on the features. In order to effectively reduce the number of parameters and computation, a deep separable convolution is used, and the local channel context It can be expressed as
[0066] φ(X2)=β(DSConv2(δ(β(DSConv1(X2))))) (8)
[0067] Among them, X2 is the input feature map, and DSConv1 uses a 1×1 convolution layer to convert the number of channels of the input feature X to the previous β is the BN layer, δ is the Relu activation function, and DSConv2 uses 1×1 convolution to restore the number of channels, so the shape of the local channel information φ(X2) is consistent with the shape of the input. Re-extract features The process is expressed as:
[0068]
[0069] Among them, X2 is the input feature map, It represents the attention weight generated by MS-AM, not the final output of the MS-AM module. G(X2) is the global channel context operation, φ(X2) is the local channel context operation, and δ is the ReLU activation function. It represents broadcast addition. Here, the global channel information uses the global average pooling operation, so the feature height and width shape is 1×1. The local channel information and the input feature have the same height and width, so the addition of the two requires a broadcast operation. Represents element-wise multiplication, where two corresponding elements are multiplied.
[0070] The invention introduces the GloRe global graph reasoning module, whose specific structure is as follows Figure 6 This module realizes the interaction between global and local features by introducing the interaction space, which includes:
[0071] The structure of the GloRe unit contains five main convolutional layers. First, there are two convolutional layers for dimensionality reduction and expansion of the input feature X3 and the output feature Y3 to increase the dimension and expressiveness of the features. Next, there is a top convolutional layer for generating a double projection B between the coordinates and the potential interaction space, thereby realizing the mapping from the coordinate space to the interaction space. Then, there are two intermediate convolutional layers for global reasoning in the interaction space, where regional features are encoded as graph nodes, and the parameters of the graph convolution are used to capture global relationships. Finally, there is a convolutional layer for projecting the features of the interaction space back to the coordination space to obtain the final output feature Y3. Through such a structural design, the GloRe unit can achieve effective interaction between global and local features and improve the model's ability to model complex relationships. First, a projection function Ω(·) is constructed to convert the structured feature map of the pixel space into Node matrix mapped to interaction space To achieve the aggregation of regional features, the projection function is characterized as weighted average pooling:
[0072]
[0073] represents a set of learnable weight vectors, b ij is 0 or 1, represents the texture feature vector decomposed by spatial dimension, Represents the aggregated feature vector. Then, global reasoning of the relationship between nodes is carried out in the form of simplified graph convolution. After projecting the features, the features are used as nodes of the fully connected graph. The fully connected graph is reasoned by learning the edge weights of each node. During the reasoning process, the graph convolution first performs Laplace smoothing to pass the node features to the fully connected graph. You can see two graph convolution modules that implement this process in the interaction space. The efficient graph convolution module is applied in the fully connected graph to enhance the reasoning process. During the training process, the randomly initialized adjacency matrix Ag is learned through gradient icon learning and combined weights. The shortcut connection method of the unit matrix greatly alleviates the difficulty of reasoning optimization. The graph convolution formula is expressed as:
[0074] Z=((IA g )V)W g (11)
[0075] Among them, I is the identity matrix, Ag is the M×N node adjacency matrix, Wg is the state update function, V is the node feature matrix, and Z is the output result of the graph convolution operation. After completing the semantic graph reasoning, in order to map the node matrix from the sparse semantic space back to the dense pixel space, it is also necessary to construct a back-projection function Ω(·). This function needs to complete feature diffusion and restore the node features to the regional level features before aggregation. According to the idea of feature aggregation, the back-projection function is also modeled as two convolution operations. and θ′(·,w θ′ ),in For dimensionality reduction, θ′(·,w θ′ ) is used for feature diffusion, so the output feature It can be characterized as:
[0076]
[0077] in, and θ′(·,w θ′ ) are two convolution operations, x tex is the input feature matrix, w θ′ is the weight parameter of the convolution operation, is the feature matrix at time step t,l+1, maj is the aggregation operation, and Y3 is the output feature matrix.
[0078] Step 3: Train the neural network; After building the network using the Keras deep learning framework, use the labeled training data to train the network, and select the best performing network parameters from the test data for saving; During the training process, the number of network iterations is initialized to 1000. Based on the downward trend of the test loss curve, the experiment finally sets the epoch to 100 to prevent the model from having the best generalization performance in theory under overfitting. Eight ground cloud images are sent to the network for training in a single batch, and the initial learning rate of the Ranger optimizer is set to 0.001. Considering that the convergence speed of different ground cloud image segmentation networks varies, the increase or decrease of the validation set loss function value determines whether to terminate the training. If the validation set loss function value does not decrease after 50 complete trainings, the training is terminated.
[0079] Setp4: Perform fine-grained segmentation on the cloud image; Use the network parameters trained in Step 3 to perform fine-grained segmentation on the cloud image. Input the RGB cloud image and perform automatic image processing to obtain the label output image. The segmentation result is as follows: Figure 7 As shown, the image can be used to analyze the composition of different types of clouds in the current cloud map, which is beneficial to improving the accuracy of photovoltaic power prediction.
[0080] All experiments in this paper were completed on a GPU server with a graphics card model of NVIDIA GeForce GTX4090 and a memory of 24GB. The ground-based cloud image segmentation methods involved were implemented in Python 3.8 based on the Keras 2.6.0 deep learning framework under the Linux operating system with version number Ubuntu 22.04.
[0081] Compared with the prior art, the above technical method has the following advantages:
[0082] The present invention utilizes a combination of a convolutional network and a Transformer model to solve the problem of poor ground-based cloud image segmentation, thereby achieving the effect of improving the accuracy of cloud image segmentation and ensuring that the segmented cloud image can support photovoltaic power prediction.
[0083] The present invention discloses a ground-based cloud image fine-grained segmentation method based on convolution and attention. The method solves the problems of low segmentation accuracy and large number of calculation parameters of the existing cloud image segmentation model, designs a new model that can extract multi-scale features, fuse context information, and realize the interaction between global and local features, and fully learn the codec long-range dependency information, so as to achieve the purpose of improving feature representation.
[0084] The present invention adopts a coding and decoding structure that can supplement the information ignored in feature extraction while mining the high-level semantic information of the image; the Swin Transformer module, the GloRe module and the Fg-FFM module can further enhance the ability to extract global information from the cloud layer while maintaining low computational complexity, which is conducive to application in actual scenarios. This technology has significant application value in the management of photovoltaic power stations, energy scheduling and power market optimization. For example, in the management of photovoltaic power stations, accurate power forecasting can help optimize the operation strategy of the power station, improve power generation efficiency, and reduce economic losses caused by prediction errors. In terms of energy scheduling, accurate forecasting helps the power system to arrange power generation plans more reasonably, avoiding supply and demand imbalances and energy storage difficulties caused by fluctuations in photovoltaic power generation. In power market optimization, accurate photovoltaic power forecasting can improve the rationality and market competitiveness of power transactions, thereby promoting the healthy development of the power market.
[0085] In addition, this technology has good scalability and is not limited to photovoltaic power forecasting. It can also be applied to other fields that require refined meteorological analysis. For example, in wind power forecasting, fine-grained meteorological data processing can improve the ability to capture changes in wind speed and wind direction, thereby improving the accuracy of power generation forecasts. In terms of agricultural meteorological services, refined meteorological analysis can provide more accurate meteorological guidance for crop growth and improve the efficiency and benefits of agricultural production.
[0086] By applying advanced cloud image segmentation technology to actual energy management systems, the present invention not only improves the accuracy of photovoltaic power prediction, but also provides important technical support for the efficient use of renewable energy. This will help achieve energy conservation and emission reduction goals and promote environmental protection and sustainable development.
[0087] The various parts in this manual are described in a progressive manner, and each part focuses on the differences from other parts. The same or similar parts between the various parts can be referenced to each other.
[0088] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A convolution and attention-based ground-based cloud image fine-grained segmentation method, characterized in that: The following steps are involved: Obtaining cloud map fine-grained data set data to be processed, preprocessing the cloud map fine-grained data set data to be processed, and obtaining cloud map fine-grained data set data; Input the cloud image fine-grained data set data into the preset ground-based cloud image fine-grained segmentation network to obtain the cloud image fine-grained segmentation result; The preset ground-based cloud map fine-grained segmentation network is a network built using the Keras deep learning framework. The preset ground-based cloud map fine-grained segmentation network is constructed using a mixed convolution and Transformer ground-based cloud map fine-grained segmentation network. The Swin Transformer Block is used as a basic module to build a segmentation network. The network is a typical encoding-decoding structure. The Fg-FFM module is added to the encoding structure to fuse the multi-scale information extracted by different Transformer layers. The GloRe global graph module is used to replace the jump connection operation. The GloRe unit can realize effective interaction between global and local features. Its lightweight design improves the global modeling efficiency of the model while maintaining low parameters and computational complexity; wherein, the Fg-FFM module is added to the encoding structure to fuse the multi-scale information extracted by different Transformer layers, specifically including: The Fg-FFM module is used to cascade between the Swin Transformer Block layers in the encoder; the feature fusion module Fg-FFM can be represented by the following mathematical model: Among them, X1 is the low-level feature map, Y1 is the high-level semantic feature map, X1 and Y1 are used as input features, Z1 is the output of attention feature fusion, and η represents the feature weighting operation implemented by the multi-scale attention module; Among them, the structure of the GloRe unit contains five main convolutional layers. First, there are two convolutional layers for reducing and expanding the input feature X3 and the output feature Y3 to increase the dimension and expressiveness of the features. Next, there is a top convolutional layer for generating a double projection B between the coordinate and potential interaction space, thereby realizing the mapping from the coordinate space to the interaction space. Then, there are two middle convolutional layers for global reasoning in the interaction space, in which regional features are encoded as graph nodes, and the parameters of the graph convolution are used to capture global relationships. Finally, there is a convolutional layer for projecting the features of the interaction space back to the coordination space to obtain the final output feature Y3.
2. According to the convolution and attention based ground cloud image fine-grained segmentation method of claim 1, it is characterized by: The preset ground-based cloud image fine-grained segmentation network training data is 300 images selected from the comprehensive all-sky cloud image dataset of Kiel University in Germany. These images cover various types of clouds. Under the guidance of meteorological experts, these clouds are divided into five categories and labeled using professional software.
3. According to the convolution and attention based ground cloud image fine-grained segmentation method of claim 1, it is characterized by: The cloud image fine-grained data set data to be processed is data from a cloud image analysis of a designated photovoltaic power station that has been put into use.
4. According to the convolution and attention based ground cloud image fine-grained segmentation method described in claim 1, characterized in that: The designed Fg-FFM module adopts the MS-AM multi-scale attention module; The MS-AM module adopts a dual-branch structure, in which the spatial branch takes the feature map X2 as input and obtains the correlation distribution in the spatial dimension through a compression excitation operation. The compression excitation operation consists of a convolution layer, a batch normalization layer, and a nonlinear activation function. The convolution layer uses a depth-separable convolution to reduce the number of parameters and calculations. The channel branch first performs a global adaptive pooling operation on the input feature map X2, and then performs a compression excitation operation to obtain the correlation distribution in the channel dimension. The specific size change can be expressed as the initial size C×H×W feature map is obtained through the depth-separable convolution layer and the BN layer. The feature map is then processed by the ReLu activation function and the depth-separable convolution layer to restore the feature map size to C×H×W and C×1×1. After the BN layer, the two feature maps are weighted concatenated to obtain a feature map of size C×H×W. Finally, the Sigmoid activation function is used to multiply the original feature map to obtain the final output feature map.
5. According to the convolution and attention based ground cloud image fine-grained segmentation method of claim 1, characterized in that: Taking Swin Transformer Block as the basic module, the encoding and decoding structure is adopted. The input image is segmented and linearly embedded into Swin Transformer Block for feature extraction. At the same time, patch expansion is used for upsampling and then input into Swin Transformer Block. Transformer Block restores and reconstructs semantic information.
6. According to the convolution and attention based ground cloud image fine-grained segmentation method of claim 1, characterized in that: During the training process, the number of network iterations was initially set to 1000. Based on the downward trend of the test loss curve, the experiment finally set the epoch to 100 to prevent the model from having the theoretically optimal generalization performance under overfitting conditions.
Citation Information
Cited By
Foundation cloud picture segmentation method based on deformable Transform-U-shaped network
CN120259327A
E-commerce live broadcast image background replacement method
CN120658920A
Foundation cloud picture classification method and system based on fisheye system
CN120783121A
Tumor cell image extraction method and system based on deep learning
CN120807494A
A deep learning-based tumor cell image extraction method and system
CN120807494B
