A cloud image segmentation method and device, and a storage medium

Abstract

The application discloses a cloud image segmentation method and device, a storage medium and belongs to the technical field of photovoltaic monitoring. In view of the problems that the multi-scale feature extraction of the existing photovoltaic scene cloud image segmentation is insufficient, the lightness adaptability is poor, it is difficult to balance the precision and efficiency, and the embedded deployment is difficult, the standardized image is obtained by preprocessing the input cloud image; the multi-scale feature is extracted by using the encoder containing residual convolution and joint attention module; the features are extracted in parallel by the multi-branch different expansion rate hollow convolution and are adaptively weighted and fused; the lightness adaptation is realized based on the dynamic generation of the global brightness convolution parameter; the light weight is realized by combining the depth separable convolution; and the binary segmentation result is output by the output module. The parameter quantity of the application is controlled within 7.8M, the average intersection over union of segmentation reaches 95.8%, the accuracy reaches 98.0%, the application can be deployed on the embedded terminal, and the all-weather high-precision cloud image segmentation of the photovoltaic scene is realized.

Description

Technical Field

[0001] This invention relates to the field of cloud image processing technology, and in particular to a cloud image segmentation method, device, and storage medium, which are applicable to intelligent monitoring and power prediction scenarios of photovoltaic power plants, and provide data support for the safe and stable operation of photovoltaic power plants and the improvement of power prediction accuracy. Background Technology

[0002] In the actual operation of photovoltaic power plants, cloud cover is one of the core factors causing drastic fluctuations in photovoltaic output power. This not only reduces the accuracy of photovoltaic power prediction but also threatens the safe and stable operation of the power plant. Cloud image segmentation, as a core supporting technology for cloud monitoring and power prediction in photovoltaic power plants, has the core function of performing pixel-level classification of cloud images of photovoltaic scenes captured by ground-based sky cameras. This enables precise differentiation between cloud areas and non-cloud areas, providing a reliable image data foundation for intelligent monitoring and power prediction of the power plant.

[0003] Currently, mainstream ground-based cloud image segmentation technologies all adopt an encoder-decoder architecture. A core feature extraction module extracts cloud image features, an image restoration module recovers the image size, and finally, cloud / non-cloud segmentation is performed according to a preset classification standard, forming a standardized processing flow common in the industry. However, existing cloud image segmentation technologies still have significant technical shortcomings in practical applications in photovoltaic scenarios, making it difficult to meet the actual needs of photovoltaic power plants for all-weather, high-precision, and lightweight applications: First, existing technologies have poor adaptability to lighting environments. Under complex lighting conditions such as strong light, backlight, and weak light, feature extraction accuracy decreases, leading to unstable segmentation results. Second, feature extraction is insufficient for complex cloud layers of different scales and shapes, such as thin clouds and fragmented clouds, making accurate segmentation difficult. Third, existing segmentation models have a large number of parameters and high computational complexity, making them difficult to deploy in embedded terminal devices in photovoltaic power plants, resulting in significant engineering implementation challenges.

[0004] In summary, existing cloud image segmentation technology cannot simultaneously achieve segmentation accuracy, lightweight model, and all-weather adaptability during the day. Its robustness is insufficient, which not only limits the improvement of photovoltaic power prediction accuracy but also fails to meet the actual needs of intelligent monitoring of photovoltaic power plants, thus becoming a technical bottleneck in the process of intelligent upgrading of photovoltaic power plants.

[0005] Therefore, there is an urgent need for a robust, adaptable, and lightweight cloud image segmentation scheme that can adapt to different light variations and accurately identify clouds of various scales, providing reliable technical support for intelligent monitoring and power prediction of photovoltaic power plants. Summary of the Invention

[0006] The main objective of this invention is to provide a cloud image segmentation method, apparatus, and storage medium, aiming to solve the technical problems in existing photovoltaic scenario cloud image segmentation methods, such as insufficient multi-scale feature extraction, poor adaptability to lighting environment, high network computational complexity, difficulty in balancing segmentation accuracy, and inability to meet the high-precision, high-efficiency, and lightweight requirements of all-weather cloud image segmentation for photovoltaic power plant cloud shading monitoring and photovoltaic power prediction.

[0007] To achieve the above-mentioned objectives, this invention provides a cloud image segmentation method. This method employs a neural network architecture consisting of an encoder, a multi-branch feature fusion module, and a lightweight decoder to perform a photovoltaic scene cloud image segmentation process. The method includes the following steps: Step 1: Perform preprocessing operations on the input photovoltaic scene cloud map to obtain a standardized image; the preprocessing includes image numerical normalization and size unification, which are used to eliminate the influence of image size differences and pixel value fluctuations on subsequent feature extraction.

[0008] Step 2: Using an encoder containing a residual convolutional structure and a joint attention module, multi-scale feature extraction is performed on the standardized image to obtain a multi-scale feature map; the residual convolutional structure is configured with an identity mapping branch to maintain feature stability during feature propagation in deep networks; the joint attention module sequentially performs channel dimension weight adjustment and spatial dimension weight adjustment to enhance the feature response of the target region in the feature map.

[0009] Step 3: Perform multi-branch parallel feature extraction on the multi-scale feature map. Each branch uses depthwise separable convolution with different dilation rates to achieve multi-receptive field feature acquisition. After concatenating the outputs of each branch along the channel dimension, dynamic convolution is used to adaptively generate fusion parameters based on global statistical information to achieve dynamic fusion of multi-scale and multi-receptive field features and obtain a fused feature map.

[0010] Step 4: Based on the global feature statistics after the fused feature map is refined by depthwise separable convolution, dynamically generate convolution kernel parameters, use this dynamic convolution to adaptively map the refined features, and obtain an initial segmentation feature map adapted to different lighting conditions after batch normalization and activation.

[0011] Step 5: Use a lightweight convolutional structure to optimize the illumination adaptation feature map. By combining the lightweight convolutional structure with the size magnification operation, the size of the feature map is restored and the features are refined. Finally, the binary segmentation results of the cloud and non-cloud regions are output. The neural network model implementing the method has fewer than 8M parameters to meet the deployment requirements of embedded terminals.

[0012] As a further optimization of the method of the present invention, the image numerical normalization is used to map the image pixel values ​​to a preset numerical range, and bicubic interpolation is used to unify the spatial size, adjusting the image to an RGB image of a preset size; the preset numerical range can be flexibly adjusted according to the actual application scenario, and the preset size can be adapted to the input requirements of the subsequent feature extraction module.

[0013] As a further optimization of the method of the present invention, the residual convolutional structure includes a convolutional layer, a normalization layer, and a ReLU / HardSwish activation layer. The identity mapping branch is set in parallel with the convolutional layer, the normalization layer, and the activation layer to realize the direct propagation of features and stable gradient transfer.

[0014] As a further optimization of the method of the present invention, the multi-branch parallel feature extraction adopts at least four parallel branches, and each branch adopts dilated convolution with different dilation rates to achieve feature extraction of different receptive fields. The dilation rate can be flexibly configured according to cloud scale features. The output features of each branch are concatenated along the channel dimension, and the convolution kernel parameters are adaptively generated based on global statistical information through dynamic convolution to achieve adaptive channel fusion of the concatenated features and obtain a fused feature map.

[0015] As a further optimization of the method of the present invention, the global feature statistics are obtained by performing global average pooling on the intermediate feature map of the decoder. The pooling kernel size is consistent with the feature map size and is compressed into a one-dimensional global feature vector. The feature vector is then passed through two independent two-layer fully connected networks (containing ReLU activation in the middle) to generate convolution kernel weights and bias terms.

[0016] As a further optimization of the method of the present invention, the lightweighting utilizes depthwise separable convolution for feature refinement and channel adjustment; the depthwise separable convolution includes depthwise convolution and pointwise convolution, the depthwise convolution performs convolution on each input channel individually, and the pointwise convolution completes channel fusion, which greatly reduces the computational complexity.

[0017] As a further optimization of the method of the present invention, the number of parameters of the neural network model is in the range of 5M to 8M, the average intersection-union ratio of the cloud map segmentation method is not less than 95%, and the accuracy is not less than 98%, which can meet the requirements of high-precision segmentation in photovoltaic scenarios; in the binary segmentation result, the cloud area and the non-cloud area are marked with different binary pixel values, which facilitates subsequent data processing and application.

[0018] As a further optimization of the method of the present invention, the binary segmentation result can be directly applied to cloud shading monitoring or photovoltaic power prediction scenarios of photovoltaic power plants: In the cloud shading monitoring scenario, the cloud shading area ratio is calculated based on the binary segmentation result, and when the shading area ratio exceeds a preset threshold, a shading warning is triggered; In the photovoltaic power prediction scenario, the binary segmentation result is converted into cloud coverage features and input into the photovoltaic power prediction model to provide auxiliary support for photovoltaic power prediction.

[0019] Furthermore, to achieve the aforementioned objectives, this invention also provides a cloud image segmentation device. The device includes a preprocessing module, a multi-scale feature extraction module, a multi-branch feature fusion module, a feature decoding and illumination adaptation module, and a lightweight module. These modules work collaboratively to complete the segmentation processing of photovoltaic scene cloud images. The preprocessing module is used to standardize the input photovoltaic scene cloud image, perform image numerical normalization and spatial size unification operations, and output a standardized image. The multi-scale feature extraction module includes an encoder containing a residual convolutional structure and a joint attention module, which is used to extract multi-scale features from the standardized image and output a multi-scale feature map. The multi-branch feature fusion module is configured to use a multi-branch parallel feature extraction method to collect features from multi-scale feature maps. Each module is configured to extract features from a single-scale feature map in parallel using multi-branch dilated convolutions with different dilation rates, and then concatenate the outputs of each branch along the channel and perform adaptive fusion through dynamic convolution to obtain an enhanced feature map. The feature decoding and illumination adaptation module is used to extract global brightness information from the fused feature map, dynamically generate convolution parameters based on the global brightness information, and perform convolution processing on the fused feature map using the convolution parameters to output an illumination adaptation feature map. The lightweight module integrates a depthwise separable convolutional structure to reduce the overall computational cost of the model. The output module is used to upsample the segmentation feature map to the original input size through bilinear interpolation, and then output the cloud / non-cloud binary segmentation result by combining the softmax function and threshold determination rules. The device carries fewer than 8M model parameters and is configured for use in photovoltaic cloud shading monitoring and photovoltaic power prediction scenarios.

[0020] In addition, to achieve the above-mentioned objectives, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the cloud map segmentation method described in any of the above technical solutions; the computer-readable storage medium is one or more of a USB flash drive, hard disk, optical disk, or read-only memory (ROM), which facilitates the storage, transmission, and deployment of the program.

[0021] The beneficial effects that this invention can achieve are as follows: This invention proposes a cloud image segmentation method, apparatus, and computer-readable storage medium. Through targeted structural design and process optimization, it accurately addresses the core technical pain points of existing cloud image segmentation technologies in photovoltaic applications, achieving multiple technical advantages, as detailed below: This invention introduces a residual convolutional structure and a joint attention module into the encoder. The residual convolutional structure achieves stable feature transfer through an identity mapping branch, effectively avoiding the gradient vanishing problem during deep network training and ensuring the depth and stability of feature extraction. The joint attention module, through dual weight adjustment of the channel and spatial dimensions, can accurately enhance the feature response of the target region, suppress background interference, and improve the accuracy of feature extraction. Simultaneously, a multi-branch parallel feature extraction structure, combined with an adaptive weight allocation method, achieves full acquisition and dynamic fusion of features from different receptive fields, solving the problem that single-scale features are difficult to adapt to cloud layers of different sizes and shapes, further improving segmentation accuracy.

[0022] This invention designs an illumination-adaptive convolution processing flow. By extracting global brightness information from the fused feature map, it dynamically generates convolution parameters, enabling adaptive adjustment of convolution operation parameters. This allows it to adapt to different lighting conditions such as strong light and weak light, effectively overcoming the impact of lighting changes on segmentation results. It ensures accurate cloud image segmentation in all weather conditions and lighting scenarios, solving the technical pain point that traditional fixed convolution kernels cannot adapt to complex lighting environments.

[0023] This invention adopts a lightweight design approach, and through the collaborative optimization of depthwise separable convolution and multi-branch dynamic fusion, it significantly reduces the number of network parameters and computational load, strictly controlling the number of neural network model parameters to within 8M (preferably 5M~8M). At the same time, through process optimization, it improves the segmentation accuracy (average intersection-over-union ratio not less than 95%), which can meet the real-time segmentation requirements of photovoltaic scenarios and is easy to deploy in embedded terminal devices in photovoltaic power plants, reducing the difficulty of engineering implementation.

[0024] The technical solution of this invention has strong practical application value. The output cloud / non-cloud binary segmentation results can be directly connected to the core application scenarios of photovoltaic power plants. By calculating the proportion of cloud shading area, shading early warning can be achieved. The segmentation results are transformed into cloud coverage characteristics to assist in photovoltaic power prediction. This builds a bridge between cloud image segmentation technology and actual photovoltaic applications, effectively improving the intelligence level of photovoltaic power plant operation monitoring and the accuracy of power prediction, and providing reliable support for the safe and stable operation of photovoltaic power plants.

[0025] In summary, this invention solves the technical problems of insufficient multi-scale feature extraction, poor illumination adaptability, high computational complexity, and difficulty in balancing accuracy and efficiency in existing photovoltaic scene cloud image segmentation through a complete technical chain of "preprocessing-multi-scale feature extraction-multi-branch dynamic fusion-illumination adaptive convolution-lightweighting". It achieves all-weather, high-precision, high-efficiency, and lightweight cloud image segmentation, and provides reliable technical support for photovoltaic power plant monitoring and power prediction, with broad application prospects and practical value. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the all-weather cloud image segmentation process according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the dynamic multi-branch adaptive cloud map segmentation network architecture according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the multi-scale dynamic fusion module MSDFB structure according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the adaptive dynamic convolution module ADConv in an embodiment of the present invention. Figure 5 This is a schematic diagram of the device structure according to an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to technical details, experimental data, and practical application scenarios. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0028] In related technologies, photovoltaic scene cloud image segmentation methods generally suffer from problems such as insufficient multi-scale feature extraction, poor adaptability to lighting environments, complex network structures, and difficulty in balancing segmentation accuracy and efficiency, failing to meet the requirements of embedded deployment and all-weather real-time monitoring. This invention addresses these technical pain points through modular design, lightweight architecture optimization, and the introduction of adaptive mechanisms. The following detailed description, in conjunction with specific embodiments corresponding to the claims, illustrates these issues.

[0029] Example 1 like Figure 1 – Figure 4 As shown, this embodiment provides a cloud map segmentation method as a basic embodiment of the present invention, used to illustrate the complete segmentation process and core technical indicators. The specific implementation process is as follows: The cloud image segmentation method in this embodiment is based on a neural network model. The experimental environment configuration is as follows: hardware uses a 32-core Intel(R) Xeon(R) CPU E5-2620v4@2.10GHz processor and an NVIDIA Tesla V100-DGXS-32GB graphics card; software uses the PyTorch 1.12 framework and Python 3.8 programming language; the dataset uses a publicly available photovoltaic scene cloud image dataset, containing 6768 samples of strong daytime light, weak evening light, and low nighttime illumination, with the resolution uniformly adjusted to 320×320. The Adam optimizer is used during training, with an initial learning rate of 1×10⁻⁶. -3 The batch size is set to 16, and the training rounds are 100.

[0030] Step 1: Input photovoltaic scene cloud map preprocessing to obtain standardized image; The input image is an RGB cloud image captured by a ground-based camera in a photovoltaic power station, which is prone to problems such as inconsistent pixel value ranges and inconsistent sizes. The preprocessing operations specifically include: scaling the image to 320×320 pixels, using bicubic interpolation to maintain image integrity, and normalizing the pixel values ​​from 0–255 to the 0–1 range to obtain a standardized image, laying the foundation for subsequent feature extraction.

[0031] Step 2: Using an encoder containing a residual convolutional structure and a joint attention module, perform multi-scale feature extraction on the standardized image to obtain a multi-scale feature map; The encoder employs an improved residual model structure. It first reduces the input resolution using two 3×3 convolutions with a stride of 2, generating a shallow feature map with 64 channels. This is followed by three residual stages: Stage 2, Stage 3, and Stage 4. Each stage contains one downsampled convolutional residual block and three identity residual blocks. The ConvBlock achieves downsampling through 1×1 convolutions with a stride of 2. The residual blocks use a bottleneck structure: the main branch sequentially performs 1×1 convolution (dimensionality reduction) - batch normalization - ReLU / HardSwish activation, 3×3 convolution - batch normalization - ReLU / HardSwish activation, and 1×1 convolution (dimensionality increase) - batch normalization. The identity mapping branch, when the input and output channels mismatch, is aligned using a 1×1 convolution, added to the main branch, and then activated by ReLU or HardSwish for output.

[0032] CBAM joint attention modules are embedded in specific identity residual blocks of Stage 2, Stage 3, and Stage 4. Channel attention uses both global average pooling and global max pooling to obtain two feature descriptors, which are then summed after sharing two fully connected layers and processed by Sigmoid to generate channel weights. Spatial attention performs channel-dimensional average and max pooling on the channel-weighted features, concatenates them, and then generates a spatial weight map through a 7×7 convolution for pixel-wise weighting.

[0033] The backbone network ultimately outputs feature maps at four scales: 64 channels, 128 channels, 256 channels, and 512 channels.

[0034] Step 3: Perform multi-branch parallel feature extraction on the input multi-scale feature map, and use an adaptive weight allocation strategy to dynamically weight and fuse the output features of each branch to obtain a fused feature map.

[0035] refer to Figure 3 As shown, its specific implementation is as follows: each single-scale feature map is input into three independent feature extraction modules for parallel processing; each module has four built-in parallel branches, and the branch structure is configured as follows: Branch 0: a conventional depthwise separable convolution with a stride of 1 and an dilation rate of 1 followed by a 1×1 convolution; Branch 1: a depthwise separable convolution with a stride of 1 and a dilation rate of 3 followed by a 1×1 convolution; Branch 2: a 1×1 convolution, a 5×5 convolution, and a depthwise separable convolution with a dilation rate of 5 are cascaded in sequence; Branch 3: a 1×1 convolution, a 7×7 convolution, and a depthwise separable convolution with a dilation rate of 7 are cascaded in sequence.

[0036] Each branch forms a differentiated receptive field by setting different void expansion rates, which is adapted to the feature representation of four types of cloud targets: fragmented clouds, thin clouds, thick clouds, and cloud edges.

[0037] The adaptive dynamic weighted fusion process is as follows: The output features of the four branches within the same module are concatenated along the channel dimension. Global average pooling is then performed on the concatenated features to obtain a one-dimensional global feature vector. This vector is then input into a dynamic convolutional layer parameter generator to dynamically generate the weights and biases of a 1×1 convolutional kernel. The generated dynamic 1×1 convolution is used to complete the channel fusion mapping of the concatenated features, while residual connections are constructed using dynamic convolutions with the same structure. The two feature paths are then added element-wise and non-linearly activated by ReLU to output an enhanced feature map with a fixed channel dimension.

[0038] Step 4: Based on the global brightness information of the fused feature map, dynamically generate convolution parameters, and use the convolution parameters to perform convolution processing on the fused feature map to obtain the illumination adaptation feature map; refer to Figure 4As shown, specifically: global average pooling is performed on the decoder feature map after depthwise separable convolution thinning to compress it into a one-dimensional global feature vector; this global feature vector is then input into two independent fully connected networks. The weight generation network outputs convolutional kernel weights through two fully connected layers, and the bias generation network outputs bias terms through two fully connected layers. Using the generated dynamic convolutional weights and biases, a 1×1 convolution operation with a stride of 1 and padding of 1 is performed on the thinned feature map, followed by batch normalization and HardSwish activation to obtain the initial segmentation feature map.

[0039] The dynamically generated convolutional kernel can adaptively adjust parameters based on global statistical characteristics such as average brightness and contrast: in daytime strong light scenes, it can suppress high light noise and enhance the feature discrimination ability of clouds and sky; in nighttime low light scenes, it can strengthen weak feature signals and suppress dark area noise, realize adaptive feature mapping under different lighting conditions, and finally output lighting-adaptive feature maps.

[0040] Step 5: Upsample the illumination adaptation feature map and output the binary segmentation results of the cloud and non-cloud regions; The segmentation feature map obtained in step 4 is directly upsampled to the original input size through bilinear interpolation; the output is mapped to the 0–1 interval through Softmax, with 0.5 as the segmentation threshold. Output values ​​greater than or equal to 0.5 are recorded as cloud regions, and those less than 0.5 are recorded as non-cloud regions, thus obtaining the binary segmentation results of cloud and non-cloud regions.

[0041] The neural network model in this embodiment is optimized using lightweight techniques such as depthwise separable convolution and channel pruning, keeping the total number of parameters to 7.8M, which is less than 8M, thus meeting the lightweight requirements. Tested on the aforementioned experimental dataset, the average intersection-over-union ratio is 95.8%, which can meet the all-weather real-time monitoring needs of photovoltaic scenarios.

[0042] refer to Figures 1-2 As shown, Figure 2 This diagram illustrates the overall architecture of the Dynamic Multi-Branch Adaptive Cloud Image Segmentation Network (DMASNet), showcasing the end-to-end processing flow from input photovoltaic scene cloud image to output binary segmentation results of cloud and non-cloud regions. The network consists of three main parts: an encoder, a multi-scale dynamic fusion module (MSDFB), and a decoder. The encoder extracts multi-scale features from the input image through a multi-level residual stage (including downsampling convolutional blocks and identity residual blocks embedded with CBAM joint attention) and outputs feature maps with different channel numbers. The extracted multi-scale features are processed by three MSDFB modules, using parallel dilated convolutional branches with different dilation rates to complete multi-receptive field feature acquisition, and then dynamically fused to generate enhanced features. After receiving the fused features, the decoder performs illumination-robust feature mapping through an adaptive dynamic convolution (ADConv) module, and then restores the feature map to its original size through bilinear upsampling, finally outputting the cloud image. Figure 2 The image below shows the cloud image segmentation results. The legend symbols below the image explain core operations such as ordinary convolution, dilated convolution, depthwise separable convolution, batch normalization, activation functions, upsampling, feature concatenation, and addition, fully presenting the network's hierarchical structure, data flow, and implementation details of key modules. The overall cloud image segmentation network adopts a hierarchical architecture: residual convolution feature extraction backbone - joint attention enhancement module - multi-branch dilated convolution feature fusion module - illumination adaptive dynamic convolution mapping module, constructing an end-to-end complete inference link from the original remote sensing cloud image input to the cloud segmentation result output.

[0043] The network first inputs the original remote sensing cloud image into a bottleneck-type residual convolutional backbone network, and then mines shallow texture features and deep semantic features layer by layer based on the residual structure. The main branch of the residual module adopts a three-layer stacked structure of 1×1 convolutional channel compression + 3×3 convolutional spatial feature extraction + 1×1 convolutional channel expansion, and completes nonlinear transformation of features by combining batch normalization and nonlinear activation. At the same time, the identity mapping branch realizes cross-layer residual connection, gradient backpropagation and channel dimension alignment, which can not only fully preserve the detailed information of the cloud image, but also effectively alleviate the gradient degradation problem of deep networks, and output the basic cloud image feature map after multi-level encoding.

[0044] The basic feature map is integrated into the joint attention enhancement module to enhance features from both channel and spatial dimensions. Channel attention employs global average pooling and global max pooling in parallel to mine global information of each channel. Channel weights are generated by encoding and fusing through a shared fully connected network, achieving channel-wise weighted importance of the feature map. Spatial attention aggregates the mean and maximum features along the channel dimension of the enhanced feature map, and after concatenation, generates a spatial weight map through a 7×7 convolution. This completes pixel-wise adaptive calibration, highlighting the target cloud area, suppressing background noise on the ground and sky, and widening the feature difference between cloud areas and non-cloud areas.

[0045] The attention-enhanced feature map is input into the multi-branch dilated convolution feature fusion module. Four parallel dilated convolution branches with dilation rates of 1, 3, 5, and 7 are set to adapt to capture multi-scale contextual features of fragmented clouds, thin clouds, thick clouds, and cloud edges, respectively. After the features of each branch are concatenated by channels, dynamic convolution guided by global features adaptively generates fusion weights, intelligently aggregates multi-scale cloud features, adapts to complex and ever-changing cloud morphologies, and outputs a global fused feature map.

[0046] Finally, the global fused feature map is fed into the illumination-adaptive dynamic convolutional mapping module. By extracting global statistical information such as brightness and contrast from the feature map, the module dynamically generates 1×1 convolutional kernel weights and bias parameters using fully connected layers. This allows for adaptive adaptation to three types of illumination scenarios: strong daylight, low nighttime illumination, and mixed day and night illumination. In strong light scenarios, it suppresses high light interference, while in low-light scenarios, it enhances details of weak cloud layers. After convolution, normalization, and nonlinear activation, the module completes the illumination-robust feature mapping and outputs the illumination-adaptive feature map.

[0047] The network achieves high-precision semantic segmentation of cloud images under complex lighting and multi-morphological cloud conditions by progressively completing feature extraction, attention enhancement, multi-scale fusion, and adaptive illumination correction. It can maintain stable segmentation performance in daytime, nighttime, and mixed day-night scenes, and has excellent scene generalization ability and illumination robustness.

[0048] Performance metrics and application advantages; Model parameter count: 7.8M; Average intersection-over-union ratio: 95.8%; Accuracy: 98.0%; Compared with traditional segmentation networks such as U-Net, SegNet, and FCN, this network has lower parameter count, higher segmentation accuracy, and faster inference speed. It can be directly deployed in embedded monitoring terminals of photovoltaic power plants to meet the engineering application requirements of all-weather, high-precision, and real-time cloud map segmentation.

[0049] Example 2 like Figure 5 As shown, this embodiment provides a cloud image segmentation device, which corresponds to the cloud image segmentation method described in Embodiment 1. This device is used to achieve all-weather, high-precision, lightweight real-time segmentation of photovoltaic scene cloud images and can be deployed in embedded terminals of photovoltaic power plants. The device includes: a preprocessing module, a multi-scale feature extraction module, a multi-branch feature fusion module, a feature decoding and illumination adaptation module, a lightweight module, and an output module. Each module is connected sequentially according to data flow logic, forming a complete end-to-end cloud image segmentation processing link. The function of each module corresponds one-to-one with the processing steps of the method embodiment. The specific structure, connection relationship, and function are as follows:

[0050] The output of the preprocessing module is connected to the input of the multi-scale feature extraction module, used to transmit the preprocessed standardized image to the multi-scale feature extraction module; the output of the multi-scale feature extraction module is connected to the input of the multi-branch feature fusion module, used to transmit the extracted multi-scale feature map to the multi-branch feature fusion module; the output of the multi-branch feature fusion module is connected to the input of the feature decoding and illumination adaptation module, used to transmit the dynamically weighted fused feature map to the feature decoding and illumination adaptation module; the output of the feature decoding and illumination adaptation module is connected to the input of the output module, used to transmit the illumination-adapted feature map to the output module; the output of the output module serves as the overall output of the device, used to output the binary segmentation results of cloud and non-cloud areas for subsequent analysis and application by the photovoltaic power station monitoring system.

[0051] The preprocessing module corresponds to the preprocessing steps in Embodiment 1 and the preprocessing optimization embodiment. It is configured to receive RGB cloud images captured by a ground-based camera in a photovoltaic power station, perform standardized preprocessing operations on the input cloud images, eliminate image scale and pixel differences, and unify input specifications, providing high-quality image input for subsequent feature extraction. Specifically, it includes a numerical normalization unit and a size unification unit. The numerical normalization unit uses a minimum-maximum normalization method to map the cloud image pixel values ​​from 0–255 to the 0–1 range, accelerating the convergence speed of the network model. The size unification unit uses a bicubic interpolation algorithm to uniformly adjust the input cloud images to a fixed-size 320×320 RGB image. For ultra-high resolution images, the effective central field of view is retained before scaling to ensure uniform input image specifications. After processing, the standardized image is sent to the multi-scale feature extraction module.

[0052] The multi-scale feature extraction module corresponds to the feature extraction steps in Embodiment 1 and the residual convolution and joint attention optimization embodiments. It is configured to receive the standardized image output by the preprocessing module, and use a multi-level residual convolution network integrating residual convolution structure and joint attention module to complete multi-level, multi-scale feature extraction and output multi-scale feature maps. The specific configuration is as follows: First, the input resolution is compressed using two 3×3 convolutions with a stride of 2 to generate a 64-channel shallow feature map. Subsequently, three residual stages are set up sequentially. Each residual stage contains one convolutional residual block with downsampling and multiple identity residual blocks, outputting 128-channel, 256-channel, and 512-channel deep features level by level. The downsampling convolutional residual block halves the feature map resolution using a 1×1 convolution with a stride of 2, while the identity residual blocks maintain the same feature scale. Each residual block adopts a bottleneck structure design, consisting of stacked 1×1, 3×3, and 1×1 convolutions combined with batch normalization and ReLU / HardSwish activation functions, and configured with an identity mapping branch. A CBAM channel-space joint attention module is embedded within some identity residual blocks to adaptively correct the channel and spatial dimensions of the features, enhancing the feature response of the cloud target region. The encoder finally outputs a multi-scale feature map and sends it to the downstream module.

[0053] The multi-branch feature fusion module corresponds to the feature fusion step in Embodiment 1 and the multi-branch fusion optimization embodiment. It is configured to receive the single-scale feature map output by the multi-scale feature extraction module, complete the multi-receptive field feature acquisition in parallel through multiple sets of dilated convolution branches with different dilation rates, and complete adaptive fusion by concatenating the multi-branch output features and combining them with dynamic convolution to output an enhanced fused feature map. The specific configuration includes four parallel feature extraction branches and a set of dynamic convolutional fusion units. Each branch first performs channel compression through 1×1 convolution. The first branch uses a depthwise separable convolution with an expansion rate of 1. The second, third, and fourth branches are configured with 3×3 depthwise separable convolutions with expansion rates of 3, 5, and 7, respectively, to adapt to the feature extraction requirements of small-scale fragmented clouds, medium-scale thin clouds, large-scale thick clouds, and cloud edge contours. The output features of the four branches are concatenated and fused along the channel dimension. Global average pooling is performed on the concatenated features to obtain a one-dimensional global feature vector. This feature vector is input into the dynamic convolution parameter generator to adaptively generate 1×1 convolution kernel weights and bias parameters. The dynamic convolution is used to complete the cross-channel feature fusion mapping, outputting an enhanced feature map with a fixed number of channels. The fused feature map is then sent to the feature decoding and illumination adaptation module.

[0054] The illumination adaptation processing steps in the feature decoding and illumination adaptation module corresponding to Embodiment 1 and the illumination adaptation convolution optimization embodiment are configured to receive the fused feature map output by the multi-branch feature fusion module, dynamically generate convolution operation parameters based on the global brightness information of the image, complete feature mapping using adaptive convolution, and output illumination adaptation feature map to improve the segmentation robustness of the device in complex illumination scenes. The specific configuration includes a global brightness extraction unit, a dynamic convolution parameter generation unit, and an adaptive convolution processing unit. The global brightness extraction unit performs global average pooling on the fused feature map to compress and generate a one-dimensional brightness feature vector, with the vector mean representing the global illumination intensity of the image. The dynamic convolution parameter generation unit integrates a global average pooling layer, a flattening layer, and two sets of independent fully connected networks to generate convolution kernel weight parameters and bias term parameters, respectively. The parameter dimensions match the input and output channels of the convolution operation and the convolution kernel size. The adaptive convolution processing unit calls the dynamically generated convolution parameters to perform a convolution operation with a stride of 1 and padding of 1 on the fused feature map, generating an illumination adaptation feature map with the same scale as the original image. The convolution weights can be adaptively adjusted according to scenes such as strong light, weak light, and low illumination to achieve high light noise suppression and weak feature enhancement. The processed illumination adaptation feature map is then sent to the output module.

[0055] The lightweight module integrates a depthwise separable convolutional structure, which is composed of cascaded 3×3 depthwise convolutions and 1×1 pointwise convolutions. Relying on the characteristics of depthwise convolution for channel-wise feature extraction and pointwise convolution for cross-channel fusion, it completes feature refinement and channel dimension adjustment, significantly reducing the amount of computation and model parameters.

[0056] The output module corresponds to the upsampling and segmentation output steps in Embodiment 1 and the optimized embodiment. It is configured to receive the illumination adaptation feature map output by the feature decoding and illumination adaptation module, reconstruct the feature map scale through bilinear upsampling, and finally output the binary segmentation results of the cloud region and the non-cloud region. Specifically, it includes a multi-level bilinear upsampling unit and a binarization output unit. The upsampling unit restores the feature map to the original input image size step by step through bilinear interpolation. The binarization output unit uses the Sigmoid activation function to generate a pixel probability map, using 0.5 as the classification threshold. Regions with a probability greater than or equal to 0.5 are classified as cloud regions, and regions with a probability less than 0.5 are classified as non-cloud regions. A binarized segmented image is generated according to the threshold rule, completing the complete cloud image segmentation process.

[0057] The cloud image segmentation device described in this embodiment adopts a modular architecture design, with each module logically independent and functionally coupled, strictly corresponding to the technical solution in the method embodiment. Through optimization techniques such as depthwise separable convolution, multi-branch lightweight fusion, and channel simplification, the overall model parameter size is controlled within 7.8M, meeting the requirements for low-cost embedded deployment. This device is compatible with mainstream photovoltaic embedded terminal hardware such as ARM Cortex-A76, and stably adapts to complex working conditions such as strong daytime light, weak evening light, and low nighttime illumination, achieving high-precision real-time segmentation of photovoltaic scene cloud images in all weather conditions. It effectively solves the technical defects of traditional cloud image segmentation equipment, such as weak light adaptability, insufficient multi-scale cloud layer segmentation accuracy, and excessively large model size that makes it difficult to implement in engineering.

[0058] Example 3 This embodiment provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the cloud map segmentation method as described in Embodiment 1, ensuring that the storage medium fully corresponds to the method embodiment and the device embodiment, and can support the functional implementation of each module in the device embodiment.

[0059] The computer program is written in a processor-recognizable programming language and corresponds to each step of the cloud image segmentation method. Specifically, it includes preprocessing instructions, multi-scale feature extraction instructions, multi-branch feature fusion instructions, illumination-adaptive convolution instructions, and output instructions. Each instruction corresponds to a specific step in Example 1 and its optimized implementation. The execution logic between instructions is consistent with the order of the method steps, as detailed below:

[0060] Preprocessing instructions: When executed by the processor, these instructions normalize the values ​​and unify the size of the input photovoltaic scene cloud image, and output a standardized image, corresponding to step 1 in the method embodiment and the preprocessing optimization embodiment. Multi-scale feature extraction instruction: When executed by the processor, it implements an encoder containing a residual convolution structure and a joint attention module to perform multi-scale feature extraction on the standardized image and output a multi-scale feature map, corresponding to step 2 in the method embodiment and the residual convolution and joint attention optimization embodiment; Multi-branch feature fusion instruction: When executed by the processor, it implements parallel feature extraction using four dilated convolution branches with different dilation rates. After concatenating the output features of each branch along the channel, it generates fusion parameters based on global statistical information through dynamic convolution, adaptively completes multi-scale feature fusion, and outputs a fused feature map, corresponding to step 3 in the method embodiment and the multi-branch fusion optimization embodiment. Illumination-adaptive convolution instruction: When executed by the processor, it performs global average pooling on the feature map refined by the decoder to obtain a one-dimensional feature vector, thereby dynamically generating convolution kernel parameters. It then uses this dynamic convolution to adaptively map the refined feature map and outputs a segmentation feature map, corresponding to step 4 in the method embodiment and the illumination-adaptive convolution optimization embodiment. Output instructions: When executed by the processor, the segmentation feature map is directly upsampled to the input image size through bilinear interpolation, mapped to class probabilities by Softmax, and then the binary segmentation results of cloud and non-cloud regions are determined by thresholding, corresponding to step 5 in the method embodiment and the output optimization embodiment.

[0061] Computer-readable storage media are non-volatile storage media that can be read by a computer, including but not limited to: read-only memory (ROM), random access memory (RAM), magnetic disks (such as hard disks and floppy disks), optical disks (such as CD-ROMs and DVD-ROMs), USB flash drives, solid-state drives (SSDs), server storage media, and other media that can store computer programs. This embodiment does not make specific limitations.

[0062] This computer-readable storage medium can be applied to devices such as servers and embedded monitoring terminals in photovoltaic power plants. When the device's processor loads and executes the computer program in the storage medium, the cloud image segmentation method described in this invention can be implemented without additional hardware configuration, reducing the deployment cost of cloud image monitoring in photovoltaic scenarios. At the same time, it ensures segmentation accuracy and real-time performance, works in conjunction with the device embodiment, and supports the engineering application of this invention.

[0063] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A cloud image segmentation method, characterized in that, A neural network architecture consisting of an encoder, a multi-branch feature fusion module, and a lightweight decoder is used to perform the photovoltaic scene cloud map segmentation process, including: The input photovoltaic scene cloud map is preprocessed to obtain a standardized image; The standardized image is subjected to multi-scale feature extraction via an encoder that integrates a residual convolutional structure and a joint attention module, and a multi-scale feature map is output. Multi-branch parallel feature extraction is performed on the multi-scale feature map, and the output features of each branch are dynamically weighted and fused through an adaptive weight allocation mechanism to obtain a fused feature map. Based on the global brightness information of the fused feature map, convolution parameters are adaptively generated, and the fused feature map is convolved using the generated convolution parameters to obtain the illumination adaptation feature map. The illumination adaptation feature map is light-weighted and refined using depthwise separable convolution. The refined feature map is then upsampled to output a binary segmentation result of the cloud region and the non-cloud region.

2. The method according to claim 1, characterized in that, The preprocessing includes image numerical normalization and spatial size unification processing; The image numerical normalization is used to map image pixel values ​​to a preset numerical range; The spatial size unification processing is used to adjust the image to a preset size RGB image.

3. The method according to claim 1, characterized in that, The residual convolutional structure includes a convolutional layer, a normalization layer, and an activation layer; The residual convolutional structure is configured with identity mapping branches to maintain feature stability during propagation in deep networks.

4. The method according to claim 1, characterized in that, The joint attention module is configured to sequentially perform channel dimension weight adjustment and spatial dimension weight adjustment to enhance the feature response of the target region in the feature map.

5. The method according to claim 1, characterized in that, The multi-branch parallel feature extraction employs multiple parallel branches, each branch using separable convolutions with different dilation rates and hole depths to extract features from different receptive fields. The output features of each branch are concatenated along the channel dimension. The kernel parameters are adaptively generated based on global statistical information through dynamic convolution to achieve adaptive channel fusion of the concatenated features, resulting in the fused feature map.

6. The method according to claim 1, characterized in that, The global brightness information is obtained by performing global average pooling on the fused feature map to obtain a one-dimensional brightness feature vector. The one-dimensional brightness feature vector is used to generate convolutional kernel weights and bias terms through two independent fully connected networks.

7. The method according to claim 1, characterized in that, The lightweight approach utilizes depthwise separable convolution to achieve feature refinement and channel fusion. The depthwise separable convolution includes depthwise convolution and pointwise convolution, which effectively reduces the amount of computation.

8. The method according to claim 1, characterized in that, The number of parameters in the neural network model is in the range of 5M to 8M, and the average intersection-union ratio of the cloud map segmentation method is not less than 95%.

9. A cloud image segmentation device, characterized in that, include: The preprocessing module is used to perform standardization processing on the photovoltaic scene cloud map and output a standardized image; The multi-scale feature extraction module includes an encoder with a built-in residual convolutional structure and a joint attention module, used to extract multi-scale feature maps, which include shallow feature maps and multi-level deep features. The multi-branch feature fusion module consists of multiple independently configured multi-scale dynamic fusion units. Each multi-scale dynamic fusion unit is used to extract features in parallel from the corresponding single-scale feature map using dilated convolution branches with different dilation rates. After the feature outputs of each branch are concatenated, adaptive fusion is completed through dynamic convolution to output an enhanced feature map. The feature decoding and illumination adaptation module is used to concatenate the enhanced feature maps with the shallow feature maps, and then refine the features through depthwise separable convolution. It also dynamically generates 1×1 convolution parameters based on global statistical information and uses adaptive convolution to map the refined features to obtain the segmentation feature map. The lightweight module integrates a depthwise separable convolutional structure to reduce the overall computational cost of the model. The output module is used to upsample the segmentation feature map to the original input size through bilinear interpolation, and then combine the Softmax function and threshold determination rules to output the cloud / non-cloud binary segmentation result. The device carries fewer than 8M model parameters and is suitable for photovoltaic cloud shading monitoring and photovoltaic power prediction scenarios.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the cloud map segmentation method as described in any one of claims 1 to 8.

Images