Foundation cloud picture segmentation method

Abstract

The invention relates to a foundation cloud picture segmentation method. The method comprises the following specific steps: synchronously acquiring foundation cloud picture data of a visible light wave band and foundation cloud picture data of an infrared wave band by adopting a double-spectrum imaging system; a deformable double-branch convolutional network is constructed, and the deformable double-branch convolutional network comprises a main branch embedded into a dynamic cavity convolution module based on a CSPDarknet53 structure and an auxiliary branch integrated with a polar coordinate attention mechanism; respectively extracting multi-scale texture features of the foundation cloud picture data of the visible light band and a thermodynamic feature map of the foundation cloud picture data of the infrared band by using a deformable double-branch convolutional network; constructing a spectral migration gating unit to perform feature compensation and feature fusion on the extracted multi-scale texture features and the thermodynamic feature map to generate fusion features; constructing a semantic segmentation network model based on deep learning, and introducing a meteorological dynamics constraint item module; the method comprises the following steps: acquiring a fusion feature and a historical segmentation condition of a foundation cloud picture based on historical foundation cloud picture data, and pre-training a semantic segmentation network model;

Description

Technical Field

[0001] The present invention relates to the technical field of meteorological monitoring and artificial intelligence, and specifically to a ground-based cloud image segmentation method. Background Art

[0002] Ground-based cloud image segmentation is a key technology in meteorological monitoring. Its goal is to identify and classify cloud boundaries in cloud images acquired by ground-based observation equipment. Existing technologies suffer from the following major technical bottlenecks: First, the limitations of single-modality segmentation methods: a single spectrum can easily lead to missed detection of thin cloud boundaries and misjudgment of cloud types, while a single spectrum also lacks dynamic adaptability. Second, the technical shortcomings of multimodal fusion: difficulties in aligning cross-modal features and the lack of physical constraints for end-to-end learning. Third, the engineering challenges of edge computing deployment: existing models are complex and their real-time performance cannot meet requirements. Furthermore, there are challenges such as sensitivity to light and noise interference, insufficient coordination of multi-source data, and limited edge resources.

[0003] Currently, there are many methods for segmenting ground-based cloud images using single spectrum and deep learning models. Among them, technical means for fusing multi-source data are also used, such as a ground-based visible light cloud image recognition and processing method disclosed in the Chinese invention patent with patent number "CN102902956A" and a dual-branch infrared and visible light image fusion method disclosed in the Chinese invention patent with patent number "CN118823533A".

[0004] The aforementioned approach uses a single visible light image to identify ground-based cloud images. This is susceptible to interference from solar flares in low-light, highly reflective scenes, resulting in a high rate of missed detection of thin cloud boundaries. Furthermore, similar cloud layers often have similar reflectivities, leading to frequent misjudgments. Existing dual-spectral segmentation approaches directly combine visible and infrared feature maps, failing to account for differences in spectral response. This leads to coupling errors between thermodynamic and texture features. Summary of the Invention

[0005] In order to solve the above problems in the prior art, the present invention proposes a ground-based cloud image segmentation method.

[0006] The technical solutions of the present invention are as follows:

[0007] In one aspect, the present invention provides a ground-based cloud image segmentation method, which specifically comprises the following steps:

[0008] A dual-spectral imaging system is used to synchronously collect ground-based cloud image data in the visible light band and infrared band;

[0009] A deformable two-branch convolutional network was constructed, consisting of a main branch embedded with a dynamic dilated convolutional module based on the CSPDarknet53 architecture, and an auxiliary branch integrating a polar coordinate attention mechanism. The deformable two-branch convolutional network was used to extract multi-scale texture features of ground-based cloud image data in the visible light band and thermodynamic feature maps of ground-based cloud image data in the infrared band.

[0010] Construct a spectral migration gating unit to perform feature compensation and feature fusion on the extracted multi-scale texture features and thermodynamic feature maps to generate fusion features;

[0011] A semantic segmentation network model based on deep learning is constructed, and a meteorological dynamic constraint module is introduced. Based on the historical ground-based cloud image data, the fusion features and the historical segmentation of the ground-based cloud image are obtained, and the semantic segmentation network model is pre-trained. The fusion features of the newly collected ground-based cloud image data are input into the trained semantic segmentation network model to perform semantic segmentation on the target ground-based cloud image.

[0012] As a preferred embodiment, the dynamic dilated convolution module includes a dilation rate selection module and a multi-scale convolution kernel group.

[0013] As a preferred implementation, the dynamic dilated convolution module dynamically adjusts the dilation rate of the convolution kernel through a dilation rate selection module, and dynamically adjusts the feature scale extracted by the convolution kernel.

[0014] As a preferred embodiment, the feature scale extracted by the convolution kernel is selected by building in a set of discrete void rate candidate sets, covering multiple scales from local details to global semantics, and dynamically selecting and activating the built-in void rate based on the type of cloud layer in the collected ground-based cloud image data.

[0015] As a preferred embodiment, the spectrum migration gating unit includes a modality difference measurement module, a dynamic weight allocator and a feature fusion unit. The working steps of the spectrum migration gating unit are as follows:

[0016] The modal difference measurement module calculates the cosine similarity matrix between the multi-scale texture feature map and the thermodynamic feature map;

[0017] In the dynamic weight allocator, dynamic weights corresponding to the multi-scale texture feature map and the thermodynamic feature map are calculated and allocated based on the cosine similarity matrix;

[0018] In the feature fusion unit, based on the multi-scale texture feature map and thermodynamic feature map and their respective dynamic weights, weighted feature splicing and channel dimensionality reduction operations are performed to generate fused features.

[0019] As a preferred embodiment, before allocating the dynamic weights in the dynamic weights corresponding to the multi-scale texture feature map and the thermodynamic feature map respectively calculated based on the cosine similarity matrix, the global dependency between the multi-scale texture features and the thermodynamic features is modeled through a multi-head attention mechanism, and then the long-range correlation between the spectral features is established.

[0020] As a preferred embodiment, the meteorological dynamic constraint module includes a radiation transfer regularization term and a continuity regularization term: the radiation transfer regularization term constrains the segmentation result of the ground-based cloud image to be the same as the cloud image segmentation result predicted by the atmospheric radiation transfer model, and the continuity regularization term constrains the spatiotemporal continuity of the ground-based cloud image segmentation result.

[0021] As a preferred embodiment, the deep learning-based semantic segmentation network model adopts a lightweight U-Net variant architecture, including an encoder, a decoder and a skip connection module.

[0022] On the other hand, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a ground-based cloud image segmentation method as described in any embodiment of the present invention is implemented.

[0023] On the other hand, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a ground-based cloud image segmentation method according to any embodiment of the present invention.

[0024] The present invention has the following beneficial effects:

[0025] 1. This invention achieves a deep fusion of multi-scale texture features and thermodynamic features by synchronously collecting visible light and infrared dual-spectral data, combining dynamic dilated convolution and polar coordinate attention mechanism. It solves the problem that a single spectrum is susceptible to light interference and cross-modal feature coupling errors.

[0026] 2. This paper introduces radiative transfer and continuity regularization terms, embedding the atmospheric radiative transfer model and wind speed field dynamics into the segmentation network. The segmentation results are consistent with the physical model, enhancing the credibility of meteorological business scenarios.

[0027] 3. The spectral migration gating unit in the present invention dynamically assigns weights through the cosine similarity matrix, integrates multi-scale texture and thermodynamic features, and solves the problem of cross-modal feature alignment.

[0028] 4. The present invention solves the problem that traditional cosine similarity only captures local correlations through a multi-head self-attention mechanism, thereby improving the ability to characterize complex cloud structures (such as vortices and convergence lines). BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a schematic diagram of the method flow of embodiment 1 of the present invention;

[0030] Figure 2 Schematic diagram of the deformable two-branch convolutional network structure;

[0031] Figure 3 This is a structural diagram of the CSPDarknet53 structure. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0033] It should be understood that the step numbers used herein are only for convenience of description and are not intended to limit the order in which the steps are to be executed.

[0034] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0035] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0036] The term "and / or" refers to and includes any and all possible combinations of one or more of the associated listed items.

[0037] Example 1:

[0038] See also Figure 1 , a ground-based cloud image segmentation method, the specific steps include:

[0039] A dual-spectral imaging system is used to synchronously collect ground-based cloud image data in the visible light band and infrared band;

[0040] In this embodiment, ground-based cloud image data in the visible light band (400-700 nm) and the long-wave infrared band (8-14 μm) are collected synchronously.

[0041] A deformable two-branch convolutional network was constructed, consisting of a main branch embedded with a dynamic dilated convolutional module based on the CSPDarknet53 architecture, and an auxiliary branch integrating a polar coordinate attention mechanism. The deformable two-branch convolutional network was used to extract multi-scale texture features of ground-based cloud image data in the visible light band and thermodynamic feature maps of ground-based cloud image data in the infrared band.

[0042] In this embodiment, the structure of the deformable two-branch convolutional network is as follows: Figure 2 As shown, the structure diagram of CSPDarknet53 structure is as follows Figure 3 As shown in Figure 2, the standard 3×3 convolution part of each residual block in the CSPDarknet53 structure is replaced by a dynamic hole convolution module;

[0043] The ground-based cloud map data in the visible light band is input into the main branch network based on the CSPDarknet53 structure. First, the shallow features are extracted through the initial convolution layer (3×3 convolution, step size 2), and then the feature map is divided into two parts through cross-stage partial connection. One part is directly passed to the deep network, and the other part is extracted from the residual block replaced by the dynamic void convolution module to extract multi-scale texture features. It is then merged with the shallow features directly passed to the deep network to output the final multi-scale texture feature map.

[0044] The ground-based cloud map data in the infrared band is input into the auxiliary branch of the integrated polar coordinate attention mechanism, which converts the feature map in the Cartesian coordinate system into the polar coordinate system and decomposes it into: radial features: aggregated along the radius direction (capturing temperature gradient); angular features: aggregated along the angle direction (capturing texture directionality).

[0045] Combine radial and angular attention mechanisms to enhance radial temperature features and angular texture directionality features;

[0046] Construct a spectral migration gating unit to perform feature compensation and feature fusion on the extracted multi-scale texture features and thermodynamic feature maps to generate fusion features;

[0047] A semantic segmentation network model based on deep learning is constructed, and a meteorological dynamic constraint module is introduced. Based on the historical ground-based cloud image data, the fusion features and the historical segmentation of the ground-based cloud image are obtained, and the semantic segmentation network model is pre-trained. The fusion features of the newly collected ground-based cloud image data are input into the trained semantic segmentation network model to perform semantic segmentation on the target ground-based cloud image.

[0048] In this embodiment, the dual-spectral imaging system includes a visible light sensor and an infrared sensor, which are used to capture ground-based cloud image data in the visible and infrared bands, respectively. The visible light sensor is equipped with a fisheye lens, which has a high dynamic range and frame rate; the infrared sensor uses an uncooled focal plane array, which has high temperature measurement accuracy and wavelength coverage. The system also includes a time synchronization module, which uses an FPGA to achieve sub-millisecond timestamp alignment, ensuring precise synchronization of multimodal data.

[0049] As a preferred implementation of this embodiment, the dynamic dilated convolution module includes a dilation rate selection module and a multi-scale convolution kernel group.

[0050] In this embodiment, the dilation rate selection module dynamically calculates the dilation rate of the convolution kernel to determine the scale range of the current feature extraction. The multi-scale convolution kernel group extracts multi-scale features in parallel, covering local details to large-scale structures.

[0051] As a preferred implementation of this embodiment, the dynamic dilated convolution module dynamically adjusts the dilation rate of the convolution kernel through the dilation rate selection module, and dynamically adjusts the feature scale extracted by the convolution kernel.

[0052] In this embodiment, the calculation formula of the convolution kernel void ratio is specifically:

[0053] α=σ(W g ·[F in ,F out ])

[0054] Where W g is a learnable parameter, F in and F out are the input and output features respectively.

[0055] As a preferred implementation of this embodiment, the feature scale extracted by the convolution kernel is selected by building in a set of discrete void rate candidate sets, covering multiple scales from local details to global semantics, and dynamically selecting and activating the built-in void rate based on the type of cloud layer in the collected ground-based cloud image data.

[0056] In this embodiment, the convolution kernel has a set of discrete void ratio candidate sets R = {1, 2, 4, ...}, covering various scales from local details to global semantics. For example:

[0057] r = 1: dense sampling, capturing fine textures (such as cloud edges);

[0058] r = 2: moderate expansion, capturing mesoscale structures (such as cumulonimbus clouds);

[0059] r=4: sparse sampling, extracting large-scale features (such as weather system boundaries).

[0060] The activation expansion rate is dynamically selected by the convolution kernel dilation rate. For example:

[0061] When the input features are complex (e.g., strong convective cloud areas), the gating weights tend to have a high expansion rate r = 4;

[0062] In flat areas (such as clear sky), the emphasis is on a low expansion rate r=1.

[0063] As a preferred implementation of this embodiment, the spectrum migration gating unit includes a modality difference measurement module, a dynamic weight allocator, and a feature fusion unit. The working steps of the spectrum migration gating unit are as follows:

[0064] The modal difference measurement module calculates the cosine similarity matrix between the multi-scale texture feature map and the thermodynamic feature map;

[0065] In the dynamic weight allocator, dynamic weights corresponding to the multi-scale texture feature map and the thermodynamic feature map are calculated and allocated based on the cosine similarity matrix;

[0066] In the feature fusion unit, based on the multi-scale texture feature map and thermodynamic feature map and their respective dynamic weights, weighted feature splicing and channel dimensionality reduction operations are performed to generate fused features.

[0067] In this embodiment, the calculation formula of the cosine similarity matrix is specifically:

[0068] S=softmax(F sim ([F rgb ; F ir ]))

[0069] Where S is the cosine similarity matrix, F rgb is the visible light characteristic map, F ir It is an infrared characteristic map.

[0070] The calculation formula of the cross-modal attention weight is specifically:

[0071] W att =softmax(MLP(S))

[0072] Where W att is the cross-modal attention weight.

[0073] In this embodiment, the feature splicing calculation formula in the step of generating fusion features is specifically:

[0074] F fuse =W att ⊙F tex +(1-W att )⊙F thermo

[0075] Where, F fuse is the fusion feature; F tex is the multi-scale texture feature; F thetmo is the thermodynamic characteristic; W att is the dynamic weight of multi-scale texture features.

[0076] As a preferred implementation of this embodiment, before allocating the dynamic weights in the dynamic weights corresponding to the multi-scale texture feature map and the thermodynamic feature map respectively calculated based on the cosine similarity matrix, the global dependency between the multi-scale texture features and the thermodynamic features is modeled through a multi-head attention mechanism, and then the long-range correlation between the spectral features is established.

[0077] In this embodiment, based on the multi-head attention mechanism, the above weighted attention calculation formula can be expanded to:

[0078]

[0079] Where Q tex and K thermo are the query / key matrix for texture and thermodynamic features respectively; d k is the feature dimension.

[0080] As a preferred implementation scheme of this embodiment, the meteorological dynamics constraint module includes a radiation transfer regularization term and a continuity regularization term: the radiation transfer regularization term constrains the segmentation result of the ground-based cloud image to be the same as the cloud image segmentation result predicted by the atmospheric radiation transfer model, and the continuity regularization term constrains the spatiotemporal continuity of the ground-based cloud image segmentation result.

[0081] In this embodiment, the calculation formula of the radiation transfer regularization term is specifically:

[0082] F=∫(F sw，down -F sw，up +F Iw，down -F Iw，up )dx

[0083] Where F is the radiation transfer regularization term, F sw，down and F sw，up is the radiation flux when short wave is incident and emitted, F Iw，down and F Iw，up is the radiation flux when long wave is incident and emitted;

[0084] The calculation formula of the continuity regularization term is specifically:

[0085]

[0086] Where, L con is the continuity regularization term, is the gradient of the segmentation boundary, u(p) is the wind speed field, λ is the weight coefficient, and the weight of the balanced dynamic equation and the regularization term; I seg This is the result of ground cloud image segmentation.

[0087] In this embodiment, the continuity regularization term constrains the spatiotemporal continuity of the segmentation results, ensuring that the cloud boundary is consistent with the actual meteorological conditions (such as wind speed field) in terms of time series and spatial distribution, and avoiding isolated or jumpy erroneous segmentation areas.

[0088] The continuity regularization term also includes constrained spatial continuity: combining wind speed field data from weather radar or satellite, the gradient direction of the segmentation boundary is constrained to be consistent with the wind direction.

[0089] In this embodiment, the deep learning-based semantic segmentation network model also includes an edge computing terminal, whose hardware configuration includes a computing unit, a storage unit and a power management unit; the computing unit adopts a computing chip that supports INT8 quantization acceleration; the storage unit adopts LPDDR5 dual-channel cache; and the power management unit adopts a solar power supply system adapter module.

[0090] In this embodiment, the computing unit adopts the Rockchip RK3588S chip, which supports INT8 quantization acceleration; the storage unit adopts LPDDR5 dual-channel cache with a bandwidth of ≥6400Mbps; the power management unit adopts a solar power supply system adapter module with standby power consumption ≤0.5W.

[0091] In this embodiment, the deep learning-based semantic segmentation network model pre-sets a work plan when the embedded device is running. The work plan includes: adopting a cascaded sparse inference strategy to preferentially activate candidate areas with confidence levels greater than a preset threshold; when the network is operating normally, a dynamic resolution switching mechanism is preset, which automatically selects a target input resolution that meets the requirements based on the complexity of the cloud layer; and a bypass acceleration path is added to enable a lightweight filter group in clear sky areas.

[0092] In this embodiment, the cascaded sparse inference strategy prioritizes the activation of candidate regions with a confidence level greater than 0.8. The dynamic resolution switching mechanism automatically selects a 320×320 or 720×720 input resolution based on the cloud layer complexity, and the parameter size of the lightweight filter group in the bypass acceleration path is set to less than 50KB.

[0093] In this embodiment, the deep learning-based semantic segmentation network model adopts a federated learning framework in the pre-training stage, and the federated learning framework performs collaborative training based on the data of each distributed meteorological station; the federated learning framework adopts a knowledge distillation strategy, that is, the Teacher network guides the Student network to perform compression; the federated learning framework supports fine-tuning of model parameters for newly added meteorological data.

[0094] In this embodiment, the Teacher network is used to guide the Student network compression, and the accuracy loss is ≤1.5%;

[0095] As a preferred implementation mode of this embodiment, the deep learning-based semantic segmentation network model also includes a cumulonimbus cloud special processing module, a low-light enhancement module and a device-side self-calibration module in extreme weather scenarios; the cumulonimbus cloud special processing module adopts a lightning precursor detection subnetwork; the low-light enhancement module is constructed based on the improved MSRCR algorithm; the device-side self-calibration module performs radiation calibration compensation based on meteorological radar reflectivity data.

[0096] In this embodiment, the cumulonimbus cloud special processing module adopts a lightning precursor detection sub-network with a detection delay of less than 50ms; the signal-to-noise ratio of the low-light enhancement module is improved by ≥12dB.

[0097] As a preferred implementation of this embodiment, the deep learning-based semantic segmentation network model adopts a lightweight U-Net variant architecture, including an encoder, a decoder and a skip connection module.

[0098] In order to verify the effectiveness and superiority of the method provided in this embodiment, a specific experimental case is provided below:

[0099] 1. Experimental Design and Dataset

[0100] Dataset:

[0101] The public ground-based cloud image dataset CloudSat-Synthetic (simulating visible light and infrared dual-modal data) and the self-built ExtremeWeather-Cloud dataset (including complex scenes such as cumulonimbus clouds, low illumination, and high reflectivity) are used, totaling 1,200 images with a unified resolution of 512×512 pixels and pixel-level annotation accuracy.

[0102] Scene division:

[0103] Conventional scenes: sunny, cloudy, and stratocumulus (accounting for 60%);

[0104] Extreme scenarios: cumulonimbus clouds (including lightning precursors), blizzards, and sandstorms (accounting for 40%).

[0105] 2. Comparison method

[0106] Baseline method:

[0107] Single spectrum segmentation: using only visible light (ViT-Seg) or infrared (HRNet-IR) models;

[0108] Traditional dual-spectral fusion: U-Net (Baseline-Fuse) based on RGB-IR channel splicing;

[0109] Existing physical constraint model: Dual-Branch Net (Prior-Art) with the introduction of radiation transfer regularization terms.

[0110] Evaluation indicators:

[0111] Segmentation accuracy: IoU (Intersection over Union), Dice coefficient;

[0112] Real-time: single-frame inference time (milliseconds);

[0113] Robustness in extreme scenarios: missed detection rate, false detection rate.

[0114] 3. Parameter configuration and training

[0115] Model structure: A lightweight U-Net variant is used, the encoder is CSPDarknet53+dynamic void convolution (expansion rate {1, 2, 4}), and the decoder integrates a thermodynamic feature enhancement layer (3×3 convolution+BatchNorm).

[0116] Training parameters:

[0117] Optimizer: AdamW, initial learning rate 1e-4, weight decay 1e-4;

[0118] Loss function: cross entropy loss (weight 0.5) + Dice loss (weight 0.5) + radiative transfer regularization term (λ1 = 0.1) + continuity regularization term (λ2 = 0.05);

[0119] Batch size: 16, training period: 100 epochs.

[0120] Federated Learning Setting:

[0121] Participating weather stations: 10 distributed stations, local data accounts for 80%, global iteration number: 5 rounds;

[0122] Knowledge distillation: The Teacher network (teacher model parameters are frozen) guides the Student network (distillation temperature T = 4).

[0123] 4. Experimental Results

[0124] 4.1 Performance comparison in common scenarios

[0125] The performance comparison of various ground-based cloud image segmentation methods in common scenarios is shown in Table 1:

[0126] Table 1 Comparison of cloud image segmentation methods in different regions

[0127]

[0128] Result analysis: Through multimodal fusion and physical constraints, the IoU and Dice coefficients of the present invention are improved by 7.3% and 4.1% respectively, and the inference time is lower than that of the traditional physical model (Prior-Art).

[0129] 4.2 Extreme Weather Scenario Verification

[0130] Cumulonimbus cloud detection:

[0131] The accuracy of lightning precursor branch detection reached 92.3%, significantly higher than the traditional method (78.6%);

[0132] The low-light enhancement module improves the signal-to-noise ratio (SNR) from 15dB to 27dB.

[0133] Sandstorm interference:

[0134] After improving the MSRCR algorithm to suppress noise, the cloud boundary false detection rate is reduced from 22.1% of Baseline-Fuse to 8.4%.

[0135] 4.3 Edge Computing Deployment

[0136] Hardware configuration:

[0137] Computing chip: RK3588S (INT8 quantization acceleration);

[0138] Memory: LPDDR5 dual-channel (6400Mbps);

[0139] Power consumption: average power consumption ≤ 2.5W (solar power supply can last for 8 hours).

[0140] Performance:

[0141] The bypass filter group is enabled in clear sky areas, and the inference speed is increased to 90ms / frame;

[0142] The dynamic resolution switching mechanism improves the efficiency of complex cloud processing by 35%.

[0143] Example 2:

[0144] A ground-based cloud image segmentation system, comprising:

[0145] The data acquisition module uses a dual-spectral imaging system to synchronously collect ground-based cloud image data in the visible light band and the infrared band;

[0146] The dual-feature extraction module constructs a deformable dual-branch convolutional network, including a main branch embedded with a dynamic dilated convolutional module based on the CSPDarknet53 architecture, and an auxiliary branch integrating a polar coordinate attention mechanism. This deformable dual-branch convolutional network is used to extract multi-scale texture features of ground-based cloud image data in the visible light band and thermodynamic feature maps of ground-based cloud image data in the infrared band.

[0147] The dual-feature fusion module constructs a spectral migration gating unit to perform feature compensation and feature fusion on the extracted multi-scale texture features and thermodynamic feature maps to generate fused features;

[0148] The ground-based cloud image segmentation module constructs a semantic segmentation network model based on deep learning and introduces a meteorological dynamic constraint module. It obtains fusion features based on historical ground-based cloud image data and the historical segmentation of ground-based cloud images, pre-trains the semantic segmentation network model, inputs the fusion features of the newly collected ground-based cloud image data into the trained semantic segmentation network model, and performs semantic segmentation on the target ground-based cloud image.

[0149] As a preferred implementation of this embodiment, the dynamic dilated convolution module dynamically adjusts the feature scale extracted by the convolution kernel by dynamically adjusting the dilation rate of the convolution kernel.

[0150] As a preferred implementation of this embodiment, the spectrum migration gating unit includes a modality difference measurement module, a dynamic weight allocator, and a feature fusion unit. The working steps of the spectrum migration gating unit are as follows:

[0151] The modal difference measurement module calculates the cosine similarity matrix between the multi-scale texture feature map and the thermodynamic feature map;

[0152] In the dynamic weight allocator, dynamic weights corresponding to the multi-scale texture feature map and the thermodynamic feature map are calculated and allocated based on the cosine similarity matrix;

[0153] In the feature fusion unit, based on the multi-scale texture feature map and thermodynamic feature map and their respective dynamic weights, weighted feature splicing and channel dimensionality reduction operations are performed to generate fused features.

[0154] As a preferred implementation of this embodiment, before allocating the dynamic weights in the dynamic weights corresponding to the multi-scale texture feature map and the thermodynamic feature map respectively calculated based on the cosine similarity matrix, the global dependency between the multi-scale texture features and the thermodynamic features is modeled through a multi-head attention mechanism, and then the long-range correlation between the spectral features is established.

[0155] As a preferred implementation scheme of this embodiment, the meteorological dynamics constraint module includes a radiation transfer regularization term and a continuity regularization term: the radiation transfer regularization term constrains the segmentation result of the ground-based cloud image to be the same as the cloud image segmentation result predicted by the atmospheric radiation transfer model, and the continuity regularization term constrains the spatiotemporal continuity of the ground-based cloud image segmentation result.

[0156] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention's description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A ground-based cloud image segmentation method, characterized in that: The specific steps include: A dual-spectral imaging system is used to synchronously collect ground-based cloud image data in the visible light band and infrared band; A deformable two-branch convolutional network was constructed, consisting of a main branch embedded with a dynamic dilated convolutional module based on the CSPDarknet53 architecture, and an auxiliary branch integrating a polar coordinate attention mechanism. The deformable two-branch convolutional network was used to extract multi-scale texture features of ground-based cloud image data in the visible light band and thermodynamic feature maps of ground-based cloud image data in the infrared band. Construct a spectral migration gating unit to perform feature compensation and feature fusion on the extracted multi-scale texture features and thermodynamic feature maps to generate fusion features; A semantic segmentation network model based on deep learning is constructed, and a meteorological dynamic constraint module is introduced. Based on the historical ground-based cloud image data, the fusion features and the historical segmentation of the ground-based cloud image are obtained, and the semantic segmentation network model is pre-trained. The fusion features of the newly collected ground-based cloud image data are input into the trained semantic segmentation network model to perform semantic segmentation on the target ground-based cloud image.

2. A ground-based cloud image segmentation method according to claim 1, characterized in that: The dynamic dilated convolution module includes a dilation rate selection module and a multi-scale convolution kernel group.

3. A ground-based cloud image segmentation method according to claim 2, characterized in that: The dynamic dilated convolution module dynamically adjusts the dilation rate of the convolution kernel through the dilation rate selection module, and dynamically adjusts the feature scale extracted by the convolution kernel.

4. A ground-based cloud image segmentation method according to claim 3, characterized in that: The feature scale extracted by the convolution kernel is selected by building in a set of discrete void rate candidate sets, covering multiple scales from local details to global semantics. Based on the type of cloud layers in the collected ground-based cloud image data, the built-in void rate is dynamically selected and activated.

5. The ground-based cloud image segmentation method according to claim 1, characterized in that: The spectrum migration gating unit includes a modality difference measurement module, a dynamic weight allocator, and a feature fusion unit. The working steps of the spectrum migration gating unit are as follows: The modal difference measurement module calculates the cosine similarity matrix between the multi-scale texture feature map and the thermodynamic feature map; In the dynamic weight allocator, dynamic weights corresponding to the multi-scale texture feature map and the thermodynamic feature map are calculated and allocated based on the cosine similarity matrix; In the feature fusion unit, based on the multi-scale texture feature map and thermodynamic feature map and their respective dynamic weights, weighted feature splicing and channel dimensionality reduction operations are performed to generate fused features.

6. A ground-based cloud image segmentation method according to claim 5, characterized in that: Before allocating the dynamic weights in the dynamic weights corresponding to the multi-scale texture feature map and the thermodynamic feature map based on the cosine similarity matrix, the global dependency between the multi-scale texture features and the thermodynamic features is modeled through a multi-head attention mechanism, and then the long-range correlation between the spectral features is established.

7. The ground-based cloud image segmentation method according to claim 1, characterized in that: The meteorological dynamics constraint module includes a radiation transfer regularization term and a continuity regularization term: the radiation transfer regularization term constrains the segmentation result of the ground-based cloud image to be the same as the cloud image segmentation result predicted by the atmospheric radiation transfer model, and the continuity regularization term constrains the spatiotemporal continuity of the ground-based cloud image segmentation result.

8. The ground-based cloud image segmentation method according to claim 5, characterized in that: The deep learning-based semantic segmentation network model adopts a lightweight U-Net variant architecture, including an encoder, a decoder, and a skip connection module.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the ground-based cloud image segmentation method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a ground-based cloud image segmentation method according to any one of claims 1 to 8 is implemented.

Images