Foundation cloud identification method deployed on unmanned aerial vehicle

Abstract

The invention discloses an unmanned aerial vehicle foundation cloud identification method and system based on spatial semantic alignment, and relates to the technical field of meteorological monitoring. Aiming at the problem that similar cloud semantic feature distribution is inconsistent due to different shooting angles and positions of foundation cloud images, the invention provides a spatial semantic alignment network. The method comprises the following steps: firstly, constructing a standardized cloud picture tensor; capturing initial visual information through a multi-level feature extractor; a multi-scale semantic alignment module is utilized, and similar cloud semantic features at different spatial positions are compulsively mapped to the same feature subspace through an expansion convolution group and a semantic gating mechanism; introducing a semantic manifold calibration unit, and refining a feature boundary by using nonlinear mapping; and finally, establishing long-range dependence of the cloud category through the global topological association network and outputting a result. According to the method, the problem of spatial heterogeneity of the cloud image is remarkably solved, and high-precision foundation cloud recognition is realized under complex and changeable observation conditions.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of computer vision, meteorological detection and artificial intelligence, and specifically to a method and system based on a spatial semantic alignment network architecture for solving the spatial heterogeneity problem in ground-based cloud image recognition tasks from the perspective of UAVs. Background Technology

[0002] Ground-based cloud observation is a crucial component of meteorological monitoring, playing an irreplaceable role in local weather forecasting, solar power generation prediction, and aviation safety. With the widespread adoption of drone technology, using drones equipped with optical cameras for ground-based cloud observation has become a flexible and efficient new method. However, compared to traditional ground-based cloud images from fixed locations, cloud images acquired by drones face more significant challenges.

[0003] First, the attitude changes and positional shifts of drones during flight result in highly unstable spatial distribution of clouds in the captured images. The same type of cloud, such as cirrus or cumulus, often exhibits significantly different feature responses in a traditional convolutional neural network (CNN) depending on whether it appears at the image center or edge, or at different tilt angles. This spatial semantic misalignment makes it difficult for the network to classify the same type of cloud at different locations into the same semantic category through simple feature extraction, severely impacting recognition accuracy.

[0004] Secondly, existing deep learning methods, including various improved CNNs and lightweight networks, mostly focus on the local extraction of texture and shape features, lacking explicit modeling and calibration mechanisms for the spatial relationships of features. When faced with clouds that are varied in shape and randomly located, existing models often fail to achieve cross-regional semantic alignment, meaning they cannot understand the fundamental principle that similar features at different spatial locations should belong to the same category. Furthermore, uneven illumination and background noise often accompany cloud images, such as subtle changes in the blue sky background, can easily interfere with feature extraction, further reducing the robustness of the model.

[0005] To address the above issues, a series of solutions have recently been proposed in this field through multi-dimensional innovation: Patent CN118072071A proposes a ground-based cloud image classification method based on attention convolution. It combines an improved reparameterized deep convolutional neural network with a novel, efficient channel attention module, utilizes a multi-branch topology to enhance feature representation capabilities, and employs structural reparameterization to achieve lightweight inference. This method demonstrates high classification efficiency on publicly available datasets, but its feature extraction still lacks adaptability to complex cloud morphologies and its reliance on pre-trained weights limits its generalization ability. Patent CN117197462A proposes a lightweight ground-based cloud segmentation method based on multi-scale feature fusion and alignment. It employs an improved EfficientNet-S as the backbone network, combining a heterogeneous receptive field splicing ASPP module and a hybrid strip pooling mechanism to enhance multi-scale feature fusion. Furthermore, it optimizes the decoding process through an attention feature alignment module. While this method achieves improvements in segmentation accuracy and lightweight design, it still falls short in handling the problem of blurred thin cloud boundaries and has relatively high model complexity. Patent CN116740442A discloses a ground-based gimbal-based intelligent meteorological monitoring device and method based on image recognition. This method separates the sky region through mask preprocessing and combines convolutional neural networks and nonlocal neural networks for feature extraction and fusion, ultimately using a classifier to identify cloud types. While this method improves the completeness of feature representation, it requires significant computational resources and is difficult to deploy in real-time on edge devices. Patent CN115861985A proposes an automatic ground-based cloud identification method based on an improved DenseNet network. This method modifies the top-level structure of the network through transfer learning of pre-trained weights and employs a phased fine-tuning strategy to optimize the training process. The method achieves an accuracy of 93.21% on the large dataset GCD. The model is lightweight and training is efficient, but its ability to distinguish between similar cloud types, such as stratocumulus and cumulonimbus clouds, is limited, and it relies heavily on labeled data.

[0006] Therefore, there is an urgent need in this field for a recognition method that can proactively achieve spatial semantic alignment through network structure design, thereby improving the accuracy and environmental adaptability of UAV ground-based cloud recognition by eliminating the inconsistency in feature distribution caused by spatial location differences. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the present invention aims to provide a UAV ground-based cloud identification method based on spatial semantic alignment. This method no longer relies solely on the superposition of extracted features, but instead defines a spatial semantic alignment task and utilizes multi-scale semantic gating and exponential manifold calibration mechanisms to force the network to classify similar semantic information from different spatial locations into the same category.

[0008] To achieve the above objectives, the present invention provides a method for UAV ground-based cloud identification based on spatial semantic alignment, characterized by comprising the following steps: Step S1: Acquire the original ground-based cloud image collected by the UAV and construct a normalized input tensor. Perform denoising and geometric correction preprocessing on the image, and perform regularization based on the statistical distribution of a specific dataset to generate a standardized input X. in ; Step S2: Place X in Input the feature mapping network, extract shallow feature maps containing spatial location information through hierarchical encoding, and output the initial semantic tensor H0; Step S3: Construct a multi-scale semantic alignment module. Project H0 onto feature subspaces with different receptive fields, calculate the semantic responses at different spatial scales, and use semantic alignment operators to eliminate spatial position differences, generating an aligned feature flow H. align ; Step S4: Use the semantic manifold calibration unit to calibrate H align Feature reconstruction is performed. This unit decouples spatial and channel information, uses a calibration function to perform a nonlinear transformation on the feature manifold, suppresses background noise interference with semantics, and outputs calibration features H. cal ; Step S5: Capture long-range semantic dependencies through a global topology association network. (The remaining text appears to be incomplete and requires further context.) cal The mapping is performed on a serialized token, topological position encoding is introduced, and a global semantic context is calculated through an adaptive association mechanism to output a global descriptor V. global ; Step S6: V global The input classifier calculates the confidence distribution of ground-based cloud categories using a probability mapping function, and selects the category corresponding to the maximum confidence value as the final recognition result.

[0009] Compared with the prior art, the present invention has the following beneficial effects: 1. Unique spatial semantic alignment capability: This invention pioneered a semantic gating alignment mechanism, which explicitly solves the problem of different phases of the same cloud in cloud images due to different perspectives and positions, making the model's adaptability to changes in cloud morphology significantly better than traditional CNNs. 2. Robust exponential calibration: By using the semantic manifold calibration unit and the nonlinear amplification characteristics of the exp function, the expression of weak cloud texture signals in the feature space is effectively enhanced, while suppressing additive noise caused by uneven illumination. 3. Global Topology Awareness: The introduction of topological location encoding enables the model to not only recognize local textures but also understand the distribution patterns of clouds in the sky, thereby improving the recognition accuracy of complex cloud species. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the overall process of the UAV ground-based cloud recognition method based on spatial semantic alignment provided in the embodiments of the present invention, which shows the complete steps from image acquisition, preprocessing to final classification and recognition; Figure 2 This is a schematic diagram of the overall structure of the end-to-end deep learning network architecture constructed according to an embodiment of the present invention, which shows in detail the data flow of core functional modules such as feature extraction, semantic alignment, manifold calibration and topological association. Figure 3This is a detailed structural diagram of the multi-scale semantic alignment module in an embodiment of the present invention, which focuses on demonstrating the connection method of capturing multiple receptive field features using parallel dilated convolution and eliminating spatial differences through semantic gating mechanism; Figure 4 This is a schematic diagram of the semantic manifold calibration unit in this embodiment of the invention, demonstrating a dual-path nonlinear calibration mechanism that achieves multiplicative gain by decoupling spatial domain and channel domain features and utilizing an exponential function. Detailed Implementation

[0011] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0012] This invention provides a UAV-based ground-based cloud recognition method based on spatial semantic alignment. This method addresses the recognition challenges of cloud images from a UAV perspective due to varying shooting angles and uneven spatial distribution of clouds. It constructs an end-to-end deep learning framework encompassing preprocessing, feature extraction, semantic alignment, manifold calibration, and topological association. Figure 1 As shown, the method in this embodiment includes the following specific steps: Step S101: Acquire the original ground-based cloud image collected by the UAV and construct a normalized input tensor. This step is the data entry point and aims to eliminate the influence of different lighting conditions and sensor differences. When the UAV collects images, the original image I is affected by the solar altitude angle and flight attitude. raw The pixel value distribution often varies significantly, and directly inputting it into the network can lead to gradient oscillations. First, basic preprocessing is performed on the acquired RGB three-channel images. Gaussian filtering is used to remove high-frequency thermal noise generated by the sensor, and perspective transformation is used to correct geometric distortion caused by wide-angle lenses. Then, to adapt the image data distribution to the dynamic range of the neural network, this embodiment abandons the traditional simple 0-1 normalization and instead adopts a regularization strategy based on the statistical distribution of a specific dataset. Specifically, the system pre-traverses the entire ground-based cloud dataset to calculate the statistical distribution characteristics of all images in the red, green, and blue channels. Standardized input X in The calculation process for each pixel (c,i,j) strictly follows the following physical model: The above formula describes a centering and scaling process. Where α c This is the pre-calculated global statistical mean of the c-th channel. This term is used to eliminate background brightness baseline shifts caused by shooting at different time periods, pulling the data distribution center back to near zero. β c Let be the global statistical standard deviation of the c-th channel. This term is used to standardize the dispersion of the data, ensuring that contour plots of different contrasts have similar signal amplitudes. A minimal constant is introduced into the formula. The value is set to 1e-5, which physically prevents division by zero errors caused by excessively small standard deviations, ensuring the stability of numerical calculations. γ is a learnable scaling factor. The network automatically adjusts this parameter through backpropagation, accurately mapping the dynamic range of the standardized data to the linearly sensitive region of the subsequent activation function, thereby maximizing the preservation of cloud texture details and accelerating the network's convergence speed. Step S102: Input the feature mapping network, extract shallow feature maps containing spatial location information through hierarchical encoding, and output the initial semantic tensor H0. For example... Figure 2 As shown, this step utilizes the powerful local feature extraction capabilities of convolutional neural networks to capture the texture and edge information of the cloud map. In this embodiment, the front-end portion of the DenseNet-121 network with fully connected layers removed is selected as the backbone network. Input tensor X in The network is passed layer by layer, performing convolution, pooling, and non-linear activation operations. As the network depth increases, the resolution of the feature map gradually decreases, the receptive field gradually expands, and the number of channels gradually increases, thereby transforming the original pixel signals into high-dimensional semantic features. To preserve the relative position information of clouds in the sky, this embodiment specifically extracts the output of the backbone network Stage-3 as the initial semantic tensor H0. The feature map at this level is a three-dimensional tensor that, while preserving sufficient spatial resolution, has already encoded the visual texture of the clouds, including the wispy texture of cumulus clouds or the smooth texture of stratus clouds. At this point, the vector at each spatial location of the feature map corresponds to the semantic description of a specific region in the original image, providing the necessary feature anchors for subsequent spatial alignment. Step S103: Construct a multi-scale semantic alignment module. Project features onto feature subspaces with different receptive fields, calculate semantic responses at different spatial scales, and use semantic alignment operators to eliminate spatial location differences, generating an aligned feature flow H. align .like Figure 3 As shown, this is one of the core steps of the present invention, aiming to solve the problem of different phases of the same cloud. Due to the different distances from the drone's perspective, the same cloud may exhibit huge scale differences in the image. A simple fixed convolutional kernel cannot simultaneously adapt to the feature extraction of large cloud clusters nearby and small cloud fragments in the distance. This step simulates different viewing distances by using parallel multi-path convolutional branches. In specific implementation, the module contains three parallel branches, with the dilation rate r of the dilated convolutional kernel set to 1, 2, and 5, respectively. The branch with dilation rate r of 1 focuses on local small textures, while the branch with dilation rate r of 5 focuses on large-scale cloud structures. The initial tensor H0 is processed by these K parallel convolutional branches to generate feature responses at different scales. To intelligently fuse these features and eliminate spatial misalignment, a semantic gating mechanism is introduced. The output H of the multi-scale semantic alignment module... align Precisely described by the following formula: This formula expresses a feature fusion and filtering process. The symbol ∪ performs a cascading operation on feature channels, physically concatenating features captured by different receptive fields along the channel dimension to form an ultrawide feature volume containing rich scale information. Φ channel Channel compression is performed, and the concatenated features are fused using a 1×1 convolution to remove redundant information. The latter part of the formula illustrates the specific implementation of the "semantic alignment operator" described in the claims: where Ψ gate The (·) is a semantic gating function that generates a weight map with the same spatial dimension as H0 through a lightweight attention network. This weight map spatially assigns high weights to the cloud core region and low weights to the blue sky background. Finally, the result is calculated using a Hadamard product. The core operation of the semantic alignment operator is element-wise multiplication. The generated spatial weight map is used to filter the fused multi-scale features, forcing the network to focus on semantic core regions that remain stable at different scales, thereby generating a feature flow H that is invariant to changes in spatial location. align ; Step S104: Reconstruct the features using a semantic manifold calibration unit. This unit decouples spatial and channel information, performs a nonlinear transformation on the feature manifold using a calibration function, suppresses background noise interference with semantics, and outputs calibrated features H. cal .like Figure 4 As shown, while aligned features resolve the scale issue, they may still contain illumination noise or background redundancy. This step widens the distance between cloud-like features and background features by nonlinearly distorting the feature manifold. This embodiment employs a dual-path calibration mechanism, decoupling feature processing into parallel processing in two orthogonal directions: the spatial domain and the channel domain. Output feature H cal The calculation follows the following nonlinear dynamic formula: In the formula, F spatial The branch utilizes large-kernel convolution to extract the texture geometry features of the cloud, constructing the basic manifold of the features. The key lies in the exponential term in the formula; the exponential mapping described in the claims is specifically implemented here as an exponential function f(x) = e^(-e ... x F channel The branch first compresses spatial information into a category semantic vector using global average pooling, and then extracts channel importance scores using a multilayer perceptron. This score is divided by the temperature coefficient τ and then fed into an exponential function exp. This utilizes the multiplicative gain property of the exponential function: when a salient feature of a certain cloud class is identified, the exponential function outputs a gain coefficient much greater than 1, which is proportional to F. spatialThe extracted corresponding texture features are significantly amplified; conversely, for background noise, the exponential function outputs a coefficient close to 0, thereby achieving signal suppression. The temperature coefficient τ is set to 0.1. A smaller temperature coefficient makes the response curve of the exponential function steeper, thereby achieving binarization and sharpening of the features, and greatly enhancing the model's ability to distinguish weak cloud textures. Step S105: Capture long-range semantic dependencies through a global topological association network. Features are mapped to serialized tokens, topological location encoding is introduced, and a global semantic context is calculated through an adaptive association mechanism, outputting a global descriptor V. global Cloud identification relies not only on local texture but also on the distribution pattern of cloud clusters across the entire sky; that is, a dense distribution versus a scattered distribution indicates different cloud genera. To capture this long-range dependency, this step employs a Transformer architecture. First, the calibrated feature map H... cal The system flattens and segments the data into a series of serialized tokens in the spatial dimension. To enable the network to explicitly perceive the relative spatial relationships between cloud clusters, the system introduces a non-learnable topological location encoding matrix P. topo This matrix encodes the Euclidean distance information between different grid locations in the image. Next, a query vector Q, a key vector K, and a value vector V are generated using linear projection, and the topological association matrix A is calculated. topo : Q = H cal W Q K = H cal W K V=H cal W V QK in the formula T Content similarity between different image regions was calculated, and P topo The addition of [a specific feature] imposes spatial constraints on this similarity. This means that the attention weight between two regions will only increase significantly when they are similar in content and conform to a specific cloud genus distribution pattern in spatial topology. The final global descriptor V global We obtain the following through weighted aggregation: V global =Agg(A topo ·V) This aggregation operation performs a weighted summation of the value vectors of all regions based on the calculated topological correlation matrix. At this point, V... global It is no longer a simple stacking of local features, but a global descriptor that highly condenses the semantic information of the entire image, containing all clues from local texture details to macroscopic distribution patterns; Step S106: Input the global descriptor into the classification decision unit, calculate the confidence distribution of the ground-based cloud category through the probability mapping function, and select the category corresponding to the maximum confidence value as the final recognition result. Finally, the high-dimensional global descriptor V... global The data is input into a classification head consisting of fully connected layers. The output dimension of the classification head is set to the total number of categories in the target cloud classification system; in this embodiment, it is 10 categories. The output vector is processed by a Softmax probability mapping function, transforming the values ​​into a probability distribution with a sum of 1. The algorithm iterates through this probability distribution, searching for the index with the largest value. This index directly corresponds to a specific item in a predefined cloud category table, thus determining whether the cloud in the UAV image belongs to a specific category such as cirrus, stratocumulus, or cumulonimbus, thereby completing the identification task.

[0013] To verify the effectiveness of the method of this invention, this embodiment conducted a systematic experimental evaluation on the publicly available ground-based cloud datasets CCSN and SWIMCAT. The CCSN dataset is known for its complex cloud morphology, containing 11 cloud types; while the SWIMCAT dataset covers 5 typical sky conditions. During the data preprocessing stage, all input images were uniformly adjusted to 224×224 pixels to balance computational efficiency and feature preservation. To enhance the robustness of the model, an online data augmentation strategy was adopted during training, with the following specific parameter settings: random horizontal flip probability set to 0.5; random rotation angle range [-15°, +15°]; and brightness, contrast, and saturation adjustment factors for color jitter all set to 0.2. During the network training stage, the optimizer AdamW was selected, with parameters set to β1 = 0.9, β2 = 0.999, and a weight decay coefficient of 1×10⁻⁶. -4 To prevent overfitting, the batch size is set to 32. The learning rate scheduling employs a warm-up + cosine annealing strategy: linear warm-up is performed in the first 5 epochs, with the initial learning rate linearly increasing from 0 to 1×10⁻⁶. -3 The subsequent 95 epochs decay to 1×10⁻⁶ using a cosine function. -6 The total number of iterations is set to 100, and early stopping is performed when the validation set loss no longer decreases.

[0014] As shown in Table 1, the method proposed in this invention achieves excellent performance across all sub-metrics of the CCSN dataset. Specifically, for contrail clouds with relatively fixed morphologies, the recognition accuracy reaches 100%; for easily confused altocumulus and convolutional clouds, the F1-Scores reach 0.9782 and 0.9809 respectively, demonstrating extremely high recognition accuracy. Even when facing complex textured and blurred-boundary nimbostratus and stratus clouds, this method still maintains an accuracy of over 96%. This fully demonstrates that the multi-scale semantic alignment module in this invention can effectively capture the fine texture features of different cloud genera, significantly improving the robustness of classification; Table 1. Indicators by Category Table 2 Comparison of Single Models Table 3 Performance of different datasets

[0015] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A method for UAV ground-based cloud identification based on spatial semantic alignment, characterized in that, Includes the following steps: Step S1: Acquire the original ground-based cloud image collected by the UAV and construct a normalized input tensor. Perform denoising and geometric correction preprocessing on the image, and regularize it based on the statistical distribution of a specific dataset to generate a standardized input X. in ; Step S2: Place X in Input the feature mapping network, extract shallow feature maps containing spatial location information through hierarchical encoding, and output the initial semantic tensor H0; Step S3: Construct a multi-scale semantic alignment module. Project H0 onto feature subspaces with different receptive fields, calculate the semantic responses at different spatial scales, and use semantic alignment operators to eliminate spatial position differences, generating an aligned feature flow H. align ; Step S4: Use the semantic manifold calibration unit to calibrate H align Feature reconstruction is performed. This unit decouples spatial and channel information, uses a calibration function to perform a nonlinear transformation on the feature manifold, suppresses background noise interference with semantics, and outputs calibration features H. cal ; Step S5: Capture long-range semantic dependencies through a global topology association network. (The remaining text appears to be incomplete and requires further context.) cal The mapping is performed on a serialized token, topological position encoding is introduced, and a global semantic context is calculated through an adaptive association mechanism to output a global descriptor V. global ; Step S6: V global The input classifier calculates the confidence distribution of ground-based cloud categories using a probability mapping function, and selects the category corresponding to the maximum confidence value as the final recognition result.

2. The method according to claim 1, characterized in that, In step S1, the standardized input X in The calculation formula is redefined as: Among them, I raw Represents the original image data, where (c, i, j) represent the channel, row, and column indices, respectively; α c and β c represents the first and second moments of the statistical mean and variance of the c-th channel, respectively; ò is a minimal constant to prevent the denominator from being zero; γ is a preset scaling factor used to adjust the dynamic range of the data.

3. The method according to claim 1, characterized in that, In step S3, the output H of the multi-scale semantic alignment module align Determined by the following formula: Where * represents convolution operation, W k and b k These are the weights and biases of the convolution kernel at the k-th scale, respectively. This convolution kernel uses an expansion rate of r. k The dilated convolution is used to capture semantics across different spatial spans; ∪ represents the cascade operation of feature channels; σ is a non-linear activation function; Ψ gate (·) is a semantic gating function used to generate spatial alignment weights; It represents the Hadamardi (or Hadama) stack.

4. The method according to claim 1, characterized in that, In step S4, the semantic manifold calibration unit employs a dual-path calibration mechanism, and its output H cal The calculation formula is as follows: Among them, F spatial (·) denotes the spatial domain transformation function, used to extract texture geometric features; F channel (·) represents the channel domain transformation function, used to extract categorical semantic features; τ is the temperature coefficient, used to adjust the smoothness of the calibration; exp(·) is the exponential mapping, used to transform additive features into multiplicative gains, achieving nonlinear calibration of the feature manifold.

5. The method according to claim 1, characterized in that, In step S5, the global descriptor V of the global topology association network... global The calculation is as follows: Q=H cal W Q ,K=H cal W K ,V=H cal W V V global =Agg(A topo ·V) Among them, W Q W K W V P is a learnable projection matrix; topo This is a topological location encoding matrix used to characterize the relative spatial relationships of clouds in the sky; d k is the scaling factor; Agg(·) is the adaptive aggregation operation.

6. A UAV ground-based cloud identification system based on spatial semantic alignment, comprising a memory and a processor, characterized in that, When the processor executes a computer program in memory, it implements the method as described in any one of claims 1 to 5.

Images