A cloud snow identification method and device based on multi-level attention interaction and a storage medium

Abstract

The application discloses a cloud and snow identification method and device based on multi-level attention interaction and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring an image to be identified; inputting the image to be identified into a pre-trained multi-level attention interaction network; wherein the multi-level attention interaction network comprises a down-sampling feature extraction module, a detail feature extraction module, a deep multi-head attention information enhancement module and a feature interaction fusion up-sampling module; obtaining a cloud and snow mask image according to the output of the multi-level attention interaction network, and completing the identification of the cloud and snow. The cloud and snow identification under the interference of various ground backgrounds can be realized, the detection capability of thin clouds and snow ground edges can be enhanced, the influence of similar spectral characteristics of the cloud and snow on the detection can be reduced, the cloud and snow regions can be more accurately distinguished, the generalization capability is strong, and good effects are also achieved in the segmentation of other ground targets.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a cloud and snow identification method based on multi-level attention interaction, a device and a storage medium. BACKGROUND

[0002] Remote sensing satellites observe the earth for a long time, capture and characterize the surface information. About 66.7% of the earth's surface is covered by clouds every year, and 40-50% of the northern hemisphere is covered by snow in winter. Clouds and snow exist in many satellite images, which can blur the surface objects and reduce the utilization rate of observation data. However, for the research on climate-human-environment interaction, it is of great significance for climate change research, hydrological resource investigation and snow disaster warning. Therefore, an efficient and accurate cloud and snow detection algorithm is needed. Moreover, since snow and clouds have similar radiation temperatures and similar characteristics in the optical band, it is difficult to distinguish them in the detection task. The cloud and snow identification task still faces great challenges. Although many methods have realized the task of cloud and snow identification, there are still deficiencies in the detection results. The robustness and generalization ability of traditional methods are poor, and they cannot automatically extract high-order semantic information of cloud and snow regions in images and ignore some valuable features. The existing deep learning methods cannot detect small regions covered by thin clouds and fine snow in some scenes, and false detection may occur at the overlapping and boundary of clouds and snow. Therefore, it is necessary to strengthen the detection of small-scale objects and the prediction of boundary details. SUMMARY

[0003] In view of the deficiencies of the prior art, the purpose of the present application is to provide a cloud and snow identification method based on multi-level attention interaction, a device and a storage medium, which solves the technical problem that the classification of clouds and snow is not good enough in the prior art.

[0004] The purpose of the present application can be achieved by the following technical solutions:

[0005] In a first aspect, the present application provides a cloud and snow identification method based on multi-level attention interaction, comprising:

[0006] obtaining an image to be identified;

[0007] inputting the image to be identified into a pre-trained multi-level attention interaction network; wherein the multi-level attention interaction network comprises a down-sampling feature extraction module, a detail feature extraction module, a deep multi-head attention information enhancement module and a feature interaction and fusion up-sampling module;

[0008] obtaining a mask image of clouds and snow according to the output of the multi-level attention interaction network, and completing the identification of clouds and snow.

[0009] In some embodiments, the multi-level attention interaction network is trained based on the following method:

[0010] obtain a cloud and snow dataset required for training and perform data augmentation on the cloud and snow dataset, wherein the cloud and snow dataset comprises images of backgrounds, clouds and snow, and corresponding labels of the images;

[0011] train the pre-constructed multi-level attention interaction network using the cloud and snow dataset after data augmentation, to obtain a trained multi-level attention interaction network.

[0012] In some embodiments, the data augmentation on the images in the cloud and snow dataset comprises random rotation, horizontal flipping or vertical flipping operations on the images in the dataset.

[0013] In some embodiments, the processing procedure of the multi-level attention interaction network comprises:

[0014] The down-sampling feature extraction module is configured to perform down-sampling feature extraction on the input images, to obtain shallow features and deep features at different levels, and output the shallow features and the deep features to the detail feature extraction module, and output local features to the deep multi-head attention information enhancement module;

[0015] The detail feature extraction module is configured to extract same information and difference information of the shallow features and the deep features between different levels of the down-sampling feature extraction module, to obtain detail features.

[0016] The deep multi-head attention information enhancement module combines the local features extracted by the down-sampling feature extraction module and the global features extracted by the multi-head self-attention mechanism unit through the bridging unit, to obtain global enhanced features.

[0017] The feature interaction and fusion up-sampling module is configured to perform feature interaction and fusion up-sampling processing according to the detail features and the global enhanced features, to obtain a mask image of clouds and snow.

[0018] In some embodiments, the detail feature extraction module is configured to extract same information and difference information of the shallow features and the deep features between different levels of the down-sampling feature extraction module, to obtain detail features, comprising:

[0019] perform convolution operation and up-sampling operation on the deep features F y ∈R C / 2×H / 2×W / 2 to obtain up-sampled deep features

[0020] subtract the shallow features F x ∈R C×H×W from the up-sampled deep features to obtain difference detail information F d ∈R C×H×W , and set a learnable weight coefficient λ, to obtain a difference information branch F d ;

[0021] The shallow feature F x is added to the deep feature F up-sampled to obtain the same detail information F s ∈R C×H×W , and the information is refined by convolution to obtain the same detail branch F s ;

[0022] The same detail branch and the difference information branch are superimposed to obtain the detail feature F de ;

[0023] The calculation process of the detail feature extraction module is as follows:

[0024]

[0025] F de = Conv 1×1 (F s )+λ×F d

[0026] Where Conv 1×1 (·) represents a 1x1 convolution operation; up(·) represents an up-sampling operation to unify the feature size; λ represents a learnable weight coefficient; F de is the final output detail feature after information superposition.

[0027] In some embodiments, the deep multi-head attention information enhancement module includes a bridging conversion module and a multi-head self-attention mechanism unit; the deep multi-head attention information enhancement module combines the local features extracted by the down-sampling feature extraction module and the global features extracted by the multi-head self-attention mechanism unit through the bridging unit to obtain global enhanced features, including:

[0028] The local features output by the down-sampling feature extraction module are taken as input, and the down part of the bridging unit is used to map the local features to the patch subspace in the multi-head attention, and a 1x1 convolution is used to compress the input channel number to the patch subspace channel number used in the multi-head attention;

[0029] The shape of the local features is converted from BxC t ×H×W to B×HWxC t , which is input into the multi-head attention mechanism unit;

[0030] There are multiple independently randomly initialized learning weight matrices in the multi-head self-attention mechanism unit, which map the input vector to different subspaces, enrich the feature expression of the information, and make the model focus on the key areas in multiple global areas;

[0031] The feature information extracted by the multi-head self-attention mechanism unit is passed through the UP part of the bridge unit to restore the feature map to multi-head attention features, and a 1x1 convolution is used to map the channel dimension to a channel dimension C / 2 to obtain global features;

[0032] The local features and the global features are combined, a 3x3 convolution is used to realize local feature and global feature encoding fusion to obtain global enhanced features;

[0033] The calculation process of the deep multi-head attention information enhancement module is as follows:

[0034] F down =reshape1(Conv 1×1 (F n ))

[0035] F up =Conv 1×1 (reshape2(M t (F down ))

[0036] F dm =Conv 3×3 (Cat(F n +F up ))

[0037] Wherein F n ∈R C×H×W represents the input local feature, represents the mapping to the multi-head attention sub-space, M t (·) represents a multi-head attention operation, reshape1(·) represents a local feature transformation into a sub-space operation, reshape2(·) represents a feature map restoration into a multi-head attention feature, F up ∈R C / 2×H×W represents the restored multi-head attention feature, Cat(·) represents a concatenation in the channel dimension, Conv 3×3 (·) represents a 3x3 convolution operation, F dm ∈R C×H×W represents the global enhanced feature output by the deep multi-head attention information enhancement module.

[0038] In some embodiments, the feature interaction fusion up-sampling module includes a multi-scale and strip boundary convolution part, a spatial channel attention part, and a spatial feature interaction fusion part; the feature interaction fusion up-sampling module is used to perform feature interaction fusion up-sampling processing on the detail features and the global enhanced features to obtain a cloud and snow mask image, including:

[0039] The two inputs of the first feature interaction fusion up-sampling module are the global enhanced feature and the detail feature output by the corresponding detail feature extraction module respectively; the two inputs of the last three feature interaction fusion up-sampling modules are the fusion feature output by the last feature interaction fusion up-sampling module and the detail feature output by the corresponding detail feature extraction module respectively;

[0040] The detail feature passes through the multi-scale and strip boundary convolution part, and different numbers of 3*3 convolution structures are combined in the left branch to enable the module to aggregate 3*3, 5*5 and 7*7 receptive fields, and further filter information while extracting multi-scale features of the detail feature; in the right branch, the 9*9 convolution is designed in a parallel cascade form composed of 9*1 and 1*9, and then a 1*1 convolution is used to compress the feature channel number to obtain the cloud and snow edge feature;

[0041] The calculation process of the detail feature passing through the multi-scale and strip boundary convolution part is as follows:

[0042] F in =Conv 1×1 (F de )

[0043] L1=Conv 3×3 (F in ),L2=Conv 3×3 (L1),L3=Conv 3×3 (L2)

[0044] L=Cat(L1,L2,L3)

[0045] R1=Conv 9×1 (Conv 1×9 (F in )),R2=Conv 1×9 (Conv 9×1 (F in ))

[0046] R=Cat(R1,R2)

[0047] F ms =Conv 1×1 (Cat(L,R))

[0048] Wherein F de ∈R c×h×w represents the detail feature, F in ∈R c / 2×h×w represents the preprocessed detail feature, L i represents the feature of the multi-scale convolution branch, R i represents the feature of the strip convolution branch, Cat(·) represents concatenation in the channel dimension, Conv K×K(·) represents a convolution operation with a K×K kernel, F ms ∈R c×h×w L represents the edge features of clouds and snow; R represents the total features of multi-scale convolution;

[0049] The global augmentation features are processed through the spatial channel self-attention component, which is used to connect cloud and snow with surrounding ground features, enhance the spatial channel attention information of the global augmentation features, and capture the dependencies between channels.

[0050] Global Enhancement Feature F dm ∈R C×H×W The input spatial attention mechanism generates the first feature map through three 1x1 convolutional blocks. Second feature map and the third feature map F3∈R C×H×W ,in Equals C / 8; then, after F1 reconstruction, the first reconstructed mapping feature is obtained. The second reconstructed mapping feature is obtained after F2 reconstruction. F3 reconstructed as The third reconstruction mapping feature; will and Multiplication uses softmax to generate a spatial attention graph S∈R (H×W)×(H×W) Then After multiplying with the spatial attention map S, the original scale is restored to generate the spatial attention feature F. sa ∈R C×H×W ;

[0051] Global Enhancement Feature F dm ∈R C×H×W The input channel attention mechanism compresses the input through global average pooling (AvgPool) and global max pooling (MaxPool) to obtain the average pooled feature F. avg and max pooling feature F max , for F avg and F max Aggregation is performed, and additive operations are used to obtain the additive feature map F. add ∈R C×1×1 The splicing operation yields the spliced ​​feature map F. cat ∈R 2C×1×1 Then add the feature map F add ∈R C×1×1 , splicing feature map F cat ∈R 2C×1×1 The inputs are fed into a two-layer fully connected neural network. The first fully connected layer has C / 16 neurons, and the second fully connected layer has C neurons with shared weights. The output information is summed and activated by sigmoid to generate channel attention features F. c∈R C×H×W ;

[0052] The global enhancement feature is multiplied with the spatial attention feature and the channel attention feature respectively, and then an element sum operation is performed to generate an output attention combined feature F sc ∈R C×H×W ; the global enhancement feature and the attention combined feature F sc ∈R C×H×W are multiplied, and the feature channel number is increased through a 1x1 convolution to obtain a self-attention feature F a ;

[0053] The spatial channel self-attention part calculation formula is as follows:

[0054] F1=Conv 1×1 (F dm ), F2=Conv 1×1 (F dm ), F3=Conv 1×1 (F dm )

[0055]

[0056] F avg =AvgPool(F dm ), F max =MaxPool(F dm )

[0057] F add =F avg +F max , F cat =Cat(F avg , F max )

[0058] F c =σ(V2(V0(F add ))+V2(V1(F cat ))

[0059]

[0060] Wherein, re(·) represents a reconstruction operation, F sa represents the original scale generation spatial attention feature, represents matrix multiplication; σ(·) is a softmax operator; V0 represents the first layer full connection layer weight of F add , V1 represents the first layer full connection layer weight of F cat ; V2 is a shared weight representing the second layer full connection layer of F add and F cat , F a ∈RC×H×W This represents the self-attention characteristics of the output of the self-attention component of the spatial channel;

[0061] The cloud and snow edge features and self-attention features are combined through a spatial feature exchange and fusion component, and the cloud and snow edge features F ms Mapped to self-attention features F by downsampling a After size, with self-attention feature F a Add them together to obtain the overlapping feature U;

[0062] Then, global max pooling and global average pooling operations are applied to the overlapping features U respectively, and spatial information is extracted through two layers of convolution to obtain the total spatial weight w;

[0063] Then, perform a softmax operation on the total spatial weights w to obtain the spatial weights w1, w2 ∈ R. 2×h×w Spatial weights w1 and cloud / snow edge features F ms Matrix multiplication yields weighted cloud and snow edge features, spatial weights w2, and self-attention features F. a Matrix multiplication yields weighted self-attention features, which are then upsampled and added to weighted cloud / snow edge features. This is followed by normalized attention processing to obtain the final output fused feature F. u ;

[0064] The calculation formula for the spatial feature exchange and integration part is expressed as follows:

[0065] U = F a +down(F ms )

[0066] w = Conv 1×3 (Conv 3×1 (Cat(AvgPool(U),MaxPool(U)))

[0067] (w1, w2) = σ(w)

[0068]

[0069] In the formula, down(·) represents the downsampling operation, σ(w) represents the total spatial weights after the softmax operation, and w1 and w2 represent F respectively. a and F ms Spatial weights; This represents the weighted self-attention features. This represents the weighted edge features of clouds and snow, up(·) represents a 2x upsampling operation using bilinear interpolation, NAM represents the normalized attention mechanism, and F u This indicates the fusion characteristics output by the module;

[0070] The output of the previous feature interaction fusion upsampling module is taken as the input of the next feature interaction fusion upsampling module, and after being processed by three feature interaction fusion upsampling modules, a mask image of clouds and snow is obtained through bilinear interpolation upsampling.

[0071] In some embodiments, the background includes forest, water area, city, grassland, bare land;

[0072] In some embodiments, the cloud includes cumulus cloud, alto-cumulus cloud, cirrus cloud, cumulus cloud and stratus cloud;

[0073] In some embodiments, the snow includes permanent snow, stable snow and unstable snow.

[0074] In a second aspect, the present application provides a cloud and snow identification device with multi-level attention interaction, comprising a processor and a storage medium.

[0075] The storage medium is used to store instructions.

[0076] The processor is used to operate according to the instructions to perform the steps of the method according to the first aspect.

[0077] In a third aspect, the present application provides a storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the method according to the first aspect.

[0078] In a fourth aspect, the present application provides a device, comprising,

[0079] One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the methods of the first aspect.

[0080] The application has the beneficial effects that: in the use process, the improved residual network is used as the backbone network for feature extraction of input data according to the basic structure of the encoder-decoder, a detail feature extraction module, a deep multi-head attention information enhancement module, a feature interaction fusion up-sampling module are designed, and a multi-level attention interaction network is constructed. Then, the training data including background, cloud and snow are obtained, and data enhancement processing is performed, then the image and the corresponding label are converted into a tensor, and then input into the multi-level attention interaction network for training; finally, the trained multi-level attention interaction network is used for prediction, a color picture is input, and the mask image of cloud and snow is output after feature extraction and encoding and decoding operation; cloud and snow recognition under the interference of various ground surface backgrounds can be realized, the detection ability of thin cloud and snow ground edge is enhanced, the influence of similar spectral characteristics of cloud and snow on detection is reduced, the cloud and snow regions are more accurately distinguished, and the generalization ability is strong, and good effects are also obtained in the segmentation of other ground targets. BRIEF DESCRIPTION OF DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0082] Figure 1 The figure is a multi-level attention interaction network structure for cloud and snow recognition of the application;

[0083] Figure 2 The figure is a detail feature extraction module of the application;

[0084] Figure 3 The figure is a deep multi-head attention information enhancement module of the application;

[0085] Figure 4 The figure is a feature interaction fusion up-sampling module of the application;

[0086] Figure 5 The figure is a training set sample of the embodiment of the application, wherein the upper half is a remote sensing image original picture, and the lower half is a corresponding label picture;

[0087] Figure 6 The figure is a remote sensing image original picture for prediction of the embodiment of the application;

[0088] Figure 7 The figure is a prediction picture of the remote sensing image original picture for prediction of the embodiment of the application. DETAILED DESCRIPTION

[0089] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0090] Embodiment 1

[0091] A cloud and snow recognition method based on multi-level attention interaction, comprising:

[0092] obtaining an image to be recognized;

[0093] inputting the image to be recognized into a pre-trained multi-level attention interaction network; wherein the multi-level attention interaction network comprises a down-sampling feature extraction module, a detail feature extraction module, a deep multi-head attention information enhancement module and a feature interaction fusion up-sampling module;

[0094] obtaining a cloud and snow mask image according to the output of the multi-level attention interaction network, and completing the recognition of the cloud and snow.

[0095] In some embodiments, the multi-level attention interaction network is obtained by training based on the following method:

[0096] obtaining a cloud and snow data set required for training and performing data enhancement processing on the cloud and snow data set, wherein the cloud and snow data set comprises images of backgrounds, clouds and snow, and corresponding labels of the images;

[0097] training the pre-constructed multi-level attention interaction network using the cloud and snow data set after data enhancement processing, to obtain a trained multi-level attention interaction network.

[0098] In some embodiments, the data enhancement processing on the images in the cloud and snow data set comprises random rotation, horizontal flipping or vertical flipping operations on the images in the data set.

[0099] In some embodiments, the processing process of the multi-level attention interaction network comprises:

[0100] The down-sampling feature extraction module is configured to perform down-sampling feature extraction on the input image, to obtain shallow features and deep features of different levels, and output the shallow features and the deep features to the detail feature extraction module, and output local features to the deep multi-head attention information enhancement module;

[0101] The detail feature extraction module is configured to extract the same information and the different information of the shallow features and the deep features between different levels of the down-sampling feature extraction module, to obtain detail features.

[0102] The deep multi-head attention information enhancement module combines the local features extracted by the down-sampling feature extraction module and the global features extracted by the multi-head self-attention mechanism unit through a bridging unit to obtain global enhanced features.

[0103] The feature interaction fusion up-sampling module is configured to perform feature interaction fusion up-sampling processing on the detail features and the global enhanced features to obtain a cloud and snow mask image.

[0104] In some embodiments, as shown in Figure 1 The cloud and snow recognition method with multi-level attention interaction includes the following steps.

[0105] S1, obtaining a training data set:

[0106] A public medium-high spatial resolution cloud and snow data set is used. It should be further explained that the data set includes 27 medium-high resolution images containing clouds and snow, covering various complex backgrounds, including forests, water areas, cities, grasslands, bare land, etc. The cloud types include convective clouds, alto cumulus clouds, cirrus clouds, cumulus clouds and stratus clouds. The snow covered includes permanent snow, stable snow and unstable snow. The diversity of cloud and snow and background makes the data set more representative and general. Due to the limitation of the performance of the graphics card, the original picture is uniformly cut in a sliding cover-free manner to obtain 3×256×256 size images. As shown in Figure 5 Some training images in the cloud and snow data set are shown. The first row in the figure is the original color image, and the second row is the corresponding label.

[0107] S2, constructing a multi-level attention interaction network

[0108] In some embodiments, the structure of the multi-level attention interaction network is as shown in Figure 1 The entire network is based on an encoder-decoder structure, adopts an end-to-end training method, and is mainly composed of a down-sampling feature extraction module, a detail feature extraction module, a deep multi-head attention information enhancement module and a feature interaction fusion up-sampling module. Two auxiliary branches (AUX) during training are set as additional prediction outputs and added to the calculation of the loss function. In the process of identifying cloud and snow in remote sensing images, the residual network is selected as the backbone network to extract feature information, which can greatly improve the detection efficiency of the network. The auxiliary branch is set to speed up the training efficiency and convergence speed of the model and improve the prediction accuracy.

[0109] The detail feature extraction module is configured to extract the same information and difference information of the shallow features and the deep features between different levels of the down-sampling feature extraction module to obtain detail features, reduce the loss of detail information in the feature extraction process, and realize the refinement of the cloud and snow edge detection.

[0110] First, the deep feature F y ∈R C / 2×H / 2×W / 2 is enlarged to the same size as the shallow feature by convolution operation and up-sampling operation to obtain the up-sampled deep feature F Then, the shallow feature F x ∈R C×H×W is subtracted from the up-sampled deep feature F to obtain the difference detail information F d ∈R C×H×W , and a learnable weight coefficient λ is set to allow the network to independently adjust the important difference area, obtaining the difference information branch F d ; at the same time, the shallow feature F x is added to the up-sampled deep feature F to obtain the same detail information F s ∈R C×H×W , and the information is refined by convolution to obtain the same detail branch F s .

[0111] Finally, the same detail branch and the difference information branch are superimposed to obtain the detail feature F de .

[0112] The calculation process of the detail feature extraction module is as follows:

[0113]

[0114] F de =Conv 1×1 (F s )+λ×F d

[0115] where Conv 1×1 (·) represents a 1x1 convolution operation; up(·) represents up-sampling operation to unify the feature size; λ represents a learnable weight coefficient; and F de is the detail feature output after information superposition.

[0116] It should be further pointed out that in the specific implementation process, the deep multi-head attention information enhancement module includes a bridging conversion module and a multi-head self-attention mechanism unit; the deep multi-head attention information enhancement module combines the local features extracted by the down-sampling feature extraction module and the global features extracted by the multi-head self-attention mechanism unit through the bridging unit to obtain the global enhanced features, including:

[0117] The local feature output by the down-sampling feature extraction module is taken as input, the down part of the bridge unit is used to map the local feature to a patch subspace in multi-head attention, and a 1*1 convolution is used to compress the input channel number to the patch subspace channel number used in multi-head attention;

[0118] The shape of the local feature is converted from BxC t xHxW to the shape form BxHWxC t in the patch subspace in multi-head attention, and is input into the multi-head attention mechanism unit;

[0119] There are multiple independently randomly initialized learning weight matrices in the multi-head self-attention mechanism unit, which map the input vector to different subspaces, enrich the feature expression of information, and make the model focus on the key areas in multiple global areas;

[0120] The feature information extracted by the multi-head self-attention mechanism unit is restored to the multi-head attention feature through the UP part of the bridge unit, and the channel dimension is mapped to the channel dimension C / 2 through a 1*1 convolution to obtain the global feature;

[0121] The local feature and the global feature are combined, a 3*3 convolution is used to realize the encoding fusion of the local feature and the global feature, and a global enhanced feature is obtained;

[0122] The calculation process of the deep multi-head attention information enhancement module is as follows:

[0123] F down = reshape1(Conv 1×1 (F n ))

[0124] F up = Conv 1×1 (reshape2(M t (F down ))

[0125] F dm = Conv 3×3 (Cat(F n +F up ))

[0126] Where F n ∈R C×H×W represents the input local feature, F down ∈R Ct×HW represents the mapping to the patch subspace in multi-head attention, M t (·) represents the multi-head attention operation, reshape1(·) represents the operation of converting the local feature to the patch subspace, reshape2(·) represents the operation of restoring the feature map to the multi-head attention feature, and Fup ∈R C / 2×H×W denotes the restored multi-head attention feature, Cat(·) denotes concatenation in the channel dimension, Conv 3×3 (·) denotes a 3x3 convolution operation, F dm ∈R C×H×W denotes the global enhancement feature output by the deep multi-head attention information enhancement module.

[0127] It should be further explained that, in the specific implementation process, further, the feature interaction fusion upsampling module includes a multi-scale and strip boundary convolution part, a spatial channel attention part, and a spatial feature exchange fusion part; for paying attention to the context background information and spatial information in different levels of features, promoting feature fusion and recovery to realize clear high-resolution image formation, and reducing the problem of semantic dilution and information redundancy caused by simply and directly fusing information.

[0128] The two inputs of the first feature interaction fusion upsampling module are the global enhancement feature and the detail feature output by the corresponding detail feature extraction module; the two inputs of the last three feature interaction fusion upsampling modules are the fusion feature output by the last feature interaction fusion upsampling module and the detail feature output by the corresponding detail feature extraction module;

[0129] Firstly, the detail feature passes through the multi-scale and strip boundary convolution part, in the left branch, different numbers of 3x3 convolution structure combinations are used to enable the module to aggregate 3x3, 5x5 and 7x7 receptive fields, and further filter information while extracting multi-scale features of the detail feature; in the right branch, the 9x9 convolution is designed in a parallel cascade form composed of 9x1 and 1x9, and then a 1x1 convolution is used to compress the feature channel number, which reduces the parameter amount while obtaining the cloud snow edge feature through the extraction method of its strip-shaped convolution along the spatial dimension; the calculation process of the detail feature after passing through the multi-scale and strip boundary convolution part is as follows:

[0130] F in =Conv 1×1 (F de )

[0131] L1=Conv 3×3 (F in ), L2=Conv 3×3 (L1), L3=Conv 3×3 (L2)

[0132] L=Cat(L1,L2,L3)

[0133] R1=Conv 9×1 (Conv 1×9 (F in )), R2=Conv1×9 (Conv 9×1 (F in ))

[0134] R=Cat(R1,R2)

[0135] F ms =Conv 1×1 (Cat(L,R))

[0136] where F de ∈R c×h×w denotes the detail feature, F in ∈R c / 2×h×w denotes the pre-processed detail feature, L i denotes the feature of the multi-scale convolution branch, R i denotes the feature of the strip convolution branch, Cat(·) denotes concatenation in the channel dimension, Conv K×K (·) denotes convolution operation with KxK as the convolution kernel, F ms ∈R c×h×w denotes the cloud-snow edge feature; L denotes the total feature of the multi-scale convolution, and R denotes the total feature of the strip convolution.

[0137] Secondly, the global enhancement feature passes through the spatial channel self-attention part to connect the cloud and snow with the surrounding ground object information, enhance the spatial channel attention information of the global enhancement feature, capture the dependence relationship existing between channels, better distinguish the ground object information, and reduce the influence of the pixels in the receptive field center on the output. The global enhancement feature F dm ∈R C×H×W inputs the spatial attention mechanism to generate the first feature mapping F1∈R , the second feature mapping F2∈R and the third feature mapping F3∈R C×H×W through three 1x1 convolution blocks respectively, where is equal to C / 8; then the first reconstruction mapping feature F1 is obtained after reconstruction of F1 , the second reconstruction mapping feature F2 is obtained after reconstruction of F2 , and the third reconstruction mapping feature F3 is obtained after reconstruction of F3 ; F1 is multiplied by F3 to generate the spatial attention graph S∈R (H ×W)×(H×W) using softmax; then F1 is multiplied by the spatial attention graph S to restore the original scale to generate the spatial attention feature F sa ∈R C×H×W ;

[0138] The global enhancement feature F dm ∈R C×H×WThe input channel attention mechanism compresses the input through global average pooling (AvgPool) and global max pooling (MaxPool) to obtain the average pooled feature F. avg and max pooling feature F max , for F avg and F max Aggregation is performed, and additive operations are used to obtain the additive feature map F. add ∈R C×1×1 The splicing operation yields the spliced ​​feature map F. cat ∈R 2C×1×1 Then add the feature map F add ∈R C×1×1 , splicing feature map F cat ∈R 2C×1×1 The inputs are fed into a two-layer fully connected neural network. The first fully connected layer has C / 16 neurons, and the second fully connected layer has C neurons with shared weights. The output information is summed and activated by sigmoid to generate channel attention features F. c ∈R C×H×W ;

[0139] The global enhancement features are multiplied by the spatial attention features and the channel attention features respectively, and then element-wise summation is performed to generate the output attention combination feature F. sc ∈R C×H×W Global enhancement features and attention-based combined features F sc ∈R C×H×W After multiplication, the number of feature channels is increased by 1×1 convolution to obtain the self-attention feature F. a ;

[0140] The formula for calculating the self-attention component of the spatial channel is shown below:

[0141] F1 = Conv 1×1 (F dm ),F2=Conv 1×1 (F dm ),F3=Conv 1×1 (F dm )

[0142]

[0143] F avg =AvgPool(F dm ),F max =MaxPool(F dm )

[0144] F add =F avg +F max ,F cat =Cat(Favg ,F max )

[0145] F c add cat )

[0146]

[0147] where re(·) denotes the reconstruction operation, F sa represents the original scale generation space attention feature, represents the matrix multiplication; σ(·) is the softmax operator; V0represents the first layer full connection layer weight of F add , V1represents the first layer full connection layer weight of F cat , V2is the shared weight of the second layer full connection layer of F add and F cat , F a ∈R C×H×W represents the self-attention feature output by the spatial channel self-attention part;

[0148] Finally, the cloud snow edge feature and the self-attention feature are combined through the spatial feature communication fusion part, the cloud snow edge feature F ms is mapped to the self-attention feature F a by downsampling, and then added to the self-attention feature F a to obtain the overlapping feature U.

[0149] Then the global maximum pooling and global average pooling operations are performed on the overlapping feature U, and the spatial information is extracted through two convolutional layers to obtain the total spatial weight w.

[0150] The softmax operation is performed on the total spatial weight w to obtain the spatial weights w1,w2∈R 2×h×w ; the spatial weight w1is multiplied by the cloud snow edge feature F ms to obtain the weighted cloud snow edge feature, and the spatial weight w2is multiplied by the self-attention feature F a to obtain the weighted self-attention feature. After upsampling, the weighted self-attention feature is added to the weighted cloud snow edge feature, and then processed by the normalized attention mechanism to obtain the final output fusion feature F u of the module.

[0151] The calculation formula of the spatial feature communication fusion part is represented as:

[0152] U=F a +down(F ms )

[0153] ​​w = Conv 1×3 (Conv 3×1 (Cat(AvgPool(U),MaxPool(U)))

[0154] (w1,w2) = σ(w)

[0155]

[0156] where down(·) denotes a down-sampling operation, σ(w) denotes the total spatial weights after the softmax operation, w1, w2 denote the spatial weights of F a and F ms respectively; denotes the weighted self-attention feature, denotes the weighted cloud-snow edge feature, up(·) denotes a 2x up-sampling operation using bilinear interpolation, NAM denotes a normalized attention mechanism, F u denotes the fusion feature output by the module;

[0157] The output of the previous feature interaction fusion up-sampling module is taken as the input of the next feature interaction fusion up-sampling module, and after processing by three feature interaction fusion up-sampling modules, a cloud and snow mask image is obtained after bilinear interpolation up-sampling.

[0158] In view of the problem that cloud and snow have similar features in the visible spectrum and it is difficult to extract the complexity of the features, a deep multi-head attention information enhancement (DMIE) module is proposed at the deepest layer of the backbone network to enhance the representation ability of the network. The function is to use the multi-head attention idea at the end of the down-sampling to enhance the global perception ability of the network in the channel, extract accurate dense features, combine local features and global features, and to a certain extent, weaken or remove interference information, and deeply mine high-order semantic information.

[0159] In the process of segmenting cloud and snow, part of the cloud layer and the snow area is distributed in a scattered state, and the distribution area is also different. In view of these problems, the segmentation result realized by the current algorithm is rough, and the details are lost in the thin cloud and broken snow area. Moreover, when the backbone network extracts features, a large amount of semantic feature information and class information will be generated in the network, but not all information is useful. Therefore, a detail feature extraction (DFEM) module is proposed. The module mainly establishes a jump connection between the encoder and the decoder, propagates the information of low-level features to the deep network, makes up for the dilution of the detail semantic information in the extraction operation of the backbone network, strengthens the weight of important feature information, and at the same time, can restore the lost detail information in the encoder stage, enhance the high-frequency spatial feature information, and realize the refinement of the edge.

[0160] Due to the simple operation of combining high-level features and low-level features, the lost details cannot be effectively recovered, and the false detection of the overlapping area and the missed detection of the tiny area are alleviated. In order to solve this problem, a feature interaction fusion up-sampling (FIFUM) module is proposed in the decoding stage. The module is composed of three parts. For the multi-scale and strip boundary convolution (MSBC) part, the edge detail recovery of cloud and snow in the process of segmenting cloud and snow is generally strengthened; for the spatial channel self-attention (CSA) part, the deep spatial channel information is strengthened, and the cloud and snow are connected with the surrounding ground information, so as to better distinguish the ground information. For the spatial feature interaction fusion (SFIF) part, the information communication between the above two channels is enhanced, the feature information extraction of different levels is enhanced, the information between different scales is aggregated, the elements are connected and fused, and more effective features are provided for the up-sampling process. The operation can reduce the false detection of the cloud and snow overlapping area, strengthen the detection of the tiny area, and improve the prediction accuracy.

[0161] S3, training the network model by using the data set:

[0162] The present application adopts supervised training, first performs data enhancement processing on the images in the data set, randomly rotates the pictures in the data set with a certain probability, performs horizontal flip or vertical flip operation, then converts the original picture and the corresponding label into a tensor and inputs it into the model for training. The batch size of each training is set to 8, the present application adopts the exponential transformation learning rate (poly) strategy, dynamically adjusts the learning rate to gradually decrease during the training times, so as to achieve better training effect. The initial learning rate is set to 0.0005, the decay coefficient is 0.9, the learning rate is updated once per training, and a total of 300 times of training. The Adam algorithm is used as the weight optimizer during the training process.

[0163] S4, using the trained network model to predict and generate the cloud and snow mask image:

[0164] The weights of the model are obtained after the training is completed, and then the prediction stage of the model is entered. In the prediction, the trained multi-level attention interaction network is used for prediction, and an original color picture is input, as shown in Figure 6 The original color picture for prediction is a cloud and snow scene picture, and the mask image of cloud and snow is output after the trained weights extract features and perform encoding and decoding operations, as shown in Figure 7 .

[0165] Embodiment 2

[0166] In a second aspect, the present embodiment provides a cloud and snow identification device with multi-level attention interaction, comprising a processor and a storage medium.

[0167] The storage medium is configured to store instructions.

[0168] The processor is configured to operate according to the instructions to perform the steps of the method according to embodiment 1.

[0169] Embodiment 3

[0170] In a third aspect, the present embodiment provides a storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the method according to embodiment 1.

[0171] Embodiment 4

[0172] In a fourth aspect, the present embodiment provides an apparatus comprising,

[0173] one or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods according to embodiment 1.

[0174] Those skilled in the art understand that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) having computer-usable program code embodied therein.

[0175] The present application is described with reference to the flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The flow or multiple flows and / or blocks Figure 1 The means for performing the functions specified in the flow or multiple flows and / or blocks.

[0176] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams.Figure 1 one or more processes and / or blocks Figure 1 the function(s) specified in the block or blocks.

[0177] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, so that the instructions executed on the computer or other programmable data processing devices provide processes for implementing the flow Figure 1 one or more processes and / or blocks Figure 1 the function(s) specified in the block or blocks.

[0178] In the description of the present specification, the description of the terms "one embodiment", "an example", "a specific example" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0179] The basic principles, main features and advantages of the present application are shown and described above. It should be understood by those skilled in the art that the present application is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only illustrative of the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the present application.

Claims

1. A cloud and snow recognition method based on multi-level attention interaction, characterized in that, include: Acquire the image to be recognized; The image to be identified is input into a pre-trained multi-level attention interaction network; The multi-layer attention interaction network includes: a downsampling feature extraction module, a detail feature extraction module, a deep multi-head attention information enhancement module, and a feature interaction fusion upsampling module; Based on the output of the multi-level attention interaction network, mask images of clouds and snow are obtained, and cloud and snow identification is completed. The multi-level attention interaction network processing includes: the downsampling feature extraction module extracts downsampling features from the input image, obtaining shallow and deep features at different levels, which are then output to the detail feature extraction module, and local features are output to the deep multi-head attention information enhancement module; the detail feature extraction module extracts the similarities and differences between the shallow and deep features at different levels of the downsampling feature extraction module to obtain detail features; the deep multi-head attention information enhancement module combines the local features extracted by the downsampling feature extraction module and the global features extracted by the multi-head self-attention mechanism unit through a bridging unit to obtain global enhanced features; and the feature interaction fusion upsampling module performs feature interaction fusion upsampling processing based on the detail features and global enhanced features to obtain a mask image of clouds and snow. The deep multi-head attention information enhancement module includes a bridging and transformation module and a multi-head self-attention mechanism unit. The deep multi-head attention information enhancement module combines the local features extracted by the downsampling feature extraction module and the global features extracted by the multi-head self-attention mechanism unit through the bridging unit to obtain global enhanced features, including: The local features output by the downsampling feature extraction module are used as input. The down part of the bridging unit is used to map the local features to the subspace of the multi-head attention. A 1×1 convolution is used to compress the number of input channels to the number of subspace channels used in the multi-head attention. The shape of the local feature is changed from B×C. t ×H×W, transformed into the shape form B×H×W×C of the subspace in multi-head attention. t The input is fed into the multi-head attention mechanism unit; The multi-head self-attention mechanism unit has multiple independently and randomly initialized learning weight matrices, which map the input vector to different subspaces, enrich the feature representation of information, and enable the model to focus on multiple key regions in the global context. The feature information extracted by the multi-head self-attention mechanism unit is used to restore the feature map into multi-head attention features through the UP part of the bridging unit, and the channel dimension is mapped to the channel dimension C / 2 through 1×1 convolution to obtain the global features. By combining local and global features, and using 3×3 convolution to fuse local and global feature encodings, a globally enhanced feature is obtained. The calculation process of the deep multi-head attention information enhancement module is as follows: F down =reshape1(Conv 1×1 (F n )) F up =Conv 1×1 (reshape2(M t (F down )) F dm =Conv 3×3 (Cat(F n +F up )) Where F n ∈R C×H×W Represents local features of the input. M represents the patch subspace mapped to multi-head attention. t (·) represents a multi-head attention operation, reshape1(·) represents the operation of transforming local features into patch subspaces, and reshape2(·) represents the operation of restoring the feature map into multi-head attention features. F up ∈R C / 2×H×W This represents the recovered multi-head attention feature, Cat(·) represents concatenation along the channel dimension, and Conv 3×3 (·) represents a 3x3 convolution operation, F dm ∈R C×H×W This represents the global enhancement feature output by the deep multi-head attention information enhancement module.

2. The cloud and snow recognition method with multi-level attention interaction according to claim 1, characterized in that, The multi-layered attention interaction network was trained using the following method: Obtain the cloud and snow dataset required for training and perform data augmentation processing on the cloud and snow dataset, wherein the cloud and snow dataset includes images of background, clouds and snow, and corresponding labels for the images; The pre-constructed multi-level attention interaction network was trained using the data augmented cloud and snow dataset to obtain the trained multi-level attention interaction network.

3. The cloud and snow recognition method with multi-level attention interaction according to claim 2, characterized in that, Data augmentation of images in the cloud and snow dataset includes random rotation, horizontal flipping, or vertical flipping of the images in the dataset.

4. The cloud and snow recognition method with multi-level attention interaction according to claim 1, characterized in that, The detail feature extraction module is used to extract the similarities and differences between shallow and deep features at different levels of the downsampling feature extraction module to obtain detail features, including: For deep features F y ∈R C / 2×H / 2×W / 2 Perform convolution and upsampling operations to obtain the upsampled deep features. shallow features F x ∈R C×H×W Compared with upsampled deep features Subtraction yields the detailed difference information F. d ∈R C×H×W By setting a learnable weight coefficient λ, the differential information branch F was obtained. d ; shallow features F x Compared with upsampled deep features Adding them together yields the same detailed information F s ∈R C×H×W And by refining information through convolution, the same detailed branch F is obtained. s ; By superimposing information from branches with identical details and branches with different information, we obtain the detail feature F. de ; The calculation process of the detailed feature extraction module is as follows: F de =Conv 1×1 (F s )+λ×F d Where Conv 1×1 (·) represents a 1x1 convolution operation; up(·) represents an upsampling operation to standardize feature size; λ represents the learnable weight coefficients; F de It refers to the detailed features of the final output after information is superimposed.

5. The cloud and snow recognition method with multi-level attention interaction according to claim 1, characterized in that, The feature interaction fusion upsampling module includes a multi-scale and strip boundary convolution part, a spatial channel attention part, and a spatial feature interaction fusion part; The feature interaction fusion upsampling module is used to perform feature interaction fusion upsampling processing based on the detailed features and global enhancement features to obtain a mask image of clouds and snow, including: The first feature interaction fusion upsampling module takes two inputs: the global enhanced feature and the detailed feature output from the corresponding detailed feature extraction module. The last three feature interaction fusion upsampling modules take two inputs: the fused feature output from the previous feature interaction fusion upsampling module and the detailed feature output from the corresponding detailed feature extraction module. The detailed features are processed through multi-scale and strip boundary convolutions. In the left branch, different numbers of 3×3 convolutional structures are combined to enable the module to aggregate receptive fields of 3×3, 5×5, and 7×7, which can further filter information while extracting multi-scale features of detailed features. In the right branch, the 9×9 convolution is designed as a parallel concatenation form consisting of 9×1 and 1×9, and then 1×1 convolution is used to compress the number of feature channels to obtain cloud and snow edge features. The detailed features are calculated through multi-scale and stripe-boundary convolution as follows: F in =Conv 1×1 (F de ) L1=Conv 3×3 (F in ),L2=Conv 3×3 (L1),L3=Conv 3×3 (L2) L = Cat(L1, L2, L3) R1=Conv 9×1 (Conv 1×9 (F in )),R2=Conv 1×9 (Conv 9×1 (F in )) R = Cat(R1, R2) F ms =Conv 1×1 (Cat(L,R)) Where F de ∈R c×h×w F represents detailed features. in ∈R c / 2×h×w L represents the detailed features after preprocessing. i R represents the features of multi-scale convolutional branches. i This represents the features of the strip convolutional branch, Cat(·) represents the concatenation in the channel dimension, and Conv K×K (·) represents a convolution operation with a K×K kernel, F ms ∈R c×h×w L represents the edge features of clouds and snow; R represents the total features of multi-scale convolution; The global augmentation features are processed through the spatial channel self-attention component, which is used to connect cloud and snow with surrounding ground features, enhance the spatial channel attention information of the global augmentation features, and capture the dependencies between channels. Global Enhancement Feature F dm ∈R C×H×W The input spatial attention mechanism generates the first feature map through three 1x1 convolutional blocks. Second feature map and the third feature map F3∈R C×H×W ,in Equals C / 8; then, after F1 reconstruction, the first reconstructed mapping feature is obtained. The second reconstructed mapping feature is obtained after F2 reconstruction. F3 reconstructed as The third reconstruction mapping feature; will and Multiplication uses softmax to generate a spatial attention graph S∈R (H×W)×(H×W) Then After multiplying with the spatial attention map S, the original scale is restored to generate the spatial attention feature F. sa ∈R C×H×W ; Global Enhancement Feature F dm ∈R C×H×W The input channel attention mechanism compresses the input through global average pooling (AvgPool) and global max pooling (MaxPool) to obtain the average pooled feature F. avg and max pooling feature F max , for F avg and F max Aggregation is performed, and additive operations are used to obtain the additive feature map F. add ∈R C×1×1 The splicing operation yields the spliced ​​feature map F. cat ∈R 2C×1×1 Then add the feature map F add ∈R C×1×1 , splicing feature map F cat ∈R 2C×1×1 The inputs are fed into a two-layer fully connected neural network. The first fully connected layer has C / 16 neurons, and the second fully connected layer has C neurons with shared weights. The output information is summed and activated by sigmoid to generate channel attention features F. c ∈R C×H×W ; The global enhancement features are multiplied by the spatial attention features and the channel attention features respectively, and then element-wise summation is performed to generate the output attention combination feature F. sc ∈R C×H×W Global enhancement features and attention-based combined features F sc ∈R C×H×W After multiplication, the number of feature channels is increased by 1×1 convolution to obtain the self-attention feature F. a ; The formula for calculating the self-attention component of the spatial channel is shown below: F1=Conv 1×1 (F dm ),F2=Conv 1×1 (F dm ),F3=Conv 1×1 (F dm ) F avg =AvgPool(F dm ),F max =MaxPool(F dm ) F add =F avg +F max ,F cat =Cat(F avg ,F max ) FAVORITE c =σ(V2(V0(F add ))+V2(V1(F cat )) Where re(·) represents the reconstruction operation, F sa This indicates the spatial attention features generated at the original scale. This represents matrix multiplication; σ(·) is the softmax operator; V0 represents F add The weights of the first fully connected layer, V1 represents F cat The weights of the first fully connected layer; V2 represents F add and F cat The shared weights of the second fully connected layer, F a ∈R C×H×W This represents the self-attention characteristics of the output of the self-attention component of the spatial channel; The cloud and snow edge features and self-attention features are combined through a spatial feature exchange and fusion component, and the cloud and snow edge features F ms Mapped to self-attention features F by downsampling a After size, with self-attention feature F a Add them together to obtain the overlapping feature U; Then, global max pooling and global average pooling operations are applied to the overlapping features U respectively, and spatial information is extracted through two layers of convolution to obtain the total spatial weight w; Then, perform a softmax operation on the total spatial weights w to obtain the spatial weights w1, w2 ∈ R. 2×h×w Spatial weights w1 and cloud / snow edge features F ms Matrix multiplication yields weighted cloud and snow edge features, spatial weights w2, and self-attention features F. a Matrix multiplication yields weighted self-attention features, which are then upsampled and added to weighted cloud / snow edge features. This is followed by normalized attention processing to obtain the final output fused feature F. u ; The calculation formula for the spatial feature exchange and integration part is expressed as follows: U=F a +down(F ms ) w=Conv 1×3 (Conv 3×1 (Cat(AvgPool(U),MaxPool(U))) (w1,w2)=σ(w) In the formula, down(·) represents the downsampling operation, σ(w) represents the total spatial weights after the softmax operation, and w1 and w2 represent F respectively. a and F ms Spatial weights; This represents the weighted self-attention features. This represents the weighted edge features of clouds and snow, up(·) represents a 2x upsampling operation using bilinear interpolation, NAM represents the normalized attention mechanism, and F u This indicates the fusion characteristics output by the module; The output of the previous feature interaction fusion upsampling module is used as the input of the next feature interaction fusion upsampling module. After being processed by three feature interaction fusion upsampling modules, the final image is obtained by bilinear interpolation upsampling.

6. The cloud and snow recognition method with multi-level attention interaction according to claim 2, characterized in that, The background includes forests, waterways, cities, grasslands, and bare land.

7. The cloud and snow recognition method with multi-level attention interaction according to claim 2, characterized in that, The clouds include cirrus clouds, altocumulus clouds, cirrus clouds, cumulus clouds, and stratus clouds.

8. The cloud and snow recognition method with multi-level attention interaction according to claim 2, characterized in that, The snow includes permanent snow, stable snow, and unstable snow.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

10. A device, characterized in that: include, One or more processors, one or more memories, and one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods according to claims 1 to 8.

Images