A cloud image segmentation method based on a fusion transformation network
By using a fusion transform network approach, the problem of insufficient long-distance dependencies in cloud image segmentation was solved, resulting in more accurate cloud image segmentation and improved segmentation accuracy.
Patent Information
- Application Number
- CN202211698791.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-28
- Publication Date
- 2026-01-02
- Estimated Expiration
- 2042-12-28
AI Technical Summary
Existing cloud image segmentation methods are insufficient when modeling long-distance dependencies, resulting in inaccurate segmentation results.
A method based on fusion transformation network is adopted. Multi-scale local features are extracted through a pre-trained deep learning model, long-distance dependencies are learned by the transformation network, and the feature fusion module interacts and fuses the features to finally generate a segmentation mask.
It improves the accuracy of cloud image segmentation, especially in learning multi-scale information and long-distance dependencies, and enhances the ability to represent features.
Smart Images
Figure CN116091764B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the field of computer vision, pattern recognition and artificial intelligence, and particularly relates to a cloud image segmentation method based on a fusion transformation network. BACKGROUND
[0002] Cloud image segmentation technology can realize the estimation of cloud amount and is widely used in environmental detection, weather forecasting and other fields, so it has attracted widespread attention from academia and industry. It mainly studies the method of segmenting clouds from images. However, due to the blurred edges and various shapes of clouds, cloud image segmentation technology faces great challenges.
[0003] In recent years, with the development of cloud image acquisition devices, many cloud image segmentation algorithms have been proposed. Long et al. proposed a cloud image segmentation method based on color threshold, taking the ratio of red and blue channels of cloud image as a fixed threshold for cloud image segmentation. After that, Heinle et al. used the difference between the red and blue channels of the cloud image instead of the ratio as a fixed threshold for cloud image segmentation. Li et al. proposed a hybrid method of fixed threshold and adaptive threshold for single-peak and double-peak cloud image segmentation. Liu et al. first generated high-confidence labels as hard constraint seeds, and then used graph cut method to segment the cloud image.
[0004] Due to the strong representation ability of convolutional neural network, it is used in the field of cloud image segmentation. This kind of method usually has an encoder-decoder structure. For example, Dev et al. proposed CloudSegNet, which uses convolution, max-pooling layer and up-sampling as a light version of the encoder-decoder architecture for cloud image segmentation in daytime and nighttime. Xie et al. proposed SegCloud, which effectively recovered the loss of pooling for cloud image segmentation by introducing a pooling index in the up-sampling operation. Shi et al. proposed EFCN, which improved the segmentation performance by applying skip connection and histogram equalization. Zhang et al. integrated multi-scale strategy and attention mechanism into the encoder-decoder structure, so as to learn discriminative features to improve the segmentation performance. Zhou et al. proposed TL-DeepLabV3+ for cloud image segmentation task, which overcomes the shortage of small number of training cloud image samples by using transfer learning.
[0005] Due to the local property of convolutional filter, it is difficult to model long-range dependencies. In the field of natural language processing, the transformer network has strong modeling ability for long-range dependencies. Therefore, researchers apply the transformer network to the field of computer vision. Vision transformer (ViT) proves that the transformer network can surpass the convolutional neural network-based method in image classification. Zheng et al. transform the semantic segmentation task into a sequence-to-sequence prediction task and prove the feasibility of the transformer network in the segmentation task. Petit et al. insert the transformer network into the U-Net to obtain long-range dependencies.
[0006] The above methods based on the transformer network model long-range dependencies by using single-scale features, ignoring multi-scale information, resulting in inaccurate segmentation results. SUMMARY
[0007] The purpose of the present application is to solve the technical problem that long-range modeling of cloud images has a great impact on cloud image segmentation results. To this end, the present application provides a cloud image segmentation method based on a fusion transformer network.
[0008] In order to achieve the above purpose, the present application provides a cloud image segmentation method based on a fusion transformer network, which comprises the following steps:
[0009] Step S1, a feature extraction module is constructed by using a pre-trained deep learning model, a training cloud image is input into the feature extraction module, and multi-scale local features f i of the training cloud image are obtained.
[0010] Step S2, a transformer network module is constructed, the multi-scale local features f i of the training cloud image are input into the transformer network module, and the transformer network module is used to learn long-range dependencies of different scales, and a multi-scale long-range dependency sequence
[0011] Step S3, a feature fusion module is constructed, and the multi-scale long-range dependency sequence of the training cloud image is interacted and fused by using the feature fusion module to obtain a cross-scale sequence C i .
[0012] Step S4, the cross-scale sequence C i is input into the feature fusion module to obtain the final cross-scale feature C, a segmentation mask generation module is constructed, and the segmentation mask generation module is used to obtain the segmentation mask of the training cloud image.
[0013] Step S5, the feature extraction module, the transformation network module, the feature fusion module and the loss calculation module are combined to form a cloud image segmentation model, a loss calculation module is constructed, the segmentation mask of the training cloud image and the label of the training cloud image are input into the loss calculation module, the obtained loss value is used to optimize the cloud image segmentation model, and an optimal cloud image segmentation model is obtained;
[0014] Step S6, in the test stage, the optimal cloud image segmentation model is used to calculate the final segmentation mask of the input cloud image, the similarity between the segmentation mask of the training cloud image and the final segmentation mask of the input cloud image is calculated, and the segmentation result of the input cloud image is obtained in combination with the label of the training cloud image.
[0015] Optionally, the step S1 comprises the following steps:
[0016] Step S11, a pre-trained deep learning model is determined and parameterized, the last global pooling layer and the full connection layer in the pre-trained deep learning model are removed, and the feature extraction module is obtained;
[0017] Step S12, the training cloud image in the training set is preprocessed;
[0018] Step S13, the training cloud image obtained after preprocessing is input into the feature extraction module, and the multi-scale local feature f i .
[0019] Optionally, the step S2 comprises the following steps:
[0020] Step S21, the multi-scale local feature f i of the training cloud image is converted into a multi-scale sequence E i .
[0021] Step S22, a transformation network layer is constructed based on a layer normalization module, a multi-head self-attention module, a residual structure and a multi-layer perception module;
[0022] Step S23, the transformation network module is constructed by using N layers of transformation network layers;
[0023] Step S24, the multi-scale sequence E i is input into the transformation network module, and the multi-scale sequence E i is used to construct long-distance dependence relationship of different scales, and a multi-scale long-distance dependence sequence E
[0024] Optionally, the multi-scale long-distance dependence sequence E is represented as:
[0025]
[0026]
[0027] wherein, Ci represents the sequence of the i-th scale E i The output of the n-th layer of the transformation network layer, L represents the number of sequences, D represents the dimension of each sequence, LN represents the layer normalization module, MLP represents the multi-layer perception module, MSA represents the multi-head self-attention module, Ci represents the sequence of the i-th scale output in the n-1-th layer of the transformation network layer.
[0028] Optionally, the step S3 comprises the following steps:
[0029] Step S31, constructing the feature fusion module based on the layer normalization module, the multi-head mutual attention module, the residual structure and the multi-layer perception module;
[0030] Step S32, using the feature fusion module to interact and fuse the multi-scale long-distance dependence relationship sequences of the training cloud image to obtain the cross-scale sequence Ci of the training cloud image.
[0031] Optionally, the i-th cross-scale sequence C i of the training cloud image is represented as:
[0032] Ci i = Cat(Hi , H i+1 ), (i = 1, 2)
[0033]
[0034]
[0035] wherein, Ci i ∈R 2L×D represents the i-th cross-scale sequence, Cat is a concatenation operation, MCA represents a multi-head mutual attention module, Ci represents the i-th scale long-distance dependence relationship sequence, Ci+1 represents the i+1-th scale long-distance dependence relationship sequence.
[0036] Optionally, the step S4 comprises the following steps:
[0037] Step S41, inputting the cross-scale sequences C1 and C2 into the feature fusion module to obtain the final cross-scale feature C;
[0038] Step S42, a segmentation mask generation module is constructed by using the reshaping, convolution and up-sampling module;
[0039] Step S43, inputting the final cross-scale feature C into the segmentation mask generation module, and converting the final cross-scale feature C into the segmentation mask of the training cloud image by using the segmentation mask generation module.
[0040] Optionally, the step S5 comprises the following steps:
[0041] Step S51, the feature extraction module, the transformation network module, the feature fusion module and the loss calculation module are combined to form a cloud image segmentation model;
[0042] Step S52, the loss calculation module is constructed, the segmentation mask of the training cloud image and the label of the training cloud image are input into the loss calculation module, and the loss value of the segmentation mask of the training cloud image is calculated by using the loss calculation module;
[0043] Step S53, the cloud image segmentation model is optimized by using the loss value, and an optimal cloud image segmentation model is obtained.
[0044] Optionally, the loss function Loss is represented as:
[0045]
[0046]
[0047] Wherein, T represents the total number of pixels in the training cloud image, u i is the true label of the i th pixel of the training cloud image, when the category of the i th pixel of the training cloud image is cloud, u i =1, otherwise u i =0, p i represents the category of the i th pixel in the segmentation mask of the training cloud image, when the category of the i th pixel in the segmentation mask of the training cloud image is cloud, p i =1, otherwise p i =0.
[0048] The application has the beneficial effects that: the convolutional neural network is used to learn the multi-scale local features of the cloud image, and the transformation network is used to model the long-distance dependent relationship of the multi-scale local features, so that the multi-scale local information and the long-distance dependent relationship are simultaneously learned, the representation ability of the features is improved, and the accuracy of the cloud image segmentation is improved.
[0049] The application is supported by the National Natural Science Foundation of China (Grant No. 62171321) and the Tianjin Municipal Natural Science Foundation (Grant No. 20JCZDJC00180). BRIEF DESCRIPTION OF DRAWINGS
[0050] Figure 1 is a flowchart of a cloud image segmentation method based on a fusion transformation network according to an embodiment of the application. DETAILED DESCRIPTION
[0051] To make the objectives, technical solutions, and advantages of the application clearer, further detailed descriptions will be given below with reference to the embodiments and the accompanying drawings. It should be understood that the descriptions are only exemplary and are not intended to limit the scope of the application. In addition, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessary confusion of the concept of the application.
[0052] Figure 1 is a flowchart of a cloud image segmentation method based on a fusion transformation network according to an embodiment of the application, and some specific implementation processes of the application will be described below with reference to an example as shown in Figure 1 Figure 1 The cloud image segmentation method based on the fusion transformation network includes the following steps:
[0053] Step S1, constructing a feature extraction module by using a pre-trained deep learning model, inputting a training cloud image into the feature extraction module, and obtaining multi-scale local features fi of the training cloud image;
[0054] Further, the step S1 includes the following steps:
[0055] Step S11, determining a pre-trained deep learning model and initializing parameters thereof, removing a last global pooling layer and a full connection layer in the pre-trained deep learning model, and obtaining the feature extraction module;
[0056] In an embodiment of the application, the pre-trained deep learning model used can be ResNet-50(BiT), and the model component before the last global pooling layer in the model is selected, that is, the last global pooling layer and the full connection layer in the pre-trained deep learning model are removed, and the remaining model part constitutes the feature extraction module. The feature extraction module is composed of three modules, which are named as Stage1, Stage2, and Stage3, respectively, and the feature extraction module can generate features of different scales at Stage1, Stage2, and Stage3, respectively.
[0057] Step S12, preprocessing the training cloud image in the training set;
[0058] In an embodiment of the present application, the pre-processing of the training cloud image comprises: randomly horizontally flipping the training cloud image, the probability can be set to 0.5, and reducing all pixels of the training cloud image to a preset range, such as between 0 and 1, then subtracting the pixel average value of the training cloud image from each pixel value of the training cloud image, and dividing by the pixel variance of the training cloud image.
[0059] Step S13, inputting the training cloud image obtained after the pre-processing into the feature extraction module to obtain the multi-scale local feature f of the training cloud image. i .
[0060] In an embodiment of the present application, the multi-scale local feature f of the training cloud image comprises: That is, and respectively from the outputs of Stage1, Stage2 and Stage3 in the feature extraction module. Wherein, H is the height of the training cloud image, W is the width of the training cloud image, 256·2 i-1 is the number of channels in the i-th scale feature of the training cloud image.
[0061] In an embodiment of the present application, H = 512, W = 512.
[0062] Step S2, constructing a transformation network module, inputting the multi-scale local feature f of the training cloud image into the transformation network module, learning the long-distance dependence relationship of different scales by using the transformation network module, and obtaining the multi-scale long-distance dependence relationship sequence E of the training cloud image. i
[0063] Further, the step S2 comprises the following steps:
[0064] Step S21, converting the multi-scale local feature f of the training cloud image into a multi-scale sequence E i ;
[0065] In an embodiment of the present application, the multi-scale local feature f of the training cloud image can be first uniformly divided into L feature blocks using a square grid with a size of , wherein the size of each feature block is After that, each feature block is expanded into a one-dimensional vector, and finally the dimension of the one-dimensional vector is converted to D by using a linear layer. The multi-scale local feature f of the training cloud image can be converted into L multi-scale sequences E i ∈R L×D , wherein P is a positive integer for controlling the size of the divided square grid.
[0066] In an embodiment of the present application, D = 768.
[0067] In an embodiment of the present application, P = 8, L = 4096, the multi-scale sequence E of the training cloud image is constructed based on the multi-scale sequence E of the training cloud image, and the multi-scale sequence E of the training cloud image is constructed based on the multi-scale sequence E of the training cloud image. i ∈R 4096×768 , i.e. E 1 ,E 2 ,E 3 ∈R 4096×768 .
[0068] Step S22, constructing a transformation network layer based on a layer normalization module, a multi-head self-attention module, a residual structure and a multi-layer perception module.
[0069] Among them, the layer normalization module, the multi-head self-attention module, the residual structure and the multi-layer perception module are commonly used calculation modules in the field, and the present disclosure does not make too much introduction.
[0070] In this step, first, the data distribution is standardized by using the layer normalization module to accelerate the model convergence, then the global dependency relationship is learned by using the multi-head self-attention module, and finally the number of parameters is reduced by using the multi-layer perception module, in addition, the residual structure is used to relieve the problem of gradient vanishing or explosion.
[0071] Step S23, constructing the transformation network module by using N layers of transformation network layers.
[0072] In an embodiment of the present application, N = 2.
[0073] Among them, the N layers of transformation network layers are connected in series to obtain the transformation network module, for example, when N = 2, the output of the first layer of transformation network layers is connected to the input of the second layer of transformation network layers to obtain the transformation network module.
[0074] Step S24, inputting the multi-scale sequence E i to the transformation network module, and constructing a long-distance dependency relationship of different scales by using the transformation network module on the multi-scale sequence E i to obtain a multi-scale long-distance dependency relationship sequence E of the training cloud image.
[0075] In an embodiment of the present application, the multi-scale long-distance dependency relationship sequence E of the training cloud image can be represented as:
[0076]
[0077]
[0078] Among them, represents the sequence E of the i-th scale. i The output of the nth layer of the transformation network layer, L represents the number of sequences, D represents the dimension of each sequence, LN represents the layer normalization module, MLP represents the multi-layer perception module, and MSA represents the multi-head self-attention module, represents the sequence of the i-th scale output in the n-1th layer of the transformation network layer, when n = 1,
[0079] In an embodiment of the present application, the multi-head self-attention module is composed of a plurality of individual self-attention modules, and the sequence E i The output of the multi-head self-attention module in the nth layer of the transformation network layer can be represented as:
[0080]
[0081]
[0082] wherein, represents the output of the input of the nth layer of the transformation network layer after undergoing the first layer normalization module, and M is the number of individual self-attention modules, is the sequence E i The output of the mth individual self-attention module in the nth layer of the transformation network layer, is the training parameter in the individual linear layer, and d represents the dimension of the output of each individual self-attention module.
[0083] In an embodiment of the present application, M = 12 and d = 768.
[0084] In an embodiment of the present application, the sequence E i The output of the mth individual self-attention module in the nth layer of the transformation network layer can be represented as:
[0085]
[0086]
[0087]
[0088]
[0089] wherein, softmax is an activation function, respectively, are the training parameters in the three individual linear layers.
[0090] Step S3, constructing a feature fusion module, using the feature fusion module to obtain the multi-scale long-distance dependence sequence of the training cloud image interact and fuse to obtain a cross-scale sequence C of the training cloud image i ;
[0091] Further, the step S3 comprises the following steps:
[0092] Step S31, constructing the feature fusion module based on a layer normalization module, a multi-head self-attention module, a residual structure and a multi-layer perception module.
[0093] In this step, first, data distribution standardization is performed by using the layer normalization module to accelerate model convergence, then the multi-head self-attention module is used to learn global dependency, and finally the multi-layer perception module is used to reduce the number of parameters, in addition, the residual structure is used to relieve the problem of gradient vanishing or explosion.
[0094] Step S32, using the feature fusion module to interact with the multi-scale long-distance dependency sequence of the training cloud image and fuse to obtain a cross-scale sequence C of the training cloud image i .
[0095] In an embodiment of the present application, the i-th cross-scale sequence C of the training cloud image i can be represented as:
[0096] C i = Cat(H i ,H i+1 ), (i = 1, 2)
[0097]
[0098]
[0099] wherein C i ∈ R 2L×D represents the i-th cross-scale sequence, Cat is a concatenation operation, MCA represents a multi-head self-attention module, represents the i-th scale long-distance dependency sequence, represents the i+1-th scale long-distance dependency sequence.
[0100] In an embodiment of the present application, the multi-head self-attention module is composed of a plurality of individual self-attention modules, the i-th scale long-distance dependency sequence and the i+1-th scale long-distance dependency sequence The output of the multi-head self-attention module can be represented as:
[0101]
[0102] wherein, M is the number of independent cross-attention modules, is the i-th scale long-range dependency sequence and the i+1-th scale long-range dependency sequence is the output of the m-th independent cross-attention module, is the training parameter in the individual linear layer.
[0103] In an embodiment of the present application, M = 12.
[0104] is the i-th scale long-range dependency sequence and the i+1-th scale long-range dependency sequence The output of the m-th independent cross-attention module can be expressed as:
[0105]
[0106]
[0107]
[0108]
[0109] wherein, softmax is an activation function, is the training parameter in the three individual linear layers, respectively.
[0110] Step S4, input the cross-scale sequence C i into the feature fusion module to obtain the final cross-scale feature C, and construct a segmentation mask generation module to obtain the segmentation mask of the training cloud image by using the segmentation mask generation module.
[0111] Further, when i = 1, 2, the step S4 comprises the following steps:
[0112] Step S41, input the cross-scale sequences C1 and C2 into the feature fusion module to obtain the final cross-scale feature C.
[0113] In an embodiment of the present application, C ∈ R 4L×D .
[0114] Step S42, a segmentation mask generation module is constructed by using a reshaping, convolution and up-sampling module.
[0115] Wherein, when constructing the segmentation mask generation module, the reshaping, convolution and up-sampling modules can be connected in sequence.
[0116] Step S43, input the final cross-scale feature C into the segmentation mask generation module, and convert the final cross-scale feature C into the segmentation mask of the training cloud image by using the segmentation mask generation module.
[0117] Step S5, assemble the feature extraction module, the transformation network module, the feature fusion module and the loss calculation module into a cloud image segmentation model, construct a loss calculation module, input the segmentation mask of the training cloud image and the label of the training cloud image into the loss calculation module, and optimize the cloud image segmentation model by using the obtained loss value to obtain an optimal cloud image segmentation model.
[0118] Further, the step S5 comprises the following steps:
[0119] Step S51, assemble the feature extraction module, the transformation network module, the feature fusion module and the loss calculation module into a cloud image segmentation model.
[0120] Step S52, construct a loss calculation module, input the segmentation mask of the training cloud image and the label of the training cloud image into the loss calculation module, and calculate the loss value of the segmentation mask of the training cloud image by using the loss calculation module.
[0121] In an embodiment of the present application, when training in the TLCD database, the loss function Loss of the constructed loss calculation module can be expressed as:
[0122]
[0123] Wherein, T represents the total number of pixels in the training cloud image, u i is the true label of the i-th pixel of the training cloud image, when the category of the i-th pixel of the training cloud image is cloud, u i = 1, otherwise u i = 0, p i represents the category of the i-th pixel in the segmentation mask of the training cloud image, when the category of the i-th pixel in the segmentation mask of the training cloud image is cloud, p i = 1, otherwise p i = 0.
[0124] In an embodiment of the present application, T = 262144.
[0125] Step S53, optimize the cloud image segmentation model by using the loss value to obtain an optimal cloud image segmentation model.
[0126] In this step, the random gradient descent method can be used for iterative calculation to optimize the cloud image segmentation model.
[0127] Step S6, in the test stage, the final segmentation mask of the input cloud image is calculated by using the optimal cloud image segmentation model, the similarity between the segmentation mask of the training cloud image and the final segmentation mask of the input cloud image is calculated, and the segmentation result of the input cloud image is obtained in combination with the label of the training cloud image.
[0128] The label of the training cloud image is a known real segmentation result.
[0129] The segmentation mask of the training cloud image is a segmentation result predicted by a cloud image segmentation model.
[0130] When tested on the TLCD database, the correct segmentation rate of the cloud image segmentation of the present application reaches 78.67% (F-score), 83.12% (Recall), 77.60% (Precision), 91.38% (Accuracy) and 71.63% (IoU). The present application learns the multi-scale local features of the cloud image through the convolutional neural network, and then models the long-distance dependency relationship of the multi-scale local features through the transformation network, so as to realize the simultaneous learning of the multi-scale local information and the long-distance dependency relationship, greatly improving the accuracy of the cloud image segmentation. Therefore, the effectiveness of the method of the present application is proved.
[0131] It should be understood that the above specific embodiments of the present application are only used for illustrative or explanatory purposes of the principles of the present application, and do not constitute a limitation of the present application. Therefore, any modification, equivalent replacement, improvement, etc. made without departing from the spirit and scope of the present application shall be included in the protection scope of the present application. In addition, the appended claims of the present application are intended to cover all variations and modifications falling within the scope and boundary of the appended claims, or the equivalent forms of such scope and boundary.
Claims
1. A cloud image segmentation method based on a fusion transformation network, characterized in that, The method comprises the following steps: Step S1, a pre-trained deep learning model is used to construct a feature extraction module, a training cloud image is input into the feature extraction module, and multi-scale local features of the training cloud image are obtained ; Step S2, constructing a transformation network module, inputting the multi-scale local features of the training cloud image into the transformation network module, learning long-distance dependency relationships of different scales by using the transformation network module, and obtaining a multi-scale long-distance dependency relationship sequence of the training cloud image ; and Step S3, constructing a feature fusion module, using the feature fusion module for the multi-scale long-distance dependence relationship sequence of the training cloud image Interacting and fusing to obtain the cross-scale sequence of the training cloud image ; Step S4, constructing the cross-scale sequence inputting the feature fusion module to obtain the final cross-scale feature , constructing a segmentation mask generation module, and obtaining a segmentation mask of the training cloud image by using the segmentation mask generation module; Step S5, the feature extraction module, the transformation network module, the feature fusion module and the loss calculation module are combined to form a cloud image segmentation model, a loss calculation module is constructed, the segmentation mask of the training cloud image and the label of the training cloud image are input into the loss calculation module, the obtained loss value is used to optimize the cloud image segmentation model, and an optimal cloud image segmentation model is obtained; Step S6, in the test stage, the optimal cloud image segmentation model is used to calculate the final segmentation mask of the input cloud image, the similarity between the segmentation mask of the training cloud image and the final segmentation mask of the input cloud image is calculated, and the segmentation result of the input cloud image is obtained in combination with the label of the training cloud image; The step S3 comprises the following steps: Step S31, based on the layer normalization module, the multi-head self-attention module, the residual structure and the multi-layer perception module, the feature fusion module is constructed; Step S32, using the feature fusion module for the multi-scale long-distance dependence sequence of the training cloud image Interact and fuse to obtain the cross-scale sequence of the training cloud image ; The first cross-scale sequence of the training cloud image is represented as: is represented as: ； ； ； wherein, represents the first cross-scale sequence, Cat is a concatenation operation, MCA represents a multi-head cross-attention module, represents the first scale long distance dependency relationship sequence, represents the first scale long distance dependency relationship sequence; The step S4 comprises the following steps: Step S41, inputting the cross-scale sequence into the feature fusion module to obtain the final cross-scale feature and ; Step S42, a segmentation mask generation module is constructed by using the reshaping, convolution and up-sampling modules; Step S43, the final cross-scale features are input into the segmentation mask generation module, and the segmentation mask generation module is used to convert the final cross-scale features into the segmentation mask of the training cloud image.
2. The method of claim 1, wherein, The step S1 comprises the following steps: Step S11, a pre-trained deep learning model is determined, and the parameters thereof are initialized, the last global pooling layer and the fully connected layer in the pre-trained deep learning model are removed, and the feature extraction module is obtained; Step S12, the training cloud images in the training set are preprocessed; Step S13, input the training cloud image obtained after preprocessing into the feature extraction module to obtain the multi-scale local features of the training cloud image .
3. The method of claim 1, wherein, The step S2 comprises the following steps: Step S21, converting the multi-scale local features of the training cloud image into a multi-scale sequence ; Step S22, based on the layer normalization module, the multi-head self-attention module, the residual structure and the multi-layer perception module, the transformation network layer is constructed; Step S23, using The transform network module is constructed by a layer transform network layer. Step S24, inputting the multi-scale sequence to the transformation network module, using the transformation network module to transform the multi-scale sequence to construct long-distance dependency relationships of different scales, to obtain a multi-scale long-distance dependency relationship sequence of the training cloud image .
4. The method of claim 3, wherein, The multi-scale long-distance dependency sequence is represented as: ； ； wherein, represents the output of the th scale in the th layer of the transformer network layer, th layer of the transformer network layer, represents the number of sequences, D represents the dimension of each sequence, LN represents a layer normalization module, MLP represents a multi-layer perception module, and MSA represents a multi-head self-attention module, represents the output of the th scale in the th layer of the transformer network layer.
5. The method of claim 1, wherein, The step S5 comprises the following steps: Step S51, the feature extraction module, the transformation network module, the feature fusion module and the loss calculation module are combined to form a cloud image segmentation model; Step S52, a loss calculation module is constructed, the segmentation mask of the training cloud image and the label of the training cloud image are input into the loss calculation module, and the loss value of the segmentation mask of the training cloud image is calculated by using the loss calculation module; Step S53, the cloud image segmentation model is optimized by using the loss value, and an optimal cloud image segmentation model is obtained.
6. The method of claim 5, wherein, loss function of the loss calculation module is represented as: ； wherein, denotes the total number of pixels in the training cloud image, is the true label of the i-th pixel of the training cloud image, when the class of the i-th pixel of the training cloud image is cloud, = 1, otherwise denotes the class of the i-th pixel in the segmentation mask of the training cloud image, when the class of the i-th pixel in the segmentation mask of the training cloud image is cloud, = 1, otherwise .