Improved U-Net cloud image segmentation method based on multi-scale loss function

Abstract

The present invention discloses an improved U-Net cloud image segmentation method based on a multi-scale loss function, comprising the following steps: (1) obtaining a full sky image segmentation database, pre-processing the cloud image contained in the full sky image segmentation database and the binary labels corresponding thereto to obtain a data set; (2) constructing an improved U-Net framework by changing the convolution mode, adding an efficient channel attention mechanism, modifying normalization, and adding a multi-scale feature fusion method; (3) combining the advantages of the Dice loss function, the cross entropy loss function, the boundary loss function, and the Log_Cosh loss function, an improved composite loss function is proposed; (4) inputting the data set obtained in step 1 into the improved U-Net model for training and testing, obtaining an optimal parameter model, and outputting a prediction effect diagram. The present invention makes the cloud image segmentation accuracy higher and the generalization effect better, and has very important research and application value.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning, and in particular relates to an improved U-Net cloud graph segmentation method based on a multi-scale loss function. Background Art

[0002] With the development of remote sensing image processing technology, clouds, as an important meteorological element, have been widely used in short-term weather forecasts, precipitation forecasts, satellite communication connections, etc. Cloud image segmentation, as the first step in cloud image analysis, has very important research significance. The spectral information of clouds is determined by factors such as particle size, water vapor, height, and optical thickness. The spectral characteristics of clouds on images have various manifestations, and the brightness, transparency, texture and shape of clouds themselves have different manifestations. In high-resolution remote sensing satellite images, some areas will be contaminated by clouds or even completely covered, which will affect the classification, segmentation, change detection and image matching of remote sensing images.

[0003] Experts and scholars have done a lot of research. Based on the threshold method, Long et al. calculated the ratio of the R channel to the B channel in the RGB color space, and then selected a suitable threshold value based on the ratio to determine the cloud and non-cloud areas; with the development of machine learning, Taravat et al. expanded the machine learning algorithm - support vector machine and neural network automatic segmentation algorithm in the field of cloud maps; with the development of deep learning, Ronneberger et al. proposed U-Net for image segmentation, which spliced ​​the encoded and decoded feature maps through jump connections to effectively integrate shallow detail information and deep semantic information. However, traditional cloud segmentation based on threshold methods and machine learning methods is very sensitive to the selection of algorithm parameters, and relies on researchers to do a lot of feature extraction. U-Net for image segmentation will equally distribute the information on all spatial positions and channels on the feature tensor, which will generate a lot of computational redundancy, resulting in slower model training and lower segmentation accuracy. Summary of the invention

[0004] Purpose of the invention: In view of the problems existing in cloud image segmentation in the background technology, a remote sensing cloud image segmentation method based on an improved U-Net with an improved loss function is proposed, which has higher accuracy and better generalization effect, and has very important research and application value.

[0005] Technical solution: An improved U-Net cloud image segmentation method based on a multi-scale loss function includes the following steps:

[0006] (1) obtaining a full sky image segmentation database, and preprocessing the cloud images and the binary labels corresponding to the cloud images contained in the full sky image segmentation database to obtain a data set;

[0007] (2) An improved U-Net model is constructed by changing the convolution mode, adding an efficient channel attention mechanism, modifying normalization, and adding multi-scale feature fusion;

[0008] (3) An improved composite loss function of the improved U-Net model is proposed, which is expressed as:

[0009] L improve =βL improve2 +(1-β)L B

[0010] Among them, L B Represents the boundary loss function, and the formula of the boundary loss function is as follows:

[0011]

[0012] in, ΔS=|S|-|S∩G|+|G|-|S∩G|, S is the pixel point on the boundary of the prediction result, G is the pixel point on the boundary of the true label, p is any pixel point on the boundary of the label, q is any pixel point within ΔS, D G (q) is the distance between two pixels;

[0013] The initial value of β is set to 1 and decreases by 0.01 in each round of training;

[0014] L improve2 It is a function that combines the Dice loss function and the cross entropy loss function. The expression is:

[0015] L improve2 =αL improve1 +(1-α)L CE

[0016] Where α is a hyperparameter, L CE is the cross entropy loss function, and the formula is as follows:

[0017]

[0018] Where N is the number of pixels in the feature map, pi∈[0,1] is the value of the i-th pixel in the segmentation map predicted by the model, and gi∈{0,1} is the value of the i-th pixel in the binary label;

[0019] L improve1 It is a function that combines the Dice loss function and the Log_Cosh loss function. The expression is:

[0020] L improve1 =log(Cosh(L Dice ))

[0021] The formula of the Log_Cosh loss function is as follows:

[0022] L Log_Cosh =log(Cosh(x))

[0023] Among them, L Dice is the Dice loss function, expressed as follows:

[0024]

[0025] Among them, A and B refer to the model prediction result and the true label result respectively, smooth is the smoothing factor, N is the number of pixels in the feature map, and p i ∈[0,1] is the value of the i-th pixel in the segmentation map predicted by the model, g i ∈{0,1} is the value of the i-th pixel in the binary label.

[0026] (4) Input the data set obtained in step (1) into the improved U-Net model for training and testing, obtain the optimal parameter model, and output the prediction effect diagram.

[0027] The specific process of step (1) is as follows:

[0028] (1.1) Obtain the sky-cloud image and its corresponding binary label contained in the full sky image segmentation database;

[0029] (1.2) The training samples are randomly divided into small samples, and the data enhancement method is used on the sample set to expand the data set ten times the original size and divide it into three parts, with the training set accounting for 80%, the validation set accounting for 10%, and the test set accounting for 10%.

[0030] In the step (1.2), the data enhancement method includes random pruning, translation transformation and noise perturbation.

[0031] The specific process of step (2) is as follows:

[0032] (2.1) The method of changing the convolution mode is used to improve U-Net. Specifically, based on the U-Net segmentation model, the first two layers and the first convolution of the third layer in the encoding part are replaced with deformable convolutions;

[0033] (2.2) The method of adding an efficient channel attention mechanism to improve U-Net is as follows: Based on the model (2.1), a channel attention mechanism is added after the feature map output by the first three layers of the encoding part. After the feature map output by the encoding part generates a one-dimensional attention vector through the efficient channel attention mechanism, the corresponding elements are multiplied with the original feature map to obtain a weighted feature map. The size of the feature map remains unchanged. The weighted feature map of the first three layers is downsampled, and the weighted feature map of the first three layers is concatenated with the feature map of the decoding part.

[0034] (2.3) Using the modified normalization method to improve U-Net, specifically: Based on the model (2.2), WeightNormalization is added between the convolutional layer and the activation layer of U-Net, the weights of the deep network are rewritten, and the normalization operation is achieved by standardizing the network parameters;

[0035] (2.4) The method of adding multi-scale feature fusion is used to improve U-Net. Specifically, based on the model (2.3), multi-scale feature fusion is performed on the feature map output by the last layer of the encoding part and the feature map output by each layer of the decoding part.

[0036] The encoding part in (2.1) is 5 layers: the first layer includes 2 identical variable convolution blocks deform-conv11 and deform-conv12, and the variable convolution block is mainly composed of two convolution blocks, one of which is composed of 18 offset convolution kernels with a step size of 1, a padding of 1, and a size of 3×3, and the other convolution block is composed of 32 standard convolution kernels conv11 and conv12 with a step size of 1, a padding of 1, and a size of 3×3; the second layer includes 2 identical variable convolution blocks deform-conv21 and deform-conv22, and the variable convolution block is mainly composed of two convolution blocks, one of which is composed of 18 offset convolution kernels with a step size of 1, a padding of 1, and a size of 3×3, and the other convolution block is composed of 64 standard convolution kernels conv21 and conv22 with a step size of 1, a padding of 1, and a size of 3×3; the third layer includes 2 convolution blocks deform-conv3 and conv3, where con v3 consists of 128 standard convolution kernels with a stride of 1, padding of 1, and a size of 3×3. deform-conv3 consists of 18 offset convolutions with a stride of 1, padding of 1, and a size of 3×3 and conv3. The fourth layer includes two identical convolution blocks conv4, each consisting of 256 standard convolution kernels with a stride of 1, padding of 1, and a size of 3×3. The fifth layer includes two identical convolution blocks conv5, each consisting of 512 stride of 1, padding of 1, and a standard convolution kernel of size 3×3; the decoding part of this model consists of 4 layers: each of them is composed of an upsampling module and two identical standard convolution modules corresponding to the encoding part. The number of convolution kernels in each module of each layer is (512, 256, 256), (256, 128, 128), (128, 64, 64), (64, 32, 32), and the kernel size is 3×3. At the same time, the fourth layer includes a convolution block, which consists of 2 convolution kernels of size 1×1 with a stride of 1 and padding of 1.

[0037] In the step (2.4), the feature map output by the last layer of the encoding part is upsampled and spliced ​​with the feature map output by the first layer of the decoding part, the spliced ​​feature map is upsampled and subjected to a 3×3×256 convolution and spliced ​​with the feature map output by the second layer of the decoding part, the spliced ​​feature map is upsampled and subjected to a 3×3×128 convolution and spliced ​​with the feature map output by the third layer of the decoding part, the spliced ​​feature map is upsampled and subjected to a 3×3×64 convolution and spliced ​​with the feature map output by the fourth layer of the decoding part, the spliced ​​feature map is subjected to a 3×3×32 convolution and then to a 1×1 convolution kernel.

[0038] The specific process of step (4) is as follows:

[0039] (4.1) 80% of the data set in step (1) is used as a training set and 10% as a validation set to be input into the improved U-Net model for training. Through supervised learning with labeled data, the gradient descent algorithm is used to fine-tune the entire network parameters, minimize the composite loss function, and obtain the optimal parameter model;

[0040] (4.2) Input 10% of the data set in step (1) as a test set into the optimal parameter model in (4.1) for testing, and output a prediction effect graph;

[0041] (4.3) Compare the predicted effect graph in (4.2) with the label graph to obtain the improved U-Net comparison output result.

[0042] Beneficial effects: By changing the convolution mode of U-Net, adding an efficient channel attention mechanism, and adding a multi-scale feature fusion method, richer and more accurate feature information can be extracted, making the segmentation results and generalization effects more accurate; Weight Normalization is added between the convolution layer and the activation layer of U-Net instead of Batch Normalization to achieve normalization, which introduces less noise and faster processing speed; an improved composite loss function is proposed to strengthen the detection of cloud map boundaries during network training, effectively improving the accuracy of cloud map segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A cloud image segmentation flow chart provided for an embodiment;

[0044] Figure 2 An improved U-Net structure diagram based on multi-scale feature fusion provided in the embodiment;

[0045] Figure 3 A comparison chart of generalization experiments provided for the embodiments. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0047] Figure 1 The cloud image segmentation flow chart provided in the embodiment includes the following steps:

[0048] (1) Obtain the Singapore full sky image segmentation database, and preprocess the 1013 sky-cloud images and their corresponding binary labels contained in the Singapore full sky image segmentation database to obtain a dataset;

[0049] (2) An improved U-Net framework is constructed by changing the convolution mode, adding an efficient channel attention mechanism, modifying normalization, and adding multi-scale feature fusion;

[0050] (3) Combining the advantages of Dice loss function, cross entropy loss function, boundary loss function and Log_Cosh loss function, an improved composite loss function is proposed;

[0051] (4) Inputting the data set obtained in step (1) into the improved U-Net model for training and testing, obtaining the optimal parameter model, and outputting the predicted segmentation map;

[0052] In step (1), 1013 sky-cloud images and their corresponding binary labels in the Singapore sky image segmentation database are obtained, with a size of 600×600×3. The cloud images are randomly divided into samples of 224×224×3 size, and data enhancement methods such as random pruning, translation transformation, and noise perturbation are used on the sample set to expand the data set to ten times the original size, thereby expanding the diversity of the existing data.

[0053] In step (2), this embodiment replaces the first convolution of the first two layers and the third layer of the encoding part with deformable convolution, that is, deformable convolution is used in the shallow neural network of the encoding part. The deformable convolution is dynamically adjusted according to the cloud image content that needs to be recognized at present, so as to more accurately capture the detailed information of the shallow features. The deformable convolution is mainly composed of offset convolution and standard convolution. The standard convolution kernel size used in the embodiment is 3×3. For an input feature map, in order to learn the offset offset, another offset convolution kernel size of 3×3 is defined. The output is the same size as the original feature map, and the number of channels is 2N. These two convolution kernels are learned simultaneously through the bilinear interpolation back-propagation algorithm.

[0054] This embodiment adds an efficient channel attention mechanism after the feature map output by the first three layers of the encoding part, so that the model automatically learns the important details in the shallow feature map. After the feature map output by the encoding part generates a one-dimensional attention vector through the efficient channel attention mechanism, the corresponding elements are multiplied with the original feature map, that is, the significant features are enhanced, and the non-significant features are suppressed accordingly. The feature map after feature recalibration is output, and the size of the feature map remains unchanged. The weighted feature map of the first three layers is downsampled, and the weighted feature map of the first three layers is spliced ​​with the feature map of the decoding part.

[0055] This embodiment adds Weight Normalization between the convolution layer and the activation layer of U-Net to perform normalization processing, standardizes the parameters of the network, and adds it by rewriting the weights of the deep network, introducing less noise, and does not require additional space to store the mean and variance of the minibatch. It has a small time overhead, so it is faster. For a neuron in the network, the input is x, the output is y, and the calculation process is y = φ(ω*x+b), where ω is the weight connected to the neuron. The process of optimizing the network through the loss function and gradient descent is the process of solving the optimal ω. Decoupling the length and direction of ω can be expressed as Where g is a scalar whose magnitude is equal to the modulus of ω, is a unit vector with the same direction as ω, so the learning of ω in the training process is transformed into the learning of g and v, and the gradient of the loss function L with respect to g and v is expressed as:

[0056]

[0057]

[0058] Figure 2 An improved U-Net structure diagram based on multi-scale feature fusion provided in an embodiment. Figure 2 As shown in the figure, the dataset size of the input network is 224×224×3. The encoding part has five layers. The first two layers are composed of two variable convolution blocks deform-conv11, deform-conv12, efficient channel attention mechanism and maximum pooling module. The variable convolution block includes offset convolution kernel and convolution block. The convolution block includes 3×3 convolution kernel, WN and activation function Relu; the third layer is composed of variable convolution block, convolution block, efficient channel attention mechanism and maximum pooling module; the fourth layer is composed of two convolution blocks and maximum pooling module; the fifth layer is composed of two convolution blocks. The decoding part has four layers, all of which are composed of upsampling module, splicing module and two convolution blocks. A multi-scale feature fusion module is added to the improved U-Net to perform multi-scale feature fusion on the feature map output by the last layer of the encoding part and the feature map output by each layer of the decoding part. The feature map output by the last layer of the encoding part is upsampled and concatenated with the feature map output by the first layer of the decoding part. The concatenated feature map is upsampled and concatenated with the feature map output by the second layer of the decoding part through a 3×3×256 convolution. The concatenated feature map is upsampled and concatenated with the feature map output by the third layer of the decoding part through a 3×3×128 convolution. The concatenated feature map is upsampled and concatenated with the feature map output by the fourth layer of the decoding part through a 3×3×64 convolution. The concatenated feature map is concatenated with a 3×3×32 convolution and then classified through a 1×1 convolution kernel.

[0059] The 224×224×3 feature map is input to the first layer of the encoding part, and after two variable convolution blocks deform-conv11, a 224×224×32 feature map is output. After the efficient channel attention mechanism, the feature is recalibrated to output a 224×224×32 feature map, and after the pooling layer Down1, a 112×112×32 feature map is output; the 112×112×32 feature map is input to the second layer of the encoding part, and after two variable convolution blocks deform-conv12, a 112×112×64 feature map is output. After the efficient channel attention mechanism, the feature is recalibrated to output a 112×112×64 feature map, and after the pooling layer Down 2 outputs a 56×56×64 feature map; the 56×56×64 feature map is input to the third layer of the encoding part, and after the deformable convolution block deform-conv13, a 56×56×128 feature map is output; after the convolution block conv13, a 56×56×128 feature map is output; after the efficient channel attention mechanism, feature recalibration is performed to output a 56×56×128 feature map; after the pooling layer Down3, a 28×28×128 feature map is output; the 28×28×128 feature map is input to the fourth layer of the encoding part, after two convolution blocks conv14, a 28×28×256 feature map is output; after the pooling layer Down4, a 14×14× 256 feature map; the 14×14×256 feature map is input to the fifth layer of the encoding part, and after two convolution blocks conv15, a 14×14×512 feature map is output; the 14×14×512 feature map is input to the first layer of the decoding part, and after upsampling Up14, a 28×28×512 feature map is output. After the concatenation operation Concat14 connects the feature maps output by Up14 and conv14 to obtain a 28×28×768 feature map, and after two convolution blocks conv24, a 28×28×256 feature map is output; the 28×28×256 feature map is input to the second layer of the decoding part, and after upsampling Up13, a 56×56×25 6 feature map, after the concatenation operation Concat13 connects Up13 and conv13 and the feature map output by the efficient channel attention mechanism, a feature map of 56×56×384 is obtained, and after two convolution blocks conv23, a feature map of 56×56×128 is output; the feature map of 56×56×128 is input to the third layer of the decoding part, and after upsampling Up12, a feature map of 112×112×128 is output, and after the concatenation operation Concat12 connects Up12 and conv12 and the feature map output by the efficient channel attention mechanism is obtained. A feature map of 112×112×192 is obtained, and after two convolution blocks conv22, a feature map of 112×112×64 is output;The 112×112×64 feature map is input to the fourth layer of the decoding part, and after upsampling Up11, a 224×224×64 feature map is output. After the concatenation operation Concat11, Up11 and conv11 are connected to output the feature map through efficient channel attention to obtain a 224×224×96 feature map. After two convolution blocks conv21, a 224×224×32 feature map is output. The feature map output from the last layer of the encoding part is upsampled Up21 to output a 28×28×512 feature map. After the concatenation operation Concat21 and the feature map output from the first layer of the decoding part, a 28×28×768 feature map is obtained. After upsampling Up22 and convolution conv31, a 56×56×256 feature map is obtained. After the concatenation operation Concat22 and the feature map output from the second layer of the decoding part, a 56×56×384 feature map is obtained. After upsampling Up23 and convolution conv32, a 112×112×128 feature map is obtained. After the concatenation operation Concat23 and the feature map output from the third layer of the decoding part, a 112×112×192 feature map is obtained. After upsampling Up24 and convolution conv33, a 224×224×64 feature map is obtained. After the concatenation operation Concat24 and the feature map output from the fourth layer of the decoding part, a 224×224×96 feature map is obtained. After convolution conv34, a 224×224×32 feature map is obtained. Finally, a 1×1 convolution is performed to classify the cloud map. ;

[0060] Step (3) is as follows. The formula of Dice loss function is as follows:

[0061]

[0062] Where A and B refer to the model prediction result and the true label result respectively, smooth is the smoothing factor, which is 0.00001 in this embodiment, N is the number of pixels in the feature map, and p is the i ∈[0,1] is the value of the i-th pixel in the segmentation map predicted by the model, g i ∈{0,1} is the value of the i-th pixel in the binary label. The Dice loss function considers all pixels in the feature map, including the cloud to be segmented and the background area, that is, the Dice loss function only focuses on global information. For some cloud maps with a large background area, the segmented cloud will be inaccurately positioned or seriously missing. In addition, the Dice loss function is a non-convex function, which will lead to unstable model training and may fall into a local optimum rather than a global optimum.

[0063] Since the Dice loss function is a non-convex function, adding a smoothing factor to make the function relatively smooth can only alleviate the problem of non-convex loss functions but cannot completely solve it. It is proposed to combine it with the Log_Cosh loss function. The formula of the Log_Cosh loss function is as follows:

[0064] L Log_Cosh =log(Cosh(x))

[0065] Among them, the hyperbolic cosine function It can be used to smooth curves, and in regression-based loss functions, the logarithm of the Cosh function is often taken, and its derivative is It can be seen from this that L Log_Cosh The loss function is continuous and second-order differentiable. Combining the Dice loss function with the Log_Cosh loss function, the function expression is:

[0066] L improve1 =log(Cosh(L Dice ))

[0067] Since the Dice loss function only focuses on global information, it is proposed to combine it with the cross entropy loss function. The formula of the cross entropy loss function is as follows:

[0068]

[0069] Where N is the number of pixels in the feature map, pi∈[0,1] is the value of the i-th pixel in the segmentation map predicted by the model, and gi∈{0,1} is the value of the i-th pixel in the binary label. The cross entropy loss function has the same weight for clouds and sky, so it is very sensitive to category imbalance, but it can make the function curve smooth, and the cross entropy loss function pays attention to local information. Combining the two effectively improves the segmentation accuracy, and the function expression is:

[0070] L improve2 =αL improve1 +(1-α)L CE

[0071] In this embodiment, the value of α is 0.5.

[0072] For L improve2 The combined Dice loss function and cross entropy loss function focus on global information and local information, and are proposed to be combined with the boundary loss function. The formula of the boundary loss function is as follows:

[0073]

[0074] in, ΔS=|S|-|S∩G|+|G|-|S∩G|, S is the pixel point on the boundary of the prediction result, G is the pixel point on the boundary of the true label, p is any pixel point on the boundary of the label, q is any pixel point within ΔS, D G (q) is the distance between two pixels. The purpose of the boundary loss function is to minimize the distance between the segmentation boundary and the label boundary, focusing on the boundary information. Combining the three effectively improves the segmentation accuracy, and the final improved loss function expression is:

[0075] L improve =βL improve2 +(1-β)L B

[0076] Among them, L B represents the boundary loss function. In this embodiment, the initial value of β is set to 1 and decreases by 0.01 after each round.

[0077] Step (4) specifically includes inputting the training data into the model for training, using labeled data supervised learning, using the gradient descent algorithm to fine-tune the entire network parameters, testing the best trained model weights with test data, and directly outputting the final prediction effect graph. Figure 3 A comparison chart of generalization experiments provided for the embodiments. Figure 3 As shown in the figure, the experiment selected four images with different sky-cloud distributions in the data set. In experiment 1, the clouds are mostly distributed in the lower left, and the background area is larger; in experiment 2, the clouds are mostly distributed in the upper right, and the background area is larger; in experiment 3, the clouds are more sparsely distributed, with more thin clouds; in experiment 4, the clouds are densely distributed, and the background area is smaller. By comparing the segmentation of four remote sensing images with different distributions, it can be seen that the improved U-Net has the best generalization effect. The details and edges in the cloud image are clearer than the generalization effects of other models, and it can better complete the cloud image segmentation task.

Claims

1. An improved U-Net cloud image segmentation method based on a multi-scale loss function, characterized in that: The following steps are involved: (1) obtaining a full sky image segmentation database, and preprocessing the cloud images and the binary labels corresponding to the cloud images contained in the full sky image segmentation database to obtain a data set; (2) An improved U-Net model is constructed by changing the convolution mode, adding an efficient channel attention mechanism, modifying normalization, and adding multi-scale feature fusion; The specific process of step (2) is as follows: (2.1) The method of changing the convolution mode is used to improve U-Net. Specifically, based on the U-Net segmentation model, the first two layers and the first convolution of the third layer in the encoding part are replaced with deformable convolutions; (2.2) The method of adding an efficient channel attention mechanism to improve U-Net is as follows: Based on the model (2.1), a channel attention mechanism is added after the feature map output by the first three layers of the encoding part. After the feature map output by the encoding part generates a one-dimensional attention vector through the efficient channel attention mechanism, the corresponding elements are multiplied with the original feature map to obtain a weighted feature map. The size of the feature map remains unchanged. The weighted feature map of the first three layers is downsampled, and the weighted feature map of the first three layers is concatenated with the feature map of the decoding part. (2.3) Using the modified normalization method to improve U-Net, specifically: Based on the model (2.2), WeightNormalization is added between the convolutional layer and the activation layer of U-Net, the weights of the deep network are rewritten, and the normalization operation is achieved by standardizing the network parameters; (2.4) The method of adding multi-scale feature fusion is used to improve U-Net. Specifically, based on the model (2.3), multi-scale feature fusion is performed on the feature map output by the last layer of the encoding part and the feature map output by each layer of the decoding part; (3) An improved composite loss function of the improved U-Net model is proposed, which is expressed as: L improve =βL improve2 +(1-β)L B Among them, L B Represents the boundary loss function, and the formula of the boundary loss function is as follows: in, △S=|S|-|S∩G|+|G|-|S∩G|, S is the pixel point on the boundary of the prediction result, G is the pixel point on the boundary of the true label, p is any pixel point on the boundary of the label, q is any pixel point within △S, D G (q) is the distance between two pixels; The initial value of β is set to 1 and decreases by 0.01 in each round of training; L improve2 It is a function that combines the Dice loss function and the cross entropy loss function. The expression is: L improve2 =αL improve1 +(1-α)L CE Where α is a hyperparameter, L CE is the cross entropy loss function, and the formula is as follows: Among them, N is the number of pixels in the feature map, p i ∈[0,1] is the value of the i-th pixel in the segmentation map predicted by the model, g i ∈{0,1} is the value of the i-th pixel in the binary label; L improve1 It is a function that combines the Dice loss function and the Log_Cosh loss function. The expression is: L improve1 =log(Cosh(L Dice )) The formula of the Log_Cosh loss function is as follows: L Log_Cosh =log(Cosh(x)) Among them, L Dice is the Dice loss function, expressed as follows: Among them, A and B refer to the model prediction results and the true label results respectively, and smooth is the smoothing factor; (4) Input the data set obtained in step (1) into the improved U-Net model for training and testing, obtain the optimal parameter model, and output the prediction effect diagram.

2. The improved U-Net cloud image segmentation method based on multi-scale loss function according to claim 1, characterized in that: The specific process of step (1) is as follows: (1.1) Obtain the sky-cloud image and its corresponding binary label contained in the full sky image segmentation database; (1.2) The training samples are randomly divided into small samples, and the data enhancement method is used on the sample set to expand the data set ten times the original size and divide it into three parts, with the training set accounting for 80%, the validation set accounting for 10%, and the test set accounting for 10%.

3. The improved U-Net cloud image segmentation method based on multi-scale loss function according to claim 2, characterized in that: In the step (1.2), the data enhancement method includes random pruning, translation transformation and noise perturbation.

4. The improved U-Net cloud image segmentation method based on multi-scale loss function according to claim 1, characterized in that: The encoding part in (2.1) is 5 layers: the first layer includes 2 identical variable convolution blocks deform-conv11 and deform-conv12, and the variable convolution block is mainly composed of two convolution blocks, one of which is composed of 18 offset convolution kernels with a step size of 1, a padding of 1, and a size of 3×3, and the other convolution block is composed of 32 standard convolution kernels conv11 and conv12 with a step size of 1, a padding of 1, and a size of 3×3; the second layer includes 2 identical variable convolution blocks deform-conv21 and deform-conv22, and the variable convolution block is mainly composed of two convolution blocks, one of which is composed of 18 offset convolution kernels with a step size of 1, a padding of 1, and a size of 3×3, and the other convolution block is composed of 64 standard convolution kernels conv21 and conv22 with a step size of 1, a padding of 1, and a size of 3×3; the third layer includes 2 convolution blocks deform-conv3 and conv3, where con v3 consists of 128 standard convolution kernels with a stride of 1, padding of 1, and a size of 3×3. deform-conv3 consists of 18 offset convolutions with a stride of 1, padding of 1, and a size of 3×3 and conv3. The fourth layer includes two identical convolution blocks conv4, each consisting of 256 standard convolution kernels with a stride of 1, padding of 1, and a size of 3×3. The fifth layer includes two identical convolution blocks conv5, each consisting of 512 stride of 1, padding of 1, and a standard convolution kernel of size 3×3; the decoding part of this model consists of 4 layers: each of them is composed of an upsampling module and two identical standard convolution modules corresponding to the encoding part. The number of convolution kernels in each module of each layer is (512, 256, 256), (256, 128, 128), (128, 64, 64), (64, 32, 32), and the kernel size is 3×3. At the same time, the fourth layer includes a convolution block, which consists of 2 convolution kernels of size 1×1 with a stride of 1 and padding of 1.

5. The improved U-Net cloud image segmentation method based on multi-scale loss function according to claim 1, characterized in that: In the step (2.4), the feature map output by the last layer of the encoding part is upsampled and spliced ​​with the feature map output by the first layer of the decoding part, the spliced ​​feature map is upsampled and subjected to a 3×3×256 convolution and spliced ​​with the feature map output by the second layer of the decoding part, the spliced ​​feature map is upsampled and subjected to a 3×3×128 convolution and spliced ​​with the feature map output by the third layer of the decoding part, the spliced ​​feature map is upsampled and subjected to a 3×3×64 convolution and spliced ​​with the feature map output by the fourth layer of the decoding part, the spliced ​​feature map is subjected to a 3×3×32 convolution and then to a 1×1 convolution kernel.

6. The improved U-Net cloud image segmentation method based on multi-scale loss function according to claim 1, characterized in that: The specific process of step (4) is as follows: (4.1) 80% of the data set in step (1) is used as a training set and 10% as a validation set to be input into the improved U-Net model for training. Through supervised learning with labeled data, the gradient descent algorithm is used to fine-tune the entire network parameters, minimize the composite loss function, and obtain the optimal parameter model; (4.2) Input 10% of the data set in step (1) as a test set into the optimal parameter model in (4.1) for testing, and output a prediction effect graph; (4.3) Compare the predicted effect graph in (4.2) with the label graph to obtain the improved U-Net comparison output result.

Images