Cloud detection method, device, electronic equipment and medium based on remote sensing image

Abstract

The present invention provides a cloud detection method, device, electronic device and medium based on remote sensing images, the method comprising: performing preprocessing based on remote sensing images to obtain a target image; inputting the target image into a cloud detection model, and outputting a cloud detection result corresponding to the target image. The cloud detection method, device, electronic device and medium based on remote sensing images provided by the present invention can increase the receptive field of convolution layer by layer without losing information, so that the output of each convolution contains a larger range of semantic information, and can improve the refinement of image semantic segmentation. This can further improve the accuracy of cloud detection.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a cloud detection method, device, electronic equipment and medium based on remote sensing images. Background Art

[0002] The cloud detection task requires assigning pixels of remote sensing images to certain categories, such as cloud and non-cloud, and accurately predicting the spatial location information of clouds and non-clouds to achieve a pixel-level understanding of the remote sensing image content. Cloud detection is of great significance to many fields, such as weather forecasting, energy exploration, precision agriculture, and natural disaster system monitoring.

[0003] In traditional cloud detection methods, the classic Fmask algorithm only considers the characteristics of the pixel value of the remote sensing image itself, without considering the spatial information characteristics, and is more sensitive to image noise. In recent years, with the increase in computer computing power and the continuous development of deep learning, some traditional segmentation methods can no longer be compared with segmentation methods based on deep learning in terms of effect. FCN fully convolutional neural network can accept image input of any size, and uses deconvolution to upsample the last convolution layer to restore it to the same size of the input image, so that the FCN network can classify each pixel of the image while retaining the spatial information of the input image. The characteristic of the FCN network is that it uses feature fusion to restore image semantic information. The network adds the feature map of the last layer to the shallow feature map of the network point by point, so that local prediction can be made while predicting the image globally, and it has richer semantic information. Although the FCN network uses feature map accumulation to supplement image details, the final result map is still not fine enough, the deconvolution result is relatively fuzzy and smooth, and is not sensitive to the details in the image. The cloud recognition accuracy of different types of remote sensing images varies greatly. Summary of the invention

[0004] The present invention provides a cloud detection method, device, electronic equipment and medium based on remote sensing images, which are used to solve the defect of poor cloud layer recognition accuracy in the prior art and improve the accuracy of cloud detection.

[0005] The present invention provides a cloud detection method based on remote sensing images, comprising:

[0006] Perform preprocessing based on remote sensing images to obtain target images;

[0007] Inputting the target image into a cloud detection model, and outputting a cloud detection result corresponding to the target image;

[0008] Wherein, the cloud detection model includes:

[0009] A multi-scale downsampling layer, used to perform downsampling operations and convolution operations of different scales based on the target image to obtain first feature images of different scales;

[0010] A first fusion layer is used to perform feature fusion based on the first feature images of different scales to obtain a first fused image;

[0011] A multi-scale upsampling layer, used for performing upsampling operations and convolution operations of different scales based on the first fused image to obtain second feature images of different scales;

[0012] A second fusion layer is used to perform feature fusion based on the first feature image and the second feature image corresponding to the scale to obtain a second fused image;

[0013] an output layer, used for obtaining a cloud detection result corresponding to the target image based on the second fused image;

[0014] The multi-scale downsampling layer includes a plurality of cascaded levels of downsampling sublayers, the multi-scale upsampling layer includes a plurality of cascaded levels of upsampling sublayers, and the number of the downsampling sublayers is the same as the number of the upsampling sublayers.

[0015] According to a cloud detection method for remote sensing images provided by the present invention, the remote sensing image is preprocessed to obtain a target image, including:

[0016] Based on a preset formula, the brightness temperature value and reflectivity of each pixel of the remote sensing image are obtained;

[0017] Based on the brightness temperature value and reflectivity of each pixel point, a target image is acquired.

[0018] According to a cloud detection method for remote sensing images provided by the present invention, the step of performing feature fusion based on the first feature images of different scales to obtain a first fused image includes:

[0019] Based on the first feature images of each scale, a corresponding pooling operation is performed to obtain first images of the same two-dimensional scale;

[0020] Based on each of the first images, a fusion operation and a convolution dimension reduction operation are performed to obtain the first fused image.

[0021] According to a cloud detection method for remote sensing images provided by the present invention, performing upsampling operations and convolution operations of different scales based on the first fused image to obtain second feature images of different scales includes:

[0022] Based on the first fused image, performing upsampling operations layer by layer to obtain second images of different two-dimensional scales;

[0023] Based on each of the second images, a corresponding convolution operation is performed to obtain second feature images of different scales.

[0024] According to a cloud detection method for remote sensing images provided by the present invention, the first feature image and the second feature image corresponding to the scale are subjected to feature fusion to obtain a second fused image, including:

[0025] Based on the first feature image and the second feature image of the same scale, a splicing operation and an upsampling operation are performed to obtain third images;

[0026] Based on each of the third images, feature fusion is performed to obtain the second fused image.

[0027] According to a cloud detection method for remote sensing images provided by the present invention, the upsampling operation includes an upsampling operation based on a bilinear interpolation method.

[0028] The present invention also provides a cloud detection device for remote sensing images, comprising:

[0029] A preprocessing module is used to perform preprocessing based on remote sensing images to obtain target images;

[0030] A detection module, used to input the target image into a cloud detection model and output a cloud detection result corresponding to the target image;

[0031] Wherein, the cloud detection model includes:

[0032] A multi-scale downsampling layer, used to perform downsampling operations and convolution operations of different scales based on the target image to obtain first feature images of different scales;

[0033] A first fusion layer is used to perform feature fusion based on the first feature images of different scales to obtain a first fused image;

[0034] A multi-scale upsampling layer, used for performing upsampling operations and convolution operations of different scales based on the first fused image to obtain second feature images of different scales;

[0035] A second fusion layer is used to perform feature fusion based on the first feature image and the second feature image corresponding to the scale to obtain a second fused image;

[0036] an output layer, used for obtaining a cloud detection result corresponding to the target image based on the second fused image;

[0037] The multi-scale downsampling layer includes a plurality of cascaded levels of downsampling sublayers, the multi-scale upsampling layer includes a plurality of cascaded levels of upsampling sublayers, and the number of the downsampling sublayers is the same as the number of the upsampling sublayers.

[0038] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the cloud detection method for remote sensing images as described in any one of the above are implemented.

[0039] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the cloud detection methods for remote sensing images described above.

[0040] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any one of the cloud detection methods for remote sensing images described above are implemented.

[0041] The cloud detection method, device, electronic device and medium based on remote sensing images provided by the present invention obtain target images based on remote sensing images with multiple channels, extract first feature images of multiple scales through cascaded multiple levels of downsampling sublayers, obtain first fused images through feature fusion of first feature images of multiple scales, restore the first fused images into second feature images of multiple scales through cascaded multiple levels of upsampling sublayers, obtain second fused images after splicing and fusing first feature images of the same scale with the first feature images, and restore cloud detection results through the second fused images. The receptive field of convolution can be increased layer by layer without losing information, so that the output of each convolution contains semantic information in a larger range, which can improve the refinement of image semantic segmentation and thus improve the accuracy of cloud detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a schematic diagram of the flow of the cloud detection method based on remote sensing images provided by the present invention;

[0044] Figure 2 It is a structural schematic diagram of a cloud detection model based on remote sensing images provided by the present invention;

[0045] Figure 3It is a structural schematic diagram of a cloud detection device based on remote sensing images provided by the present invention;

[0046] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Figure 1 FIG. 1 is a flow chart of a cloud detection method based on remote sensing images provided by the present invention. Figure 1 As shown, Figure 1 As shown, the cloud detection method based on remote sensing images provided by an embodiment of the present invention includes: step 101, preprocessing based on the remote sensing image to obtain a target image.

[0049] It should be noted that the execution subject of the cloud detection method based on remote sensing images provided in the embodiment of the present invention is a cloud detection device based on remote sensing images. The detection object of the cloud detection device based on remote sensing images is the remote sensing image.

[0050] Remote sensing images refer to electromagnetic spectrum information of objects in different bands obtained through remote sensing. Remote sensing images can be segmented according to different bands to generate two-dimensional arrays of different channels to represent the information of objects in different bands. The pixel values ​​in the two-dimensional array are called brightness values ​​(or grayscale values, DN values).

[0051] Specifically, in step 101, the cloud detection device based on the remote sensing image performs corresponding preprocessing on the remote sensing image according to the set requirements to obtain the target image.

[0052] The target image refers to an image that is as close to the real image as possible in terms of geometry and radiometry compared to the original remote sensing image. The target image has at least three channels, and the resolution and width and height of the grayscale images corresponding to different channels are different. The target image is used to input into the cloud detection model to obtain the corresponding cloud detection results.

[0053] Preprocessing refers to the processing performed on the input original remote sensing image before applying the cloud detection model. Preprocessing is used to eliminate distorted information in the remote sensing image so that the generated target image contains useful real information, enhances the detectability of relevant information and simplifies data to the maximum extent. The embodiments of the present invention do not specifically limit preprocessing.

[0054] Exemplarily, the preprocessing may be radiation correction, in which a cloud detection device based on remote sensing images respectively performs remote sensor calibration, atmospheric correction (or solar altitude correction) and terrain correction on three types of radiation distortions in the remote sensing images caused by the characteristics of the remote sensing sensor itself, the lighting conditions of the objects (the influence of terrain and the influence of solar altitude angle) and the effects of the atmosphere, so as to obtain a target image.

[0055] Exemplarily, the preprocessing can also be geometrically corrected. The cloud detection device based on remote sensing images needs to select control point pairs in the remote sensing image to be corrected and the standard map with coordinate information, perform pixel coordinate transformation, and resample the brightness value of the pixel to obtain the target image.

[0056] Preferably, the cloud detection device based on remote sensing images uses different channel data for corresponding preprocessing according to the characteristics of the cloud layer in different scenes, generates a target image, and inputs it into the cloud detection model to detect whether there are clouds in the remote sensing image.

[0057] Step 102: Input the target image into the cloud detection model, and output the cloud detection result corresponding to the target image.

[0058] Among them, the cloud detection model includes:

[0059] The multi-scale downsampling layer is used to perform downsampling operations and convolution operations at different scales based on the target image to obtain first feature images at different scales.

[0060] The first fusion layer is used to perform feature fusion based on first feature images of different scales to obtain a first fused image.

[0061] The multi-scale upsampling layer is used to perform upsampling operations and convolution operations of different scales based on the first fused image to obtain second feature images of different scales.

[0062] The second fusion layer is used to perform feature fusion based on the first feature image and the second feature image corresponding to the scale to obtain a second fused image.

[0063] The output layer is used to obtain the cloud detection result corresponding to the target image based on the second fused image.

[0064] The multi-scale downsampling layer includes a plurality of cascaded levels of downsampling sublayers, the multi-scale upsampling layer includes a plurality of cascaded levels of upsampling sublayers, and the number of the downsampling sublayers is the same as the number of the upsampling sublayers.

[0065] It should be noted that the cloud detection model is obtained after training based on remote sensing image sample data and predetermined remote sensing image type labels.

[0066] The cloud detection model may be an artificial intelligence model, and the embodiment of the present invention does not specifically limit the model type.

[0067] For example, the cloud detection model can be a deep convolutional neural network (DCNN) model. As a data-driven algorithm, DCNN relies on its stacked sampling layers and convolution layers to extract hierarchical features from data, and uses the back propagation algorithm to iteratively optimize the model parameters. The structure and parameters of the DCNN model include but are not limited to the number of layers of the model and the weight parameters of each layer.

[0068] It should be noted that the sample data includes remote sensing image data and remote sensing image type labels corresponding to the sample data. The remote sensing image sample data is divided into a training set and a test set, and the embodiment of the present invention does not specifically limit the sample ratio of the training set and the test set.

[0069] For example, the sample data is 12,000 satellite payload simulation data, and the data is divided into 3:1, 9,000 data are used as a training set for model training, and 3,000 data are used as a test set for model verification. The embodiment of the present invention does not specifically limit the training of the cloud detection model.

[0070] For example, the model can be trained directly using the training set and the test set can be used to obtain the detection results.

[0071] For example, according to the different distribution scenes of remote sensing data, the remote sensing data of the ocean area between the latitude of 60°S and 60°N in the test set can be divided into ocean scene test data, the remote sensing data of the land area between the latitude of 60°S and 60°N can be divided into land scene test data, and the remote sensing data of the polar area between the latitude of 60°N and 90°N and 60°S and 90°S can be divided into polar scene test data. Among them, the daytime scene is the solar zenith angle less than 85 degrees.

[0072] The training set and test set are divided into four types and applied to the cloud detection model to derive four types of cloud detection models, namely, the all-terrain all-day infrared model, the ocean daytime model, the land daytime model, and the polar all-day infrared model. To simplify the model, all models are binary classification models, and the value of each pixel is 0 or 1, 0 represents the pixel as background, and 1 represents the pixel as cloud. That is, the true value mask is a binary result: "cloud", "not cloud". When using the cloud detection model in a specific scenario, the model can be trained with the training set of the corresponding scenario to make the model more targeted.

[0073] It should be noted that in the cloud detection model training process, the cloud detection device based on remote sensing images uses the Pytorch framework and uses the torch.from_numpy function to process the image format of the input target image into batchsize*height*width*channel, where height is the height of the target image, width is the width of the target image, and channel is the number of channels of the target image. In addition, in one training, batchsize training samples are selected and input into the model, and then their gradients are calculated for back propagation to improve memory utilization.

[0074] Specifically, in step 102, the cloud detection device based on remote sensing images sets the cloud detection model according to the trained model parameters, and then uses the model to perform cloud detection on any target image such as in step 101 to obtain a cloud detection result corresponding to the target image.

[0075] The embodiment of the present invention does not specifically limit the form of the behavior detection result.

[0076] For example, the cloud detection result can be a two-dimensional array with the same width and height as the target image. The value range of any value in the array is [0, 1], where 0 means there is no cloud at the pixel and 1 means there is cloud at the pixel. In addition, the position information of the pixel in the two-dimensional array can be used to determine which areas in the image are clouds.

[0077] For example, the cloud detection result can be a statistical value, and the statistical value of all pixel points can be used to indicate whether the target image is "cloudy" or "cloudless".

[0078] The embodiment of the present invention does not specifically limit the cloud detection model.

[0079] Exemplarily, a cloud detection device based on remote sensing images uses a cloud detection model to detect whether there are clouds in the remote sensing images. The model is composed of at least an input layer, a hidden layer, and an output layer.

[0080] The input layer is at the front end of the entire network and directly receives the target image generated in step 101.

[0081] The hidden layer contains at least four layers, namely, the multi-scale downsampling layer, the first fusion layer, the multi-scale upsampling layer, and the second fusion layer. The function of the hidden layer is to downsample the target image layer by layer to obtain feature maps of remote sensing semantic information at different levels. The feature maps of each level are spliced ​​in the channel dimension to obtain a multi-level feature fusion image. The multi-level feature fusion image is upsampled to restore the spatial information of the image layer by layer. The feature maps of each level extracted during the downsampling process and the feature maps of each level restored during the upsampling process are spliced ​​to re-extract features.

[0082] The output layer is the last layer. The function of the output layer is to use a convolution kernel with a size of 1x1 and a number of 1 to perform full connection dimensionality reduction to obtain the width and height dimensions and the original Figure 1 The feature map is a grayscale image with a channel dimension of 1, which is the final cloud detection result.

[0083] Figure 2 Schematic diagram of the structure of the cloud detection model based on remote sensing images provided by the present invention. Figure 2 As shown, the hidden layers of the cloud detection model may include:

[0084] The multi-scale downsampling layer includes at least 4 cascaded downsampling sublayers, each of which is followed by a pooling layer, and each pooling layer is connected to the first fusion layer. After being processed by the first fusion layer, the result is input to the multi-scale upsampling layer, which includes at least 4 cascaded upsampling sublayers, each of which is followed by a deconvolution layer. At the same time, there are four skip connection chains, downsampling sublayer 4 is connected to upsampling sublayer 1, downsampling sublayer 3 is connected to upsampling sublayer 2, downsampling sublayer 2 is connected to upsampling sublayer 3, and downsampling sublayer 1 is connected to upsampling sublayer 4. The splicing result of each skip connection chain is sent to the second fusion layer, and the second fusion layer is connected to the output layer (i.e., the fully connected layer), and the activation function is the ReLU function.

[0085] The function of the multi-scale downsampling layer is to extract high-dimensional feature maps from the matrix corresponding to the remote sensing image through a large number of convolution kernels. Multiple feature maps can be extracted, each of which is a local perception extracted from the picture. Combining these feature maps can extract the part of interest in the picture. The convolution kernel size of the four downsampling sublayers used in the embodiment of the present invention is 1*1, and the number is 64, 128, 256 and 512 respectively. The pooling operation is used for downsampling, and the pooling window size is 2. In each layer, convolution is performed first, and then pooling is performed, so that the height and width of the feature map of the image are shortened by half after each layer of processing, and the number of channels is doubled. It can effectively extract the feature information of the picture, reduce the dimension of the feature, compress the number of data and parameters, and reduce training overfitting.

[0086] The function of the first fusion layer is to pool all the feature maps output by the multi-scale downsampling layer to reduce their feature size. The pooling window sizes are 16, 8, 4, and 2, respectively, so that the height and width of all feature maps are the same, and multi-level features are fused according to the channel dimension to obtain the fused feature map.

[0087] The function of the multi-scale upsampling layer is to restore the image from the fused feature map. That is, the fused feature map is upsampled, the width and height of the image are enlarged to twice the original size, and then the deconvolution operation is performed to restore the spatial information of the image (i.e., channel dimension) layer by layer.

[0088] The function of the second fusion layer is to jump-join the feature maps of each upsampling sub-layer with the feature maps of the corresponding downsampling sub-layer, and then use a convolution kernel of size 3x3 to re-extract features. Then, all feature maps are upsampled by multiples of 2, 4, 8, and 16, and the width and height are restored to the same size as the width and height of the target image, and then feature fusion is performed to generate a fused image.

[0089] In a multi-layer neural network, the role of the activation function ReLU is to perform nonlinear mapping on the feature map obtained by convolution in the output layer. Because after multiple layers of convolution, the value of the feature vector map does not change much during training, which will cause the gradient to disappear, and then make the convolution network model untrainable. Using the ReLU activation function after each convolution can perform nonlinear mapping on the results of the linear calculation of the convolution layer, so that the changes in the feature map before and after training will not be too small, and the model can be trained.

[0090] It is understandable that the embodiment of the present invention does not specifically describe the loss function used in the cloud detection model training process.

[0091] Exemplarily, the loss function uses the binary_crossentropy cross entropy loss function, and the calculation formula is as follows:

[0092]

[0093] Among them, i represents the position of a pixel in the cloud detection result, y′ i Represents the predicted pixel value of the i-th point, y i Represents the true pixel value of the i-th point.

[0094] Exemplarily, the Adam function is used as the optimization function, and the first-order gradient optimization is performed on the loss function to reduce the size of the loss function value.

[0095] It is understandable that after step 102, the cloud detection model can be evaluated using the image segmentation evaluation index mIoU. The calculation formula is as follows:

[0096]

[0097] Among them, FN is false negative, indicating that the true value of the target image is 1, and the true value of the cloud detection result is 0; TP is true positive, indicating that the true value of the target image is 1, and the true value of the cloud detection result is 1; FP is false positive, indicating that the true value of the target image is 0, and the true value of the cloud detection result is 1.

[0098] Compared with conventional CNN convolutional neural networks, the cloud detection model in the cloud detection method based on remote sensing images has a higher mIoU score. At the same time, the network with Resnet34 as the backbone has a very fast calculation speed and higher accuracy than the structure without residual network.

[0099] The embodiment of the present invention obtains a target image based on a remote sensing image with multiple channels, extracts first feature images of multiple scales through cascaded multiple levels of downsampling sublayers, obtains a first fused image through feature fusion of the first feature images of multiple scales, restores the first fused image into a second feature image of multiple scales through cascaded multiple levels of upsampling sublayers, splices and fuses the first feature image of the same scale with the first feature image to obtain a second fused image, and restores the cloud detection result through the second fused image. The receptive field of the convolution can be increased layer by layer without losing information, so that the output of each convolution contains a larger range of semantic information, which can improve the refinement of image semantic segmentation and thus improve the accuracy of cloud detection.

[0100] On the basis of any of the above embodiments, preprocessing is performed based on the remote sensing image to obtain the target image, including: obtaining the brightness temperature value and reflectivity of each pixel point of the remote sensing image based on a preset formula.

[0101] It should be noted that reflectivity refers to the percentage of reflected energy to incident energy. Different objects have different reflectivities, which mainly depend on their own properties, as well as the wavelength and angle of incidence of the incident electromagnetic wave. The reflectivity range is always less than or equal to 1. Reflectivity is used to characterize the properties of objects in the visible light band.

[0102] Brightness temperature refers to the physical temperature of a black body when the radiation brightness of an object is equal to that of the black body. Brightness temperature is used to characterize the radiation brightness of ground objects in the infrared band.

[0103] The preset formula refers to a calibration calculation formula that converts the grayscale value of each pixel in the remote sensing data into a reflectance or brightness temperature value.

[0104] Specifically, in step 101, the cloud detection device based on remote sensing images obtains multi-channel high-resolution original image data of remote sensing satellites, and performs calibration calculations according to the bands corresponding to different channels to convert the grayscale value of each pixel in the grayscale image of different channels into a brightness temperature value or reflectivity.

[0105] Among them, if the band corresponding to a certain channel is the visible light band, the cloud detection device based on the remote sensing image obtains the reflectivity corresponding to the pixel point according to the grayscale value of each pixel point in the grayscale image of the channel according to the calibration calculation formula.

[0106] If the band corresponding to a certain channel is the infrared band, the cloud detection device based on the remote sensing image obtains the brightness temperature value corresponding to the pixel point according to the calibration calculation formula for the grayscale value of each pixel point in the grayscale image of the channel.

[0107] Based on the brightness temperature and reflectivity of each pixel, the target image is obtained.

[0108] Specifically, in step 101, the cloud detection device based on remote sensing images updates each pixel value in the grayscale image corresponding to all target channels to the brightness temperature value or reflectivity corresponding to the target channel according to a preset channel selection rule, and merges all updated images into a target image.

[0109] A digital image can be represented in the form of a matrix. The pixel data of a grayscale image is a matrix. The rows of the matrix correspond to the height of the image (in pixels), the columns of the matrix correspond to the width of the image (in pixels), the elements of the matrix correspond to the pixels of the image, and the values ​​of the matrix elements are the grayscale values ​​of the pixels.

[0110] The target image can be represented in the form of a three-dimensional array, wherein the first dimension and the second dimension of the array correspond to the width and height of the target image respectively, the third dimension corresponds to the number of channels of the target image, and each element in the three-dimensional array corresponds to a pixel at a position corresponding to any set of width, height and number of channels in the target image. The number of channels of the target image is less than or equal to the number of channels of the remote sensing image, and the present invention does not specifically limit the number of channels of the target image.

[0111] Exemplarily, the cloud detection device based on remote sensing images can convert the grayscale images of 25 channels of the remote sensing image into corresponding brightness temperature images and reflectance images according to the calibration calculation process of the remote sensing image, and select 3 visible light channels, 1 near infrared channel, 6 mid-infrared and far infrared channels, a total of 10 channels. Since the resolutions corresponding to different channels are different, the width and height dimensions of different channels are not uniform, so it is necessary to perform data processing based on the grayscale value data of different channels, the calibration calculation formula and the spatial resolution to make the width and height dimensions of each channel in the target image uniform. The corresponding relationship of the relevant data of different channels is shown in Table 1.

[0112] Table 1 Channel data correspondence diagram

[0113]

[0114] If for channels 1-4, the cloud detection device based on remote sensing images can determine whether it belongs to the visible light band or the near-infrared band according to the corresponding wavelength range, and then calibrate and calculate the DN value of each pixel point of the image of each channel and convert it into reflectivity. And because the spatial resolution of channels 3 and 4 is twice that of channels 1 and 2, it is necessary to interpolate the reflectivity images corresponding to channels 3 and 4 so that their width and height dimensions are expanded to twice the original, so that the width and height dimensions of the reflectivity images of channels 3 and 4 are the same as those of the reflectivity images of channels 1 and 2.

[0115] Similarly, if for channels 20-25, the cloud detection device based on remote sensing images can determine that it belongs to the mid-infrared band or the far-infrared band according to the corresponding wavelength range, then the DN value of each pixel point of each channel image is calibrated and calculated to be converted into brightness temperature. And because the spatial resolution of channels 20-25 is 4 times that of channels 1 and 2, it is necessary to interpolate the brightness temperature value image corresponding to channels 20-25 so that its width and height are expanded to 4 times the original, so that the width and height of the reflectivity image of channels 20-25 are the same as those of the reflectivity images of channels 1 and 2.

[0116] The embodiment of the present invention is based on a remote sensing image with multiple channels, determines the selected channel through the corresponding relationship of channel data, and uses radiometric calibration calculation to update the original grayscale value of the corresponding channel to the corresponding brightness temperature value and reflectivity to obtain the target image. It can correct the errors in obtaining remote sensing images, improve the precision of image semantic segmentation, and thus improve the accuracy of cloud detection.

[0117] Based on any of the above embodiments, feature fusion is performed based on first feature images of different scales to obtain a first fused image, including: performing corresponding pooling operations based on first feature images of each scale to obtain first images of the same two-dimensional scale.

[0118] It should be noted that the first feature image refers to the feature image extracted after the target image is convolved and pooled. Different degrees of convolution and pooling correspond to different scales of the generated first feature image.

[0119] Exemplarily, a target image with an image size of 4*512*512*10 is input into the cloud detection model, and a cascaded downsampling sublayer and a corresponding connected convolutional layer are used, wherein the convolution kernel sizes are 64, 128, 256 and 512 respectively, and a pooling window of size 2 is used for downsampling, and the first feature image A1 with a scale of 4*256*256*64, the first feature image A2 with a scale of 4*128*128*128, the first feature image A3 with a scale of 4*64*64*256 and the first feature image A4 with a scale of 4*32*32*512 are obtained in sequence. A1, A2, A3 and A4 respectively contain semantic information of the target image at different levels.

[0120] Specifically, in step 102, the cloud detection device based on remote sensing images performs pooling operations on first feature images of different scales using corresponding pooling windows according to the correspondence between the scales and the pooling windows to generate a first image corresponding to the number of channels.

[0121] The first image refers to an image generated after the first feature image is pooled. First images corresponding to first feature images of different scales have the same width and height, and different numbers of channels, and different first images have the same number of channels as the corresponding first feature images. The embodiment of the present invention does not specifically limit the process of re-pooling.

[0122] Exemplarily, a cloud detection device based on remote sensing images pools the first feature images A1, A2, A3 and A4 to reduce their feature sizes. The pooling window sizes are 16, 8, 4, and 2, respectively, to obtain a first image B1 with a scale of 4*16*16*64, a first image B2 with a scale of 4*16*16*128, a first image B3 with a scale of 4*16*16*256, and a first image B4 with a scale of 4*16*16*512.

[0123] Based on each first image, a fusion operation and a convolution dimension reduction operation are performed to obtain a first fused image.

[0124] Specifically, in step 102, the cloud detection device based on remote sensing images performs feature fusion according to different channel dimensions of all first images to obtain a first fused image.

[0125] The first fused image refers to an image obtained by performing feature fusion on all first images according to the number of channels. The first fused image contains a collection of semantic information of the target image at different levels.

[0126] Preferably, the cloud detection device based on remote sensing images performs feature fusion on all first images and then performs convolution to reduce the channel dimension of the first fused image. The embodiment of the present invention does not specifically limit this process.

[0127] Exemplarily, the cloud detection device based on remote sensing images performs multi-level feature fusion on the first images B1, B2, B3 and B4 in the channel dimension to obtain a feature map with a size of 4*16*16*960. To prevent the feature dimension from being too high and causing overfitting, the fused feature map is subjected to another convolution dimensionality reduction operation with a convolution kernel size of 256 to obtain a first fused image B with a scale of 4*16*16*256.

[0128] The embodiment of the present invention performs pooling based on the first feature images of each scale to obtain a first image with uniform width and height, and obtains a first fused image after fusing and reducing the dimension of the first images with different numbers of channels. The image can contain both low-level detail features and high-level semantic features, which can improve the precision of image semantic segmentation and thus improve the accuracy of cloud detection.

[0129] On the basis of any of the above embodiments, upsampling operations and convolution operations of different scales are performed based on the first fused image to obtain second feature images of different scales, including: based on the first fused image, upsampling operations are performed layer by layer to obtain second images of different two-dimensional scales.

[0130] Specifically, in step 102, the cloud detection device based on remote sensing images performs an upsampling operation on the first fused image using a corresponding upsampling window according to the correspondence between the scale and the upsampling window, and restores the width and height of the first fused image to a second image having the same width and height as the first feature images A1, A2, A3 and A4, respectively.

[0131] The second image refers to an image generated by increasing the width and height of the first fused image B through layer-by-layer upsampling. The second images of different width and height scales have the same number of channels and the same width and height as the corresponding first image. The embodiment of the present invention does not specifically limit the upsampling process.

[0132] Exemplarily, the cloud detection device based on remote sensing images upsamples the width and height of the first fused image B layer by layer to twice the original size, and obtains a second image C1 with a scale of 4*32*32*256, a second image C2 with a scale of 4*64*64*256, a second image C3 with a scale of 4*128*128*256, and a second image C4 with a scale of 4*256*256*256.

[0133] The embodiment of the present invention does not specifically limit the upsampling method, and the upsampling method includes but is not limited to bilinear interpolation, trilinear interpolation, nearest neighbor interpolation, bicubic interpolation and the like.

[0134] Based on each second image, a corresponding convolution operation is performed to obtain second feature images of different scales.

[0135] It should be noted that the second feature image refers to the feature image restored after upsampling and deconvolution of the first fused image. Different degrees of upsampling and deconvolution correspond to different scales of the restored second feature image.

[0136] Specifically, the cloud detection device based on remote sensing images performs a deconvolution operation on the second images of different width and height sizes using corresponding convolution kernels according to the correspondence between the width and height sizes and the convolution kernels to generate a second feature image with a different number of channels.

[0137] The second characteristic image refers to a characteristic image restored by deconvolution of the second image. Different degrees of deconvolution result in different numbers of channels of the second characteristic image generated according to the second image.

[0138] Exemplarily, the cloud detection device based on remote sensing images performs deconvolution operations on the second images C1, C2, C3 and C4 according to convolution kernels of sizes 512, 256, 128 and 64, respectively, and restores the number of channels of the images layer by layer. The second image C1 is restored to obtain a second feature image D1 with a scale of 4*32*32*512, the second image C2 is restored to obtain a second feature image D2 with a scale of 4*64*64*256, the second image C3 is restored to obtain a second feature image D3 with a scale of 4*128*128*128, and the second image C4 is restored to obtain a second feature image D4 with a scale of 4*256*256*64.

[0139] The embodiment of the present invention performs upsampling and convolution at different scales based on the first fused image, and restores the scale of the first fused image to a second feature image having a scale corresponding to the first feature image. The rich features contained in the first fused image can be restored to different degrees, which can improve the refinement of image semantic segmentation and further improve the accuracy of cloud detection.

[0140] Based on any of the above embodiments, feature fusion is performed based on the first feature image and the second feature image of corresponding scales to obtain a second fused image, including: performing a stitching operation and an upsampling operation based on the first feature image and the second feature image of the same scale to obtain each third image.

[0141] It should be noted that the multiple downsampling sublayers cascaded from top to bottom in the multi-scale downsampling layer of the cloud detection model in step 102 are jump-linked to the multiple upsampling sublayers cascaded from bottom to top in the multi-scale upsampling layer, respectively, so that feature images with the same scale are integrated.

[0142] Specifically, the cloud detection device based on remote sensing images splices and fuses the first feature image and the second feature image corresponding to each skip connection chain according to the skip connection corresponding to each sublayer in the multi-scale downsampling layer and the multi-scale upsampling layer in the cloud detection model to obtain a third image.

[0143] The third image refers to an image that includes features extracted during the downsampling process and features restored during the upsampling process.

[0144] Preferably, the cloud detection device based on remote sensing images splices and fuses the first feature image and the second feature image corresponding to each jump connection chain according to the jump connection corresponding to each sub-layer in the multi-scale downsampling layer and the multi-scale upsampling layer in the cloud detection model, and then performs an upsampling operation, so that the width and height dimensions of each obtained third image are the same as the width and height dimensions of the target image. The embodiment of the present invention does not specifically limit this process.

[0145] Exemplarily, the cloud detection device based on remote sensing images splices the first feature image A4 and the second feature image D1 at both ends of the jump connection chain of the fourth layer to obtain a feature map with a scale of 4*32*32*1024 corresponding to the fourth layer, and then re-extracts the features through convolution with a 3x3 convolution kernel to obtain a third image E4 with a scale of 4*32*32*512, and then upsamples the scale of E4 to 4*515*512*512.

[0146] Similarly, the first feature image A1 and the second feature image D4 at both ends of the skip connection chain of the first layer are spliced ​​to obtain a third image E1 with a size of 4*515*512*64.

[0147] The first feature image A2 and the second feature image D3 at both ends of the skip connection chain of the second layer are spliced ​​to obtain a third image E2 with a size of 4*515*512*128.

[0148] The first feature image A3 and the second feature image D2 at both ends of the skip connection chain of the third layer are spliced ​​to obtain a third image E3 with a size of 4*515*512*256.

[0149] Based on each third image, feature fusion is performed to obtain a second fused image.

[0150] Specifically, in step 102, the cloud detection device based on remote sensing images performs feature fusion according to different channel dimensions of all third images to obtain a second fused image.

[0151] The second fused image refers to an image obtained by performing feature fusion on all third images according to the number of channels. The second fused image includes a collection of semantic information extracted from the target image at different levels and restored semantic information.

[0152] Exemplarily, the cloud detection device based on remote sensing images fuses the third images D1, D2, D3 and D4 through feature fusion to obtain a second fused image D with a scale of 4*512*512*960.

[0153] It is understandable that the cloud detection device based on remote sensing images inputs the second fused image D into the last layer of the cloud detection model to perform pixel-level detection. This embodiment of the present invention does not specifically limit this.

[0154] Exemplarily, the cloud detection device based on remote sensing images randomly discards 0.5 times the feature dimension of the second fused image D, and finally uses a convolution layer with a size of 1*1 and a convolution kernel size of 1 to reduce the dimension, and obtains a feature map of 4*512*512*1 with the same size as the original image, and each feature map is a grayscale image with a channel dimension of 1.

[0155] The value of any pixel in each grayscale image is compared with a preset threshold. If the value of the pixel is greater than or equal to the preset threshold, it means that the detection result of the pixel is "cloud", and the value of the pixel is updated to 1.

[0156] If the value of the pixel is less than the preset threshold, it means that the detection result of the pixel is “not a cloud”, and the value of the pixel is updated to 0.

[0157] Therefore, the cloud detection result output by the cloud detection model can be a cloud detection label image with an image size of 512x512x1, that is, the pixel value set is {0,1}. 0 represents that the point is not a cloud part, and 1 represents that the point is a cloud.

[0158] The embodiment of the present invention fuses the first feature image and the second feature image of the same level into a first fused image, and restores the width and height to the second fused image that is the same as the target image. It is possible to integrate features extracted and restored to different degrees, improve the precision of image semantic segmentation, and thus improve the accuracy of cloud detection.

[0159] Based on any of the above embodiments, the up-sampling operation includes an up-sampling operation based on a bilinear interpolation method.

[0160] Specifically, in step 102, the upsampling sublayers of multiple levels cascaded in the multi-scale upsampling layer, and the cloud detection device based on the remote sensing image use bilinear interpolation to upsample layer by layer, and gradually restore the width and height dimensions to the same as the target image.

[0161] For example, given a function f in Q 11 =(x1,y1),Q 12 =(x1,y2),Q21 =(x2, y1), and Q 22 = (x2, y2) four points, the pixel value at point P = (x, y) is required to be:

[0162] Performing linear interpolation in the x-direction of the coordinate axis, we get:

[0163]

[0164]

[0165] Performing linear interpolation in the y direction of the coordinate axis, we get:

[0166]

[0167] Get the size of the pixel value at point P = (x, y).

[0168] The embodiment of the present invention adopts a bilinear interpolation method to perform upsampling operations, which can restore the size of extracted features, thereby improving the accuracy of cloud detection and the precision of image segmentation.

[0169] Figure 3 It is a structural schematic diagram of a cloud detection device based on remote sensing images provided by the present invention.

[0170] like Figure 3 As shown, the device includes: a preprocessing module 310 and a detection module 320, wherein:

[0171] The preprocessing module 310 is used to perform preprocessing based on the remote sensing image to obtain a target image.

[0172] The detection module 320 is used to input the target image into the cloud detection model and output the cloud detection result corresponding to the target image.

[0173] Among them, the cloud detection model includes:

[0174] The multi-scale downsampling layer is used to perform downsampling operations and convolution operations at different scales based on the target image to obtain first feature images at different scales.

[0175] The first fusion layer is used to perform feature fusion based on first feature images of different scales to obtain a first fused image.

[0176] The multi-scale upsampling layer is used to perform upsampling operations and convolution operations of different scales based on the first fused image to obtain second feature images of different scales.

[0177] The second fusion layer is used to perform feature fusion based on the first feature image and the second feature image corresponding to the scale to obtain a second fused image.

[0178] The output layer is used to obtain the cloud detection result corresponding to the target image based on the second fused image.

[0179] The multi-scale downsampling layer includes a plurality of cascaded levels of downsampling sublayers, the multi-scale upsampling layer includes a plurality of cascaded levels of upsampling sublayers, and the number of the downsampling sublayers is the same as the number of the upsampling sublayers.

[0180] Specifically, the pre-processing module 310 and the detection module 320 are electrically connected in sequence.

[0181] The preprocessing module 310 updates each pixel value in the grayscale image corresponding to all target channels to the brightness temperature value or reflectivity corresponding to the target channel according to the preset channel selection rule, and merges all updated images into the target image.

[0182] After the detection module 320 sets the cloud detection model according to the trained model parameters, cloud detection is performed on any target image such as that in the preprocessing module 310 through the model to obtain a cloud detection result corresponding to the target image.

[0183] The preprocessing module 310 further includes a first acquisition unit and a target image acquisition unit, wherein:

[0184] The first acquisition unit is used to acquire the brightness temperature value and reflectivity of each pixel point of the remote sensing image based on a preset formula.

[0185] The target image acquisition unit is used to acquire the target image based on the brightness temperature value and reflectivity of each pixel point.

[0186] The detection module 320 further includes a first pooling unit and a first fusion unit, wherein:

[0187] The first pooling unit is used to perform corresponding pooling operations based on the first feature images of each scale to obtain first images with the same two-dimensional scale.

[0188] The first fusion unit is used to perform a fusion operation and a convolution dimension reduction operation based on the first images to obtain a first fused image.

[0189] The detection module 320 further includes an up-sampling unit and a second acquisition unit, wherein:

[0190] The upsampling unit is used to perform upsampling operations layer by layer based on the first fused image to obtain second images with different two-dimensional scales.

[0191] The second acquisition unit is used to perform corresponding convolution operations based on each second image to acquire each second feature image with different scales.

[0192] The detection module 320 further includes a splicing unit and a second fusion unit, wherein:

[0193] A stitching unit, configured to perform a stitching operation and an upsampling operation based on the first feature image and the second feature image of the same scale to obtain third images;

[0194] The second fusion unit is used to perform feature fusion based on each third image to obtain a second fused image.

[0195] The up-sampling operation of the detection module 320 includes an up-sampling operation based on a bilinear interpolation method.

[0196] The cloud detection device based on remote sensing images provided in an embodiment of the present invention is used to execute the above-mentioned cloud detection method based on remote sensing images of the present invention. Its implementation method is consistent with the implementation method of the cloud detection method based on remote sensing images provided by the present invention, and can achieve the same beneficial effects, which will not be repeated here.

[0197] The embodiment of the present invention obtains a target image based on a remote sensing image with multiple channels, extracts first feature images of multiple scales through cascaded multiple levels of downsampling sublayers, obtains a first fused image through feature fusion of the first feature images of multiple scales, restores the first fused image into a second feature image of multiple scales through cascaded multiple levels of upsampling sublayers, splices and fuses the first feature image of the same scale with the first feature image to obtain a second fused image, and restores the cloud detection result through the second fused image. The receptive field of the convolution can be increased layer by layer without losing information, so that the output of each convolution contains a larger range of semantic information, which can improve the refinement of image semantic segmentation and thus improve the accuracy of cloud detection.

[0198] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor (processor) 410, a communication interface (CommunicationsInterface) 420, a memory (memory) 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logic instructions in the memory 430 to execute a cloud detection method based on a remote sensing image, the method comprising: preprocessing based on the remote sensing image to obtain a target image; inputting the target image into a cloud detection model, and outputting a cloud detection result corresponding to the target image; wherein the cloud detection model comprises: a multi-scale downsampling layer, for performing downsampling operations and convolution operations of different scales based on the target image to obtain first feature images of different scales; a first fusion layer, for performing feature fusion based on the first feature images of different scales to obtain a first fused image; a multi-scale upsampling layer, for performing upsampling operations and convolution operations of different scales based on the first fused image to obtain second feature images of different scales; a second fusion layer, for performing feature fusion based on the first feature image and the second feature image corresponding to the scale to obtain a second fused image; an output layer, for obtaining a cloud detection result corresponding to the target image based on the second fused image; wherein the multi-scale downsampling layer comprises a plurality of cascaded levels of downsampling sublayers, the multi-scale upsampling layer comprises a plurality of cascaded levels of upsampling sublayers, and the number of downsampling sublayers is the same as the number of upsampling sublayers.

[0199] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0200] On the other hand, the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the cloud detection method based on remote sensing images provided by the above methods, and the method includes: preprocessing based on the remote sensing image to obtain a target image; inputting the target image into a cloud detection model, and outputting a cloud detection result corresponding to the target image; wherein the cloud detection model includes: a multi-scale downsampling layer, which is used to perform downsampling operations and convolution operations of different scales based on the target image to obtain first feature images of different scales; a first fusion layer, Used to perform feature fusion based on first feature images of different scales to obtain a first fused image; a multi-scale upsampling layer is used to perform upsampling operations and convolution operations of different scales based on the first fused image to obtain second feature images of different scales; a second fusion layer is used to perform feature fusion based on the first feature image and the second feature image corresponding to the scales to obtain a second fused image; an output layer is used to obtain a cloud detection result corresponding to the target image based on the second fused image; wherein the multi-scale downsampling layer includes a plurality of cascaded levels of downsampling sublayers, and the multi-scale upsampling layer includes a plurality of cascaded levels of upsampling sublayers, and the number of downsampling sublayers is the same as the number of upsampling sublayers.

[0201] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the cloud detection method based on remote sensing images provided by the above methods, the method comprising: preprocessing based on the remote sensing image to obtain a target image; inputting the target image into a cloud detection model, and outputting a cloud detection result corresponding to the target image; wherein the cloud detection model comprises: a multi-scale downsampling layer for performing downsampling operations and convolution operations of different scales based on the target image to obtain first feature images of different scales; a first fusion layer for performing downsampling operations and convolution operations of different scales based on the first feature images of different scales The multi-scale upsampling layer is used to perform upsampling operations and convolution operations of different scales based on the first fused image to obtain second feature images of different scales; the second fusion layer is used to perform feature fusion based on the first feature image and the second feature image corresponding to the scale to obtain the second fused image; the output layer is used to obtain the cloud detection result corresponding to the target image based on the second fused image; wherein the multi-scale downsampling layer includes a plurality of cascaded levels of downsampling sublayers, the multi-scale upsampling layer includes a plurality of cascaded levels of upsampling sublayers, and the number of downsampling sublayers is the same as the number of upsampling sublayers.

[0202] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0203] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0204] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cloud detection method based on remote sensing images, characterized in that: include: Perform preprocessing based on remote sensing images to obtain target images; Inputting the target image into a cloud detection model, and outputting a cloud detection result corresponding to the target image; Wherein, the cloud detection model includes: A multi-scale downsampling layer, used to perform downsampling operations and convolution operations of different scales based on the target image to obtain first feature images of different scales; A first fusion layer is used to perform corresponding pooling operations based on the first feature images of each scale to obtain first images with the same two-dimensional scale; based on each of the first images, perform fusion operations and convolution dimensionality reduction operations to obtain a first fused image; A multi-scale upsampling layer, used for performing upsampling operations and convolution operations of different scales based on the first fused image to obtain second feature images of different scales; The second fusion layer is used to perform a splicing operation and an upsampling operation based on the first feature image and the second feature image of the same scale to obtain each third image; and perform feature fusion based on each of the third images to obtain a second fused image; an output layer, used for obtaining a cloud detection result corresponding to the target image based on the second fused image; The multi-scale downsampling layer includes a plurality of cascaded levels of downsampling sublayers, the multi-scale upsampling layer includes a plurality of cascaded levels of upsampling sublayers, and the number of the downsampling sublayers is the same as the number of the upsampling sublayers.

2. The cloud detection method based on remote sensing images according to claim 1, characterized in that: The method of performing preprocessing based on remote sensing images to obtain target images includes: Based on a preset formula, the brightness temperature value and reflectivity of each pixel of the remote sensing image are obtained; Based on the brightness temperature value and reflectivity of each pixel point, a target image is acquired.

3. The cloud detection method based on remote sensing images according to claim 1, characterized in that: The performing upsampling operations and convolution operations of different scales based on the first fused image to obtain second feature images of different scales includes: Based on the first fused image, performing upsampling operations layer by layer to obtain second images of different two-dimensional scales; Based on each of the second images, a corresponding convolution operation is performed to obtain second feature images of different scales.

4. The cloud detection method based on remote sensing images according to claim 3 is characterized in that: The up-sampling operation includes an up-sampling operation based on a bilinear interpolation method.

5. A cloud detection device based on remote sensing images, characterized in that: include: A preprocessing module is used to perform preprocessing based on remote sensing images to obtain target images; A detection module, used to input the target image into a cloud detection model and output a cloud detection result corresponding to the target image; Wherein, the cloud detection model includes: A multi-scale downsampling layer, used to perform downsampling operations and convolution operations of different scales based on the target image to obtain first feature images of different scales; A first fusion layer is used to perform corresponding pooling operations based on the first feature images of each scale to obtain first images with the same two-dimensional scale; based on each of the first images, perform fusion operations and convolution dimensionality reduction operations to obtain a first fused image; A multi-scale upsampling layer, used for performing upsampling operations and convolution operations of different scales based on the first fused image to obtain second feature images of different scales; The second fusion layer is used to perform a splicing operation and an upsampling operation based on the first feature image and the second feature image of the same scale to obtain each third image; and perform feature fusion based on each of the third images to obtain a second fused image; an output layer, used for obtaining a cloud detection result corresponding to the target image based on the second fused image; The multi-scale downsampling layer includes a plurality of cascaded levels of downsampling sublayers, the multi-scale upsampling layer includes a plurality of cascaded levels of upsampling sublayers, and the number of the downsampling sublayers is the same as the number of the upsampling sublayers.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the cloud detection method based on remote sensing images as described in any one of claims 1 to 4 are implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the cloud detection method based on remote sensing images as claimed in any one of claims 1 to 4 are implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the cloud detection method based on remote sensing images as claimed in any one of claims 1 to 4 are implemented.

Images