Cloud detection model training method and cloud detection method
By using the boundary position perception module of deformable convolution and attention mechanism in the cloud detection model, the problem of low cloud detection accuracy in cloud-snow mixed scenarios is solved, and the accuracy and performance of cloud detection are significantly improved.
Patent Information
- Application Number
- CN202411283874.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-09-13
AI Technical Summary
The prior art is difficult to accurately detect clouds in mixed clouds and snow scenes in high altitude and high latitude areas, resulting in low cloud detection accuracy, and the reflection characteristics and texture characteristics of clouds and snow are similar, making boundaries difficult to distinguish.
The boundary position perception module composed of deformable convolution and attention mechanism is used for cloud detection model training. The boundary position perception module extracts complex cloud boundary features to improve the accuracy of cloud detection and recognition.
It effectively improves the recognition accuracy of cloud detection models in mixed cloud and snow scenarios, enhances the feature extraction ability of complex cloud boundaries, and improves detection performance.
Smart Images

Figure CN119068356A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of cloud detection technology, and in particular to a cloud detection model training method and a cloud detection method. Background Art
[0002] In addition to road traffic navigation, high-resolution remote sensing satellites can also capture detailed information on the ground and have a wide range of applications in land planning and security monitoring. Clouds and snow cover seriously affect satellite remote sensing images of the Earth. Clouds may block the imaging channel of optical satellites and reduce the number of pixels in the available images. Accurate cloud detection is essential for the preprocessing of most remote sensing images. In high-altitude and high-latitude areas, snow shows reflective properties and texture characteristics similar to those of clouds, resulting in low accuracy of cloud detection in existing technologies in mixed cloud and snow scenes. Summary of the invention
[0003] The purpose of the present invention is to provide a cloud detection model training method and a cloud detection method, which are applied to the field of cloud detection technology. The method performs cloud detection through an image segmentation model. The encoder and decoder of the model are skip-connected through a boundary position perception module composed of deformable convolution and attention mechanism. The boundary position perception module can effectively extract the features of complex cloud boundaries, thereby improving the accuracy of model cloud detection and recognition.
[0004] In order to solve the above technical problems, the present invention provides a cloud detection model training method, comprising:
[0005] Acquire a remote sensing image set containing a mixed scene of clouds and snow, and input training samples in the remote sensing image set into a cloud detection model to obtain an output cloud detection segmentation image;
[0006] determining a loss value based on the cloud detection segmentation image and the training sample, and updating a model parameter of the cloud detection model based on the loss value;
[0007] Retraining the cloud detection model until the model parameters converge, thereby obtaining the trained cloud detection model;
[0008] The encoder and decoder of the cloud detection model are connected via a boundary position sensing module through a skip layer;
[0009] The boundary position perception module includes: a classifier, a deformable convolution unit, a low-pass filter, a channel multiplier and a multi-head attention mechanism unit;
[0010] The input of the classifier and the low-pass filter is the features extracted by the encoder; the input of the deformable convolution unit is the pre-segmentation result image output by the classifier; the input of the channel multiplier is the pre-segmentation result image and the low-frequency features output by the low-pass filter; the input of the multi-head attention mechanism unit is the query vector encoded by the low-frequency features and the key vector and value vector encoded by the boundary features, and the output boundary attention is used as the input of the decoder.
[0011] Optionally, the encoder includes: a first double downsampling module, a second double downsampling module, a third double downsampling module, a first dilated convolution module, a second dilated convolution module, a dilated pyramid pooling layer and a first channel dimensionality reduction module connected in sequence.
[0012] Optionally, the decoder includes: a first feature fusion module, a second feature fusion module, a first double upsampling module, a channel adder and a quadruple upsampling module connected in sequence; the first hole convolution module is jump-layer connected to the first feature fusion module through a first boundary position sensing module, the third double downsampling module is jump-layer connected to the second feature fusion module through a second boundary position sensing module, the second double downsampling module is connected to the channel adder of the decoder through an external channel dimensionality reduction module, the first channel dimensionality reduction module of the encoder is connected to the channel adder of the decoder through the second double upsampling module, and the first feature fusion module is connected to the channel adder of the decoder through the third double upsampling module.
[0013] Optionally, the atrous pyramid pooling layer includes a parallel second channel dimensionality reduction module, a third atrous convolution module, a fourth atrous convolution module, a fifth atrous convolution module, a pooling module and a feature splicing module.
[0014] Optionally, the dilation ratio of the first dilated convolution module is 2, the dilation ratio of the second dilated convolution module is 4, the dilation ratio of the third dilated convolution module is 12, the dilation ratio of the fourth dilated convolution module is 24, and the dilation ratio of the fifth dilated convolution module is 36.
[0015] Optionally, the feature fusion module in the decoder includes: two convolution normalization units, two pooling units, four channel multipliers, two channel adders, an activation function unit and a channel dimension reduction unit;
[0016] The first convolution normalization unit is directly connected to the first channel multiplier and connected to the first channel multiplier through the first pooling unit; the second convolution normalization unit is directly connected to the second channel multiplier and connected to the second channel multiplier through the second pooling unit;
[0017] The first channel multiplier is connected to the first channel adder and the third channel multiplier; the second channel multiplier is connected to the first channel adder and the fourth channel multiplier;
[0018] The first channel adder is connected to the channel dimension reduction unit; the channel dimension reduction unit is connected to the activation function unit;
[0019] The probability map output by the activation function unit is used as the input of the third channel multiplier;
[0020] The result of subtracting the probability map output by the activation function unit from the all-1 matrix is used as the input of the fourth channel multiplier;
[0021] The third channel multiplier and the fourth channel multiplier are connected to the second channel adder.
[0022] Optionally, determining the loss value based on the cloud detection segmentation image and the training sample includes:
[0023] An intersection-over-union ratio is determined based on the cloud detection segmentation image and the training samples.
[0024] Optionally, the downsampling module in the encoder includes a convolution unit and a pooling unit.
[0025] Optionally, the pooling in the cloud detection model is average pooling.
[0026] In order to solve the above technical problems, the present invention provides a cloud detection method, comprising:
[0027] Acquire the image to be detected;
[0028] Inputting the image to be detected into a cloud detection model to obtain a target segmentation image output by the cloud detection model;
[0029] The cloud detection model is a model trained according to the cloud detection model training method.
[0030] It can be seen that the method of the present invention obtains a remote sensing image set containing a mixed scene of clouds and snow, inputs the training samples in the remote sensing image set into the cloud detection model to obtain the output cloud detection segmentation image; determines the loss value based on the cloud detection segmentation image and the training samples, and updates the model parameters of the cloud detection model based on the loss value; retrains the cloud detection model until the model parameters converge to obtain a trained cloud detection model; wherein the encoder and decoder of the cloud detection model are connected by a boundary position perception module; the boundary position perception module includes two branches, the first branch includes a classifier and a deformable convolution unit connected in sequence, and the second branch includes a low-pass filter; the low-pass filter and the deformable convolution unit are connected by a channel adder. The method of the present invention performs cloud detection through an image segmentation model, and the encoder and decoder of the model are connected by a boundary position perception module composed of a deformable convolution and an attention mechanism. The boundary position perception module can effectively extract the features of complex cloud boundaries, thereby improving the accuracy of model cloud detection and recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0032] Figure 1 A flow chart of a cloud detection model training method provided by an embodiment of the present invention;
[0033] Figure 2 A structural example diagram of a boundary position sensing module provided by an embodiment of the present invention;
[0034] Figure 3 A diagram showing an example structure of a cloud detection model provided by an embodiment of the present invention;
[0035] Figure 4 A structural example diagram of a hole pyramid pooling module provided by an embodiment of the present invention;
[0036] Figure 5 A structural example diagram of a feature fusion module provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0038] In addition to road traffic navigation, high-resolution remote sensing satellites can also capture detailed information on the ground, which has a wide range of applications in land planning and security monitoring. However, on average, about 66% of the sky on Earth is covered by clouds every year, of which more than about 30% is covered by seasonal snow and about 10% is covered by permanent snow.
[0039] Clouds and snow cover severely affect Earth observations from satellite remote sensing images. Clouds can obstruct the imaging channel of optical satellites, reducing the number of pixels in the available images. In addition, clouds and snow can blur or obscure ground objects, severely affecting the usability of optical satellite data. Accurate cloud detection is essential for the preprocessing of most remote sensing images.
[0040] However, at high altitudes and latitudes, snow shows similar reflectance properties and texture characteristics as clouds, which interferes with cloud detection methods. Therefore, cloud detection in areas where clouds and snow coexist is more challenging than in other cases. The boundary between clouds and snow is blurred, making accurate classification of clouds and snow difficult.
[0041] Although cloud and snow detection in remote sensing images provides critical information for various practical tasks, achieving high-precision detection remains challenging. The emergence of deep learning provides a promising solution to this challenge.
[0042] Since clouds and snow have similar radiation characteristics and texture features in mixed cloud and snow areas, existing deep learning methods are only applicable to cloud detection and recognition in a single scene, and it is difficult to distinguish between clouds and snow in such complex scenes. The presence of thin clouds further makes it difficult to distinguish the cloud-snow boundary. How to correctly detect the cloud-snow boundary in mixed cloud and snow scenes is very important for improving the usability of optical remote sensing images.
[0043] The present invention proposes a cloud detection model for cloud-snow mixed scenes based on boundary position attention, which is used for accurate detection of cloud pixels in cloud-snow mixed scenes. Specifically, variable convolution enhances the receptive field by dynamically adjusting the position of the convolution kernel. This technology enables the network to flexibly capture local target features, effectively identify complex cloud boundary features, and significantly improve detection performance in environments where clouds and snow coexist. And when variable convolution is used in combination with the Transformer model, this method further enhances the ability to pay attention and recognize boundary positions. The Transformer model is a deep neural network model based on the self-attention mechanism.
[0044] The following combination Figure 1 , Figure 1 A flow chart of a cloud detection model training method provided by an embodiment of the present invention, the method may include:
[0045] S101: Obtain a remote sensing image set containing a mixed scene of clouds and snow, and input training samples in the remote sensing image set into a cloud detection model to obtain an output cloud detection segmentation image.
[0046] The cloud detection model in this embodiment is intended to perform cloud detection in complex scenes of clouds and snow. Therefore, this embodiment can obtain a remote sensing image set containing mixed scenes of clouds and snow. This embodiment does not limit the specific method of obtaining the remote sensing image set, nor does it limit the specific type of the remote sensing image set. Specifically, the remote sensing image set of this embodiment can be the CloudS26 dataset. The CloudS26 dataset is a remote sensing image dataset focused on cloud detection. It provides a large number of high-resolution remote sensing images for training and testing deep learning models to identify and segment clouds. Each scene in the dataset has visible clouds and snow in the field of view. The CloudS26 dataset includes a training set, a validation set, and a test set, which contain 24179, 2679, and 8380 images, respectively.
[0047] In this embodiment, the remote sensing image set may include the CSWV dataset, which is a groundbreaking high-resolution cloud and snow dataset with a spatial resolution ranging from 0.5 meters to 10 meters. The scenes depicted in these images include forests, grasslands, lakes, and bare ground, and the snow is divided into three types: permanent snow, stable snow, and intermittent snow, showing a variety of scenes and enhancing the universality and representativeness of the dataset.
[0048] Before model training, this embodiment can preprocess remote sensing image sets of different sizes, such as unifying resolution and normalizing. This embodiment does not limit the size of the input image resolution, nor does it limit the specific method of normalization, and can be set based on actual applications.
[0049] This embodiment does not limit the specific method of model training. Generally, AdaGrad (adaptive gradient algorithm) can be used as the optimizer, the learning rate can be set to 0.05, and the Xavier strategy can be used for weight initialization. The Xavier strategy is a parameter initialization strategy widely used in deep learning, which aims to solve the problems of gradient disappearance and gradient explosion, and accelerate the convergence speed of the model. This embodiment can train 500 epochs (rounds), the weight decay can be 1×10^(-4), and the batch size can be set to 16.
[0050] In this embodiment, the training samples in the remote sensing image set can be input into the cloud detection model to obtain the output cloud detection segmentation image, and the model parameters are updated through the output cloud detection segmentation image.
[0051] S102: Determine a loss value based on the cloud detection segmentation image and the training samples, and update model parameters of the cloud detection model based on the loss value.
[0052] In this embodiment, the loss value can be determined by the cloud detection segmentation image and the training sample, and the model parameters of the cloud detection model can be updated based on the loss value. In this embodiment, the specific type of the model loss function is not limited. Generally, the ratio of the intersection area and the union area between the cloud detection segmentation image and the labeled true value, that is, the intersection-to-union ratio, can be obtained.
[0053] In this embodiment, an intersection-over-union ratio may be determined based on the cloud detection segmentation image and the training samples, and model parameters of the cloud detection model may be updated based on the intersection-over-union ratio.
[0054] S103: Retrain the cloud detection model until the model parameters converge to obtain a trained cloud detection model; wherein the encoder and decoder of the cloud detection model are jump-connected through a boundary position sensing module; the boundary position sensing module includes: a classifier, a deformable convolution unit, a low-pass filter, a channel multiplier and a multi-head attention mechanism unit; the input of the classifier and the low-pass filter is the feature extracted by the encoder; the input of the deformable convolution unit is the pre-segmentation result image output by the classifier; the input of the channel multiplier is the low-frequency feature of the pre-segmentation result image and the output of the low-pass filter; the input of the multi-head attention mechanism unit is the query vector encoded by the low-frequency feature and the key vector and value vector encoded by the boundary feature, and the output boundary attention is used as the input of the decoder.
[0055] This embodiment can retrain the cloud detection model based on a preset epoch until the model parameters converge to obtain a trained cloud detection model. This embodiment can perform cloud detection in complex scenarios through the trained cloud detection model.
[0056] This embodiment does not limit the specific architecture of the cloud detection model, which can generally be a U-shaped image segmentation model. The encoder and decoder of the cloud detection model are connected by jump layers through a boundary position perception module; the boundary position perception module includes: a classifier, a deformable convolution unit, a low-pass filter, a channel multiplier and a multi-head attention mechanism unit; the input of the classifier and the low-pass filter is the feature extracted by the encoder; the input of the deformable convolution unit is the pre-segmentation result image output by the classifier; the input of the channel multiplier is the pre-segmentation result image and the low-frequency feature output by the low-pass filter; the input of the multi-head attention mechanism unit is the query vector encoded by the low-frequency feature and the key vector and value vector encoded by the boundary feature, and the output boundary attention is used as the input of the decoder.
[0057] By adopting deformable convolution and attention mechanism to generate attention centered on the boundary position, the network can learn the subtle differences of the edge of the object. The boundary position perception module is crucial to improving the model's ability to accurately divide boundaries in complex visual scenes.
[0058] The specific architecture of the Boundary Position Attention Module (BPAM) can be described as follows: Figure 2 As shown, the input of the boundary position perception module can be a feature map extracted by the encoder, the feature map is pre-segmented by the classifier to obtain a pre-segmentation result map, the pre-segmentation result map is extracted by a deformable convolution unit to obtain a warp deformation field (Warp), thereby realizing the extraction of boundary information from the rough prediction result; the feature map is processed by a low-pass filter to obtain low-frequency features; the low-frequency features and the warp deformation field are input into the channel multiplier to obtain boundary features containing boundary information; the boundary features are encoded to obtain a key vector (Key) and a value vector (Value); the low-frequency features are encoded to obtain a query vector (Query); the key vector, the value vector and the query vector are used as the input of the multi-head attention mechanism unit, and the boundary attention of the boundary features is obtained through the multi-head attention mechanism unit. This embodiment does not limit the specific type of the classifier, nor does it limit the specific architecture of the multi-head attention mechanism unit.
[0059] This embodiment does not limit the specific architecture of the encoder, and can be based on the deeplabv3 architecture, including a downsampling module, a dilated convolution module, and a dilated pyramid pooling module. Deeplabv3 is a classic deep neural network for semantic segmentation tasks. This embodiment does not limit the number of downsampling modules and the downsampling ratio, nor does it limit the number of dilated convolution modules and the dilated ratio of the dilated convolution.
[0060] To extract multi-scale information from deep layers, this embodiment employs a set of parallel dilated convolutions running with different dilation rates. This design choice enables the cloud detection model to cover different receptive fields without losing resolution, thereby fully capturing the various features necessary to understand complex images.
[0061] Specifically, the encoder in this embodiment can be as follows Figure 3 As shown, it includes: a first double downsampling module, a second double downsampling module, a third double downsampling module, a first dilated convolution module, a second dilated convolution module, a dilated pyramid pooling layer and a first channel dimension reduction module connected in sequence. The double downsampling module is used for downsampling with a magnification of 2.
[0062] The downsampling module in this embodiment is used to extract features. This embodiment does not limit the specific method in which the downsampling module extracts features. Generally, the downsampling module may include a convolution unit and a pooling unit, and features are extracted through the convolution unit and the pooling unit.
[0063] This embodiment does not limit the specific architecture of the Atrous Spatial Pyramid Pooling (ASPP) layer, and can be as follows Figure 4 As shown, it generally includes a parallel second channel dimensionality reduction module, a third atrous convolution module, a fourth atrous volume module, a fifth atrous convolution module, a pooling module and a feature splicing module. The outputs of the second channel dimensionality reduction module, the third atrous convolution module, the fourth atrous volume module, the fifth atrous convolution module and the pooling module are subjected to feature splicing by the feature splicing module.
[0064] Furthermore, in this embodiment, the dilation ratio of the first dilated convolution module can be 2, the dilation ratio of the second dilated convolution module can be 4, the dilation ratio of the third dilated convolution module can be 12, the dilation ratio of the fourth dilated convolution module can be 24, and the dilation ratio of the fifth dilated convolution module can be 36.
[0065] This embodiment does not limit the specific type of the decoder, and can generally be as shown in 3, including: a first feature fusion module, a second feature fusion module, a first double upsampling module, a channel adder and a four-fold upsampling module connected in sequence; the first hole convolution module is jump-layer connected to the first feature fusion module through the first boundary position sensing module, the third double downsampling module is jump-layer connected to the second feature fusion module through the second boundary position sensing module, the second double downsampling module is connected to the channel adder of the decoder through the external channel dimensionality reduction module, the first channel dimensionality reduction module of the encoder is connected to the channel adder of the decoder through the second double upsampling module, and the first feature fusion module is connected to the channel adder of the decoder through the third double upsampling module.
[0066] The channel dimensionality reduction of the cloud detection model in this embodiment can generally be achieved through 1×1 convolution, and the pooling operation can be average pooling.
[0067] The feature fusion module is used to integrate features from different depths of the network, extract boundary details from shallow representations, and adaptively mix the boundary details learned from the shallow layer with the abstract features obtained from the deep layer. This module ensures that the network can simultaneously utilize local and global information to improve its prediction accuracy and improve its overall performance in boundary detection tasks.
[0068] In most previous studies, element summation is the most widely used method for multi-scale feature fusion. However, this simple summation operation cannot take into account the level or scale differences between different feature maps. In order to effectively fuse multi-scale feature maps, this embodiment proposes a feature fusion module to ensure seamless fusion of features of different scales. Its architecture can be as follows: Figure 5 As shown, it includes: two convolution normalization units, two pooling units, four channel multipliers, two channel adders, an activation function unit and a channel dimension reduction unit;
[0069] The first convolutional normalization unit is directly connected to the first channel multiplier and is connected to the first channel multiplier through the first pooling unit;
[0070] The second convolution normalization unit is directly connected to the second channel multiplier and is connected to the second channel multiplier through the second pooling unit;
[0071] The first channel multiplier is connected to the first channel adder and the third channel multiplier; the second channel multiplier is connected to the first channel adder and the fourth channel multiplier;
[0072] The first channel adder is connected to the channel dimension reduction unit; the channel dimension reduction unit is connected to the activation function unit;
[0073] The probability map output by the activation function unit is used as the input of the third channel multiplier; the result of subtracting the probability map output by the channel dimension reduction unit from the all-1 matrix is used as the input of the fourth channel multiplier;
[0074] The third channel multiplier and the fourth channel multiplier are connected to the second channel adder.
[0075] In this embodiment, the first convolution normalization unit and the second convolution normalization unit are used to receive features that need to be fused. The first convolution normalization unit and the second convolution normalization unit can both receive features of the current level and features of the previous level, which is not specifically limited in this embodiment.
[0076] This embodiment enhances features through a convolutional normalization (Conv Batch Normalization, CBN) unit, and then extracts a global channel descriptor through a pooling unit; multiplies the output of the first convolutional normalization unit with the output of the first pooling unit through a first channel multiplier; multiplies the output of the second convolutional normalization unit with the output of the second pooling unit through a second channel multiplier; and adds the outputs of the first channel multiplier and the second channel multiplier through a first channel adder.
[0077] In this embodiment, the output of the first channel adder is processed by a channel dimensionality reduction unit to compress the number of channels of the feature map to 1; the output of the channel dimensionality reduction unit is processed by an activation function to obtain a weight ratio of the feature map, that is, a probability map; this embodiment does not limit the specific type of the activation function, which can generally be a sigmoid activation function.
[0078] The probability map output by the activation function unit is used as the input of the third channel multiplier and multiplied with the output of the first multiplier; the result of subtracting the probability map output by the activation function unit from the all-1 matrix is used as the input of the fourth channel multiplier and multiplied with the output of the second multiplier; the output of the fourth channel multiplier of the third channel multiplier is used as the input of the second channel adder, and finally the fusion feature output by the second channel adder is obtained.
[0079] Based on the above embodiments, the method of the present invention performs cloud detection through an image segmentation model. The encoder and decoder of the model are jump-connected through a boundary position perception module composed of deformable convolution and attention mechanism. The boundary position perception module can effectively extract the features of complex cloud boundaries, thereby improving the accuracy of model cloud detection and recognition.
[0080] The following is a cloud detection method provided by an embodiment of the present invention, which may include:
[0081] Acquire the image to be detected;
[0082] Input the image to be detected into the cloud detection model to obtain the target segmentation image output by the cloud detection model;
[0083] The cloud detection model is a model trained according to the cloud detection model training method.
[0084] Based on the above embodiments, the present invention may also provide an electronic device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiments may be implemented. Of course, the device may also include various necessary network interfaces, power supplies, and other components.
[0085] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by an execution terminal or a processor, the method provided in the embodiment of the present invention can be implemented; the storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0086] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
Claims
1. A cloud detection model training method, characterized in that: include: Acquire a remote sensing image set containing a mixed scene of clouds and snow, and input training samples in the remote sensing image set into a cloud detection model to obtain an output cloud detection segmentation image; determining a loss value based on the cloud detection segmentation image and the training sample, and updating a model parameter of the cloud detection model based on the loss value; Retraining the cloud detection model until the model parameters converge, thereby obtaining the trained cloud detection model; The encoder and decoder of the cloud detection model are connected via a boundary position sensing module through a skip layer; The boundary position perception module includes: a classifier, a deformable convolution unit, a low-pass filter, a channel multiplier and a multi-head attention mechanism unit; The input of the classifier and the low-pass filter is the features extracted by the encoder; the input of the deformable convolution unit is the pre-segmentation result image output by the classifier; the input of the channel multiplier is the pre-segmentation result image and the low-frequency features output by the low-pass filter; the input of the multi-head attention mechanism unit is the query vector encoded by the low-frequency features and the key vector and value vector encoded by the boundary features, and the output boundary attention is used as the input of the decoder.
2. The cloud detection model training method according to claim 1, characterized in that: The encoder comprises: a first double downsampling module, a second double downsampling module, a third double downsampling module, a first dilated convolution module, a second dilated convolution module, a dilated pyramid pooling layer and a first channel dimension reduction module which are connected in sequence.
3. The cloud detection model training method according to claim 2, characterized in that: The decoder includes: a first feature fusion module, a second feature fusion module, a first double upsampling module, a channel adder and a four-fold upsampling module connected in sequence; the first hole convolution module is jump-layer connected to the first feature fusion module through a first boundary position sensing module, the third double downsampling module is jump-layer connected to the second feature fusion module through a second boundary position sensing module, the second double downsampling module is connected to the channel adder of the decoder through an external channel dimensionality reduction module, the first feature fusion module is connected to the channel adder of the decoder through a second double upsampling module, and the first channel dimensionality reduction module of the encoder is connected to the channel adder of the decoder through the third double upsampling module.
4. The cloud detection model training method according to claim 2, characterized in that: The dilated pyramid pooling layer includes a parallel second channel dimension reduction module, a third dilated convolution module, a fourth dilated convolution module, a fifth dilated convolution module, a pooling module and a feature splicing module.
5. The cloud detection model training method according to claim 4, characterized in that: The dilation ratio of the first dilated convolution module is 2, the dilation ratio of the second dilated convolution module is 4, the dilation ratio of the third dilated convolution module is 12, the dilation ratio of the fourth dilated convolution module is 24, and the dilation ratio of the fifth dilated convolution module is 36.
6. The cloud detection model training method according to claim 1, characterized in that: The feature fusion module in the decoder includes: two convolution normalization units, two pooling units, four channel multipliers, two channel adders, an activation function unit and a channel dimension reduction unit; The first convolution normalization unit is directly connected to the first channel multiplier and connected to the first channel multiplier through the first pooling unit; the second convolution normalization unit is directly connected to the second channel multiplier and connected to the second channel multiplier through the second pooling unit; The first channel multiplier is connected to the first channel adder and the third channel multiplier; the second channel multiplier is connected to the first channel adder and the fourth channel multiplier; The first channel adder is connected to the channel dimension reduction unit; the channel dimension reduction unit is connected to the activation function unit; The probability map output by the activation function unit is used as the input of the third channel multiplier; The result of subtracting the probability map output by the activation function unit from the all-1 matrix is used as the input of the fourth channel multiplier; The third channel multiplier and the fourth channel multiplier are connected to the second channel adder.
7. The cloud detection model training method according to claim 1, characterized in that: The determining of the loss value based on the cloud detection segmentation image and the training sample comprises: An intersection-over-union ratio is determined based on the cloud detection segmentation image and the training samples.
8. The cloud detection model training method according to claim 1, characterized in that: The downsampling module in the encoder includes a convolution unit and a pooling unit.
9. The cloud detection model training method according to claim 1, characterized in that: The pooling in the cloud detection model is average pooling.
10. A cloud detection method, characterized in that: include: Acquire the image to be detected; Inputting the image to be detected into a cloud detection model to obtain a target segmentation image output by the cloud detection model; The cloud detection model is a model trained according to the cloud detection model training method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Dermatoscope image segmentation method and system
CN114419060A
Satellite remote sensing image small target detection method based on high-resolution characteristic self-attention
CN117036980A
Target detection method, and moving-target tracking method using same
WO2023138300A1
Target detection model training method and apparatus, map generation method and apparatus, and device
WO2024037552A1
Cited By
Railway tunnel construction water leakage risk identification and disposal method and device
CN120877202A
