Satellite remote sensing image cloud classification method based on space-time coding guidance

Abstract

The invention discloses a satellite remote sensing image cloud classification method based on space-time coding guidance, and particularly relates to the technical field of satellite image processing and deep learning, and the method comprises the following steps: carrying out the preprocessing of the original data of a geostationary meteorological satellite and a Cloud Sat satellite; preprocessing operation such as cutting, projection conversion and feature selection is carried out on the geostationary meteorological satellite data, a top cloud type is selected for the Cloud Sat data to serve as a label, and space-time matching is carried out on the processed geostationary meteorological satellite data and the Cloud Sat data to obtain a training and testing data set; and establishing a GTCCT model, performing time and geocoding on the input data, inputting the coded feature data into a network for training, and finally obtaining the GTCCT model for cloud classification.

Description

Technical Field

[0001] The present invention relates to the field of satellite image processing and deep learning technology, and in particular to a satellite remote sensing image cloud classification method based on spatiotemporal coding guidance. Background Art

[0002] Clouds play a vital role in climate and weather systems. Different types of clouds have different radiation characteristics. Cloud type information can better simulate cloud morphology, quantity, distribution and other characteristics, thereby supporting more research work. Domestic and foreign researchers have actively explored cloud classification research while ensuring classification accuracy and improving its efficiency. From a methodological point of view, it can be divided into two types: traditional cloud classification methods and cloud classification methods based on deep learning.

[0003] There are four basic traditional cloud classification methods: threshold method, window method, statistical method and texture-based method. The threshold method and window method mainly distinguish cloud types or clouds and ground objects by using reflectivity, radiation brightness temperature and brightness temperature difference. Although these two methods are simple and effective under certain conditions, they often show obvious limitations when facing complex cloud systems, dynamic changes and multi-layer clouds. For example, in highly reflective desert areas, these two methods are prone to misjudgment and cannot distinguish between ground objects and clouds. Statistical methods mainly include clustering method and histogram method, but these two methods have limited ability to distinguish different spectral characteristics and are easy to confuse different types of clouds. In addition, the histogram method can only be classified based on a single band or brightness temperature information, and cannot make full use of multi-band data. In addition, the texture-based method can capture the local structure and details of the cloud layer and is suitable for the classification of clouds with specific morphology. However, this method cannot use long-term observation data, and when the cloud layer is relatively uniform, the texture features are not obvious and the classification effect is poor. In addition, support vector machine (SVM) technology has been used for cloud classification tasks and has achieved significant classification results. However, SVM requires too much training time and computing resources and is not suitable for processing large-scale satellite images. Summary of the invention

[0004] The satellite image cloud classification method based on deep learning in the present invention starts with data collection and preprocessing, and performs cloud type recognition and classification by designing and training a deep neural network model, and finally obtains the cloud classification result through reasoning. This method can effectively process a large amount of complex satellite image data, automatically extract the spatial characteristics and spatiotemporal variation patterns of clouds, avoid the complexity of artificial feature design, and can adapt to different types of cloud detection tasks. The deep learning method has a deeper neural network layer, which can better extract features across multiple scales and types, making it effective for cloud classification. However, most studies face the following problems: First, the spectral bands of satellites are not fully utilized, and the radiation differences between bands need to be further discussed comprehensively; second, the existing cloud classification research has not studied the distribution of cloud types over time and space.

[0005] To this end, the present invention provides a satellite remote sensing image cloud classification method based on spatiotemporal coding guidance to solve the problems raised in the background technology.

[0006] In order to achieve the above object, the present invention provides the following technical solution: a satellite remote sensing image cloud classification method based on spatiotemporal coding guidance, comprising the following steps: S1: Preprocess the CloudSat satellite data and geostationary meteorological satellite data, and perform spatiotemporal matching to obtain a cloud sample dataset; S2: Construct a cloud classification network based on the Transformer network, introduce a temporal geocoding module to enhance the model's perception of spatial and temporal information, and add a channel attention mechanism to integrate multiple feature channels of the data to obtain a GTCCT model for cloud classification; The construction process of the GTCCT model is: B1: Apply time and geocoding technology to expand the dimension and information content of input data: Encode time information and latitude and longitude information into sine and cosine values ​​and directly input them into the GTCCT model as separate channels, so that the GTCCT model can capture the distribution relationship of cloud types in time and space; Specifically, use sine and cosine functions to periodically encode the dates of the year. Taking a non-leap year as an example, let the first date of the year be Days, among which , use the following formula to encode the date: ; ; in, and Represents the sine and cosine encoding values ​​of a date respectively; geocoding technology for a given longitude and latitude , first convert the angle to radians, then calculate its sine and cosine: ; ; in, and denote the sine and cosine components of longitude, respectively. and represent the sine and cosine components of latitude respectively.

[0007] B2: Use the channel attention module CAM to establish the connection between channels and enhance the specific semantic response ability under the channel; analyze and weight the different channels in the image data to strengthen the key spectral features and suppress irrelevant spectral features; finally obtain the GTCCT cloud classification model; For a given input feature matrix ,in represents the number of channels of the feature matrix, represents the height of the feature map, Represents the width of the feature map. First, A is processed by two convolutional layers to obtain feature maps B and C. Then, B and C are re-dimensionalized. The transposed feature maps B and C are multiplied and passed through the softmax layer to calculate the spatial attention feature map S. The size of S is C×C. The specific operation is as follows: ; in, represents the channel index in the feature map B, Represents the channel index in the feature map C.

[0008] Then, convolution is performed on A to obtain the feature map D and matrix multiplication is performed on the spatial attention feature map S, which is then multiplied by the scale coefficient Finally, after the dimension is rearranged, it is summed with the initial feature map A to obtain the spatial attention feature map ; is the scale coefficient, initialized to 0, and gradually learns to assign larger weights. The specific operation is as follows: ; B3: Multi-Head Self-Attention (MHSA) is a key component in the Transformer model. It captures the diverse features and dependencies of different parts of the sequence by computing multiple sets of self-attention in parallel. For the input 9×9 image, the reshape operation in the MHSA converts the input 9×9 image into a one-dimensional sequence, so that the MHSA can effectively capture the relationship between different pixels in the image. The input sequence is linearly transformed to generate query, key, and value matrices: ; in, represents the original feature dimension, is the dimension of the key vector, is a learnable linear transformation matrix, which is used to map the input sequence to different feature spaces. are the representations of query, key, and value respectively; next, attention is calculated using the following formula: ; Among them, the softmax function is used to normalize the similarity scores so that the sum of the weights is 1, thereby focusing attention on the most relevant pixels; In order to improve the representation ability and generalization performance of the GTCCT model, the multi-head self-attention mechanism MHSA divides the query, key and value into heads, each head has a dimension of ; Each head independently calculates attention to capture features in the input sequence in different subspaces; for the The attention representation is calculated as follows: ; After calculating all the heads, concatenate the outputs of each head to get the aggregation matrix , and generates the final output through a linear transformation: ; in, is the linear projection matrix, which remaps the concatenated output back to the original feature dimension ; S3: Use the divided training set to train the GTCCT model, and use the test set to test the classification ability of the GTCCT model; S4: Use the trained GTCCT model to perform pixel-by-pixel cloud classification on satellite cloud images of the selected study area.

[0009] Preferably, the data preprocessing in step S1 includes: A1: Determination of CloudSat top-layer cloud type: Use 2B-CLDCLASS data to evaluate top-layer clouds, integrate cloud type information into single-layer results, and determine the cloud type of the point only for continuous and more than two layers of cloud types; in the case of multi-layer clouds of Dc and Ns cloud types, the cloud type is determined to be Dc or Ns; for other multi-layer scenes, the cloud type of the highest layer is used; when the stratus cloud type is St, it is usually at a lower altitude and is not representative enough in the sample, so this category is excluded; A2: Geostationary meteorological satellite data preprocessing and feature selection, including clipping, projection conversion and equal longitude and latitude interpolation.

[0010] A3: Data spatiotemporal matching: First, geostationary meteorological satellite data with a time difference of less than 5 minutes from the CPR scan are screened. Taking the CPR point as the center point, AHI scan points with a distance of less than 0.04° from the center point are screened as potential matching points of the CPR point. Then, the point with the smallest distance from the CPR point is selected from the potential matching points as the matching point, and a 9×9 neighborhood sample is selected with this point as the center to obtain a cloud type sample. The data set is divided into training set, validation set, and test set in a ratio of 7:2:1.

[0011] Preferably, the network training and testing process in step S3 is: C1: The training and test data sets cover the comprehensive observation data of the geostationary meteorological satellite in a certain year and the cloud classification data of CloudSat, and the classification results of CloudSat are used as labels; all selected channel data are input into the GTCCT model for training, cross entropy is used as the loss function, and Adam is selected as the optimizer of the GTCCT model; the constructed training data set is input into the GTCCT model for training with the dimensions of batch×25×9×9; C2: For the test set, the overall accuracy (OA), recall and Kappa coefficient are used as evaluation indicators to evaluate the effectiveness of the GTCCT model in the cloud classification task of Fengyun-4 satellite observation images. The calculation formulas for each evaluation indicator are: ; in, Indicates that the actual category And is correctly predicted as the category The number of samples, Represents the real type The pixel is predicted by the model to be of type The number of ; ; in, represents the total number of all samples, Representative categories are The closer the Kappa value is to 1, the better the consistency of the prediction. Preferably, the classification process in step S4 is: for the Fengyun-4 satellite cloud image of the selected study area, a 9×9 area is taken with each pixel as the center point, and the area is input into the trained GTCCT model for prediction to obtain the cloud classification result of each point.

[0012] Compared with the prior art, the present invention has the following advantages: 1. When faced with complex cloud systems, dynamic changes, and multi-layer cloud scenarios, the GTCCT model effectively solves the problems of traditional methods in single spectral dependence, poor adaptability to dynamic cloud changes, and insufficient ability to distinguish multi-layer clouds by integrating multimodal features and introducing channel attention and spatial attention mechanisms. In addition, the GTCCT model is designed to be efficient and robust, and can adapt to diverse scenarios and complex surface conditions, showing significant advantages in cloud classification tasks in complex scenarios.

[0013] 2. The GTCCT model, combined with long time series analysis, can capture the dynamic characteristics of clouds and effectively integrate multi-temporal data through the temporal geocoding module, thereby improving the classification ability of complex cloud systems. Unlike traditional methods, the GTCCT model can use temporal information to analyze the morphological evolution and coverage changes of clouds, enhance its adaptability to dynamically changing clouds, and improve the consistency of classification results, making it suitable for processing complex cloud scenes under dynamic weather conditions.

[0014] 3. The GTCCT model is designed to reduce redundant calculations by optimizing the network architecture and attention module, while introducing lightweight feature extraction and fusion mechanisms, which enables it to significantly reduce computational overhead while ensuring high classification accuracy. Compared with traditional methods and complex machine learning models, the GTCCT model is more suitable for the rapid processing of large-scale satellite images, and has high efficiency and practical applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flowchart of the method provided in this embodiment; Figure 2 Satellite data preprocessing and space-time matching flow chart provided for this embodiment; Figure 3 A schematic diagram of the time coding provided for this embodiment; Figure 4 The schematic diagram of geocoding provided for this embodiment; Figure 5 This is a structural diagram of the Transformer Encoder provided in this embodiment.

[0016] Figure 6 This is a structural diagram of the CAM module provided in this embodiment. DETAILED DESCRIPTION

[0017] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] This embodiment provides a cloud classification method for Fengyun-4 satellite images based on spatiotemporal coding guidance. Figure 1 As shown, the following steps are included: S1: Preprocess the CloudSat satellite data and Fengyun-4 satellite data, and perform spatiotemporal matching to obtain the cloud sample dataset; The specific process is: The top-level cloud assessment is performed using 2B-CLDCLASS data, and the cloud type information is integrated into a single-layer result, such as Figure 2 As shown, only when there are more than two consecutive layers of the same cloud type can the cloud type be determined. In the case of multiple layers of cloud of Dc and Ns cloud types, these locations are classified as Dc or Ns, respectively. For other multi-layer scenes, the cloud type of the highest layer is used. Stratus clouds (St) are usually at a lower altitude and therefore are not well represented in the sample, so they are excluded from this embodiment; FY-4 satellite data preprocessing and feature selection. For the original L1-level data of FY-4, the study area is first cropped, and then the FY-4 data is converted to equal longitude and latitude projection, and then the converted data is interpolated to a grid point with a spatial resolution of 0.02°× 0.02°. Feature channel selection is performed based on the relationship between the spectral characteristics of FY-4 and cloud classification.

[0019] If the AHI point scanning time is within 5 minutes of the CPR scanning time and the distance is within 0.04°, the point is selected as the potential matching point of the CPR point. Then the point with the smallest distance to the CPR point is selected from the potential matching points as the matching point, and a 9×9 neighborhood sample is selected with this point as the center. For the matched data set, the data in the time range of 5:00-18:00 is selected as the training and test data sets.

[0020] S2: Construct a cloud classification network based on the Transformer network, introduce a temporal geocoding module to enhance the model's perception of spatial and temporal information, and add a channel attention mechanism to integrate multiple feature channels of the data to obtain a GTCCT model for cloud classification; The specific process is as follows: This embodiment proposes a method using time coding technology, such as Figure 3As shown in FIG. 1 , the date information is encoded and directly used as the input of the model, so that the model can effectively capture the characteristics of cloud distribution in time. Specifically, this embodiment uses sine and cosine functions to periodically encode the dates of a year. Days, among which (Taking non-leap years as an example), use the following formula to encode the date: ; ; in, and Represents the sine and cosine encoding values ​​of the date respectively. The principle of geocoding technology is as follows Figure 4 As shown, for a given longitude and latitude , first convert the angle to radians, then calculate its sine and cosine: ; ; in, and denote the sine and cosine components of longitude, respectively. and represent the sine and cosine components of latitude respectively.

[0021] The channel attention module (CAM) is used to establish the connection between channels and enhance the specific semantic response capability under the channel. The structure of the channel attention module is as follows: Figure 5 As shown in Figure 2, the channel attention module can analyze and weight different channels in the image data, strengthen key spectral features, and suppress irrelevant spectral features. Finally, the GTCCT cloud classification model is obtained. For a given input feature matrix ,in represents the number of channels of the feature matrix, represents the height of the feature map, Indicates the width of the feature map. First, A is processed through two convolutional layers to obtain feature maps B and C. Then, B and C are re-dimensionalized. The transposed feature maps B and C are multiplied and passed through the softmax layer to calculate the spatial attention feature map S. The size of S is C×C. The specific operation is as follows: ; Then, convolution is performed on A to obtain the feature map D and matrix multiplication is performed on the spatial attention feature map S, which is then multiplied by the scale coefficient , and finally, after dimension rearrangement, the spatial attention feature map is obtained by adding it to the initial feature map A. is the scale coefficient, initialized to 0, and gradually learns to assign larger weights. The specific operation is as follows: ; The main structure of the Transformer encoder is as follows Figure 6 As shown, the traditional Transformer mainly focuses on the modeling of sequence data, while the Transformer encoding module of the present invention targets the spatial and spectral features of cloud samples, integrates multiple attention mechanisms, enhances the feature extraction capability of the model, and includes a total of 4 Transformer encoders. The Transformer encoder of the present invention includes two residual links, which are mainly used to enhance the fluidity of feature information and improve the learning ability and classification performance of the model. Each residual link includes a layer normalization (LN) operation to stabilize the training process and accelerate convergence.

[0022] The first residual link contains the channel attention module (CAM) and the multi-head self-attention mechanism. Through the self-attention mechanism, the model can effectively capture the global contextual relationship between multi-channel features. At the same time, combined with the channel attention function of the CAM module, the model's perception of important features is enhanced.

[0023] The output of the first residual module can be expressed as follows: ; The second residual link contains a feed forward neural network (FFN). The FFN module of the present invention adopts a multilayer perceptron (MLP) structure, which consists of two 1×1 convolutional layers and a ReLU activation function. Its specific function is to extract more complex intra-channel features by expanding and compressing dimensions, and at the same time, combined with residual connections, to ensure the integrity and diversity of information transmission. The formula of the MLP module used in the present invention is as follows: ; The output of this part can be expressed as: ; S3: Use the divided training set to train the GTCCT model, and use the test set to test the classification ability of the GTCCT model: The specific process is as follows: 1) The training and test data sets cover the full observation data of geostationary meteorological satellites throughout 2019 and the cloud classification data of CloudSat, and the classification results of CloudSat are used as labels. All selected channel data are input into the model for training, cross entropy is used as the loss function, and Adam is selected as the optimizer of the model. The constructed training data set is input into the model for training with the dimensions of batch×25×9×9; 2) For the test set, the overall accuracy (OA), recall and Kappa coefficient are used as evaluation indicators to evaluate the effectiveness of the GTCCT model in the cloud classification task of Fengyun-4 satellite observation images. The calculation formulas for each evaluation indicator are: ; in, Indicates that the actual category And is correctly predicted as the category The number of samples, Represents the real type The pixel is predicted by the model to be of type The number of .

[0024] ; ; in, represents the total number of all samples, Representative categories are The closer the Kappa value is to 1, the better the consistency of the prediction.

[0025] S4: Use the trained GTCCT model to perform pixel-by-pixel cloud classification on the satellite cloud image of the selected study area. The classification process in step S4 is as follows: for the Fengyun-4 satellite cloud image of the selected study area, a 9×9 area is taken with each pixel as the center point, and the area is input into the trained GTCCT model for prediction to obtain the cloud classification result of each point.

[0026] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A satellite remote sensing image cloud classification method based on spatiotemporal coding guidance, comprising the following steps: S1: Preprocess the CloudSat satellite data and geostationary meteorological satellite data, and perform spatiotemporal matching to obtain a cloud sample data set; the characteristics are: S2: Construct a cloud classification network based on the Transformer network, introduce a temporal geocoding module to enhance the model's perception of spatial and temporal information, and add a channel attention mechanism to integrate multiple feature channels of the data to obtain a GTCCT model for cloud classification; The construction process of the GTCCT model is: B1: Apply time and geocoding technology to expand the dimension and information content of input data: Encode time information and latitude and longitude information into sine and cosine values ​​and directly input them into the GTCCT model as separate channels, so that the GTCCT model can capture the distribution relationship of cloud types in time and space; Specifically, use sine and cosine functions to periodically encode the dates of the year. Taking a non-leap year as an example, let the first date of the year be Days, among which , use the following formula to encode the date: ； ； in, and Represents the sine and cosine encoding values ​​of a date respectively; geocoding technology for a given longitude and latitude , first convert the angle to radians, then calculate its sine and cosine: ； ； in, and denote the sine and cosine components of longitude, respectively. and denote the sine and cosine components of latitude respectively; B2: Use the channel attention module CAM to establish the connection between channels, analyze and weight different channels in the image data, and finally obtain the GTCCT cloud classification model; For a given input feature matrix ,in represents the number of channels of the feature matrix, represents the height of the feature map, Represents the width of the feature map. First, A is processed by two convolutional layers to obtain feature maps B and C. Then, B and C are re-dimensionalized. The transposed feature maps B and C are multiplied and passed through the softmax layer to calculate the spatial attention feature map S. The size of S is C×C. The specific operation is as follows: ； in, represents the channel index in the feature map B, Represents the channel index in the feature map C; Then, convolution is performed on A to obtain the feature map D and matrix multiplication is performed on the spatial attention feature map S, which is then multiplied by the scale coefficient Finally, after the dimension is rearranged, it is summed with the initial feature map A to obtain the spatial attention feature map ; is the scale coefficient, initialized to 0, the specific operation is as follows: ； B3: For the input 9×9 image, the reshape operation in the multi-head self-attention mechanism MHSA converts the input 9×9 image into a one-dimensional sequence, so that the head self-attention mechanism MHSA can effectively capture the relationship between different pixels in the image; The input sequence is linearly transformed to generate query, key, and value matrices: ； in, represents the original feature dimension, is the dimension of the key vector, is a learnable linear transformation matrix, which is used to map the input sequence to different feature spaces. are the representations of query, key, and value respectively; next, attention is calculated using the following formula: ； Among them, the softmax function is used to normalize the similarity scores so that the sum of the weights is 1, thereby focusing attention on the most relevant pixels; In order to improve the representation ability and generalization performance of the GTCCT model, the multi-head self-attention mechanism MHSA divides the query, key and value into heads, each head has a dimension of ; Each head independently calculates attention to capture features in the input sequence in different subspaces; for the The attention representation is calculated as follows: ； After calculating all the heads, concatenate the outputs of each head to get the aggregation matrix , and generates the final output through a linear transformation: ； in, is the linear projection matrix, which remaps the concatenated output back to the original feature dimension ; S3: Use the divided training set to train the GTCCT model, and use the test set to test the classification ability of the GTCCT model; S4: Use the trained GTCCT model to perform pixel-by-pixel cloud classification on satellite cloud images of the selected study area.

2. The satellite remote sensing image cloud classification method based on spatiotemporal coding guidance according to claim 1, characterized in that: The data preprocessing in step S1 includes: A1: Determination of CloudSat top-layer cloud type: Use 2B-CLDCLASS data to evaluate the top-layer cloud, integrate the cloud type information into a single-layer result, and determine the cloud type of the point only for continuous and more than two layers of cloud types; in the case of multi-layer clouds of Dc and Ns cloud types, the cloud type is determined to be Dc or Ns; for other multi-layer scenes, the cloud type of the highest layer is used; when the stratus cloud type is St, it is not representative enough in the sample and this category is excluded; A2: Geostationary meteorological satellite data preprocessing and feature selection, including clipping, projection conversion and equal longitude and latitude interpolation; A3: Data spatiotemporal matching: First, geostationary meteorological satellite data with a time difference of less than 5 minutes from the CPR scan are screened. Taking the CPR point as the center point, AHI scan points with a distance of less than 0.04° from the center point are screened as potential matching points of the CPR point. Then, the point with the smallest distance from the CPR point is selected from the potential matching points as the matching point, and a 9×9 neighborhood sample is selected with the matching point as the center to obtain a cloud type sample. The data set is divided into training set, validation set and test set.

3. The satellite remote sensing image cloud classification method based on spatiotemporal coding guidance according to claim 1, characterized in that: The network training and testing process in step S3 is as follows: C1: The training and test data sets cover the comprehensive observation data of the geostationary meteorological satellite in a certain year and the cloud classification data of CloudSat, and the classification results of CloudSat are used as labels; all selected channel data are input into the GTCCT model for training, cross entropy is used as the loss function, and Adam is selected as the optimizer of the GTCCT model; the constructed training data set is input into the GTCCT model for training with the dimensions of batch×25×9×9; C2: For the test set, the overall accuracy OA, recall rate Recall and Kappa coefficient are used as evaluation indicators to evaluate the effectiveness of the GTCCT model in the cloud classification task of geostationary satellite observation images; the calculation formulas for each evaluation indicator are: ； in, Indicates that the actual category And is correctly predicted as the category The number of samples, Represents the real type The pixel is predicted by the model to be of type The number of ； ； in, represents the total number of all samples, Representative categories are The closer the Kappa value is to 1, the better the consistency of the prediction.

4. The satellite remote sensing image cloud classification method based on spatiotemporal coding guidance according to claim 1, characterized in that: The classification process in step S4 is as follows: for the geostationary meteorological satellite cloud image of the selected study area, a 9×9 area is taken with each pixel as the center point, and the area is input into the trained GTCCT model for prediction to obtain the cloud classification result of each point.

Images