Multi-temporal pixel cloud removal synthesis method and system based on cloud mask by utilizing classification technology

Abstract

The invention belongs to the field of remote sensing, and discloses a multi-temporal pixel cloud removal synthesis method and system based on a cloud mask by using a classification technology, and the method comprises the steps: carrying out the classification of all daily images through employing an SVM and U-net, and generating two groups of classification grids; converting the classified grids into a cloud mask layer; using a mask layer to mask the corresponding MODIS every day; a maximum method is used to produce an optimal composite product. The product provided by the invention generates the optimal classification precision in three classifiers of RF, SVM and U-net, and proves that the method is effective in the aspect of removing cloud, noise and / or polluted pixels.

Description

Technical Field

[0001] The present invention belongs to but is not limited to the field of remote sensing technology, and in particular relates to a multi-temporal pixel cloud removal synthesis method and system based on cloud mask using classification technology. Background Art

[0002] Remote sensing images such as satellite and aerial images have been an indispensable source of information for various Earth-related applications and studies, including climate and land cover change analysis and environmental and ecological studies. However, the quality of images is affected by several factors, including the quality of the sensor, the sun angle, the viewing angle, the distance between sensors, and the target and atmospheric conditions. Among these factors, clouds are the most challenging features, greatly affecting the quality of remote sensing images. In addition, the presence of clouds in various forms and degrees on the Earth's surface exacerbates this problem, especially in large-area surveys. Despite geographical and seasonal differences, the global average monthly cloud cover can reach 35%, with Central Europe, Central Africa, South America and Southeast Asia being areas with persistent cloud cover, of which Africa is the most prone to cloud cover. In general, everything from thin / transparent clouds to thick clouds associated with shadows can significantly affect the quality of images.

[0003] The impact of clouds on image quality also varies depending on the type of cloud and the wavelength / band in which the sensor captures the image. Thick clouds can completely block the incoming radiation from the source and / or the radiation reflected back to the optical sensor, while thin / transparent clouds can reduce the amount of incoming radiation and contaminate the actual reflectivity value of the object, resulting in incorrect values. In particular, the electromagnetic spectrum in the visible, near-infrared (NIR) and mid-infrared (MIR) ranges are the most sensitive regions; while longer wavelength parts, such as thermal infrared and cirrus clouds, are relatively less affected by clouds. In contrast, the Synthetic Aperture Radar (SAR) bands of active sensors are not affected by clouds and can easily penetrate them.

[0004] Various cloud masking and removal methods have been developed and implemented. Some are specialized for certain cloud types, such as thin or thick cloud removal, while others propose a unified approach that aims to address both types of problems. These methods can be roughly divided into traditional and advanced deep learning methods based on the models and mechanisms used to reconstruct the missing information. Traditional techniques can also be divided into three types based on the type of axillary information and the assumptions used: spatial, spectral, and multi-temporal.

[0005] Based on the concept that polluted areas and cloud-free neighboring pixels have similar properties, spatially informed cloud removal methods utilize information from neighboring cloud-free areas to replace cloud pixels. While this assumption works well for cleaning small turbid areas, especially in a single image, it is not valid in large cloudy and / or areas with uneven land cover that varies over small distances.

[0006] On the other hand, spectral-based methods recover the lost / contaminated information in the image using other unaffected spectral information / bands of the same area and time. Although this technique can be used for thin cloud cover screening, it fails to achieve satisfactory results in situations where all multispectral channels are affected by clouds, especially when the cloud layer is thick, or when there is no spectral band correlation export between the contaminated and unaffected bands. However, multitemporal methods reconstruct the missing data using information collected during a specific time, with the concept that the target feature does not exhibit significant spectral or phenological changes during the specified time. This technique can be effectively applied if high temporal resolution, sufficient cloud-free time series data are available.

[0007] Recently, deep learning (DL) techniques, especially generative adversarial network (GAN) methods, have been widely applied in cloud and related shadow identification and removal and have achieved state-of-the-art results. However, these techniques have some significant limitations that affect their flexibility. Most of these techniques require the integration of auxiliary data such as SAR, which can introduce bias or result in blurred outputs, and some use infrared bands, which are also difficult to obtain uncontaminated pixels in thick and cloudy conditions. In addition, most DL techniques using SAR bands aim to remove thick clouds from monospectral or multispectral images (RGB images), while ignoring the multi-temporal intrinsic features of images contaminated by thick clouds. In addition, these techniques require cloud masks, which depend on the efficiency of cloud detection algorithms, which are mostly data-specific and still far from accurate. In addition, DL models are specific to certain data types, cloud conditions, and spatial extents, which limit their generalization and are prone to overfitting, or underfitting if insufficient training samples are available. Therefore, it is crucial to consider an alternative cloud removal method that can address the above limitations, especially for studies involving large areas with continuous cloud cover. In this regard, pixel-based synthesis techniques are the best choice. Not only because it is relatively simple and less complex compared to DL methods, but also because it produces the best results.

[0008] Pixel-based multi-temporal compositing of images over a period of time is one of the most commonly used and effective mechanisms for removing clouds and associated shadows, especially for moderately coarse resolution images captured by high temporal resolution sensors. In addition to removing or reducing clouds, noise, and artifacts, this technique also helps reduce the amount of data by selecting the best pixels from the available time series data to replace the contaminated areas in the scene. This technique is one of the classic methods for reconstructing lost information and is basically applicable to instruments with long revisit periods, regardless of the surface area. So far, several composting methods have been proposed using single, multiple, or spectral similarity criteria. Single rule-based methods use a single criterion, such as the maximum normalized difference vegetation index (MAX-NDVI), the median NIR band (MED-NIR), the maximum NDVI or maximum brightness temperature (BT) of WELD (Web-enabled Landsat data), and the maximum ratio of near-infrared (NIR) to blue bands (MAX-RNB). On the other hand, the Best Available Pixel (BAP), Phenological Adaptive Composite (PAC), Weighted Parameter Score (WPS), Measurement of Central Points (MEDOID), and National Land Cover Database (NLCD) algorithms utilize multiple criteria to select the best pixels, while cosine similarity (COSSIM and spectral similarity) employs spectral similarity.

[0009] Although these methods can produce high-quality composite products, their efficiency depends on the effectiveness of the cloud mask, synthesis criteria, and cycle. Therefore, there is no single optimal composite method that outperforms all others. In addition, most of the composite techniques are specific to certain types of sensors, employing specific cloud masks generated for them. For example, BAP, PAC, MAX-RNB, WPS, MEDOID, and NLCD were developed using Landsat imagery, and the cloud masks were generated using the mask function (Fmask) algorithm, which has its own limitations in identifying thin clouds and cloud edges and shadows. Prior to the development of the Fmask algorithm, supervised classification methods, particularly decision tree algorithms, were applied to create cloud masks for Landsat image composites. On the other hand, low- and medium-resolution schemes, such as the Advanced Very High Resolution Radiometer (AVHRR) and Moderate Resolution Imaging Spectroradiometer (MODIS) cloud masking algorithms, employ thresholding mechanisms on various spectral bands to generate composite products. However, thresholding methods are never global, as such commission and / or omission errors are common, especially near the threshold limits, where the confidence in cloud detection is low.

[0010] In view of the above analysis, the technical problems that need to be solved urgently in the prior art are:

[0011] Inspired by the existing limitations of cloud removal and pixel-based synthesis methods, inherently related to the large spatial extent and multi-temporal data from high temporal resolution sensors, a new single rule-based synthesis method is urgently needed. Summary of the invention

[0012] In view of the problems existing in the prior art, the present invention provides a multi-temporal pixel cloud removal and synthesis method and system based on cloud mask using classification technology.

[0013] The present invention is implemented as follows: a multi-temporal pixel de-clouding synthesis method based on a cloud mask using a classification technology, characterized in that the multi-temporal pixel de-clouding synthesis method based on a cloud mask using a classification technology specifically comprises:

[0014] S1: Use SVM and U-net to classify all daily images and generate two sets of classification grids;

[0015] S2: Convert the classified raster into a cloud mask layer;

[0016] S3: Use the mask layer to mask the corresponding MODIS every day;

[0017] S3: Use the maximum method to produce the best composite product.

[0018] Further, S2 converts the classified raster into a cloud mask layer by mapping the values ​​of cloud and no-data pixels to -0.999999.

[0019] Further, the S3, by using a special function 'np.where' in Python, uses the mask layer to mask the corresponding MODIS for each day, and the resulting mask image has a value of -0.999999 pixel, indicating turbidity / no data value, if the pixel is masked off, otherwise the pixel keeps its actual reflectance value.

[0020] Further, the S4 generates two sets of maximum composite products MaxComp-1 and MaxComp-2, which are associated with two sets of cloud mask layers generated using the SVM and U-net models, respectively.

[0021] Another object of the present invention is to provide a multi-temporal pixel cloud removal synthesis system based on cloud mask using classification technology, the system specifically comprising:

[0022] Input module, used to collect input data, divide training sets and test sets, and select models;

[0023] Processing module for generating 16-day composite products;

[0024] Evaluation module, used to evaluate product performance.

[0025] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0026] First, the product of the present invention produced the best classification accuracy among the three classifiers of RF, SVM and U-net, proving the effectiveness of the present invention in removing cloud, noise and / or polluted pixels.

[0027] The cloud mask-based multi-temporal pixel declouding synthesis method provided by the present invention solves many key problems in the prior art by utilizing classification technology and multi-temporal data processing technology, and has achieved significant technical progress in industrial applications.

[0028] 1. Technical problems solved:

[0029] Image quality degradation caused by cloud interference: In satellite remote sensing images, clouds often cover the surface, affecting the acquisition of surface information. Traditional image processing methods are difficult to effectively eliminate the interference of clouds on observation results. This invention introduces SVM and U-net classification technology to accurately identify and classify cloud areas and generate cloud masks, successfully solving the problem of cloud interference and improving image clarity and accuracy.

[0030] Efficiency issues of multi-temporal image processing: Traditional methods often face problems such as low processing efficiency, large data volume, and high computational complexity when processing multi-temporal satellite images. This invention effectively simplifies the image processing process, reduces computational costs, improves processing efficiency, and can quickly generate high-quality cloud-free composite images by combining multi-temporal data with the maximum synthesis method.

[0031] The elimination of cloud shadows and noise is not thorough enough: In the prior art, although the cloud layer can be eliminated to a certain extent, cloud shadows and noise interference often still exist, affecting the quality of the final image. The present invention effectively removes cloud shadows and noise in multi-temporal images by using the maximum synthesis method, making the final composite image clearer and more reliable.

[0032] 2. Significant technological progress achieved:

[0033] Innovative application of classification technology: The present invention innovatively combines SVM and U-net classification technology, utilizing both the precise classification capability of SVM and the deep learning capability of U-net to achieve high-precision recognition and segmentation of cloud areas. Through this innovative application, the present invention significantly improves the generation accuracy of cloud masks, providing a solid foundation for subsequent cloud removal processing.

[0034] Efficient fusion of multi-temporal images: The maximum value synthesis method proposed in this paper effectively avoids the influence of clouds by selecting the pixel with the largest value in the multi-temporal images for fusion, ensuring the cloud-free effect of the composite image. This method significantly improves processing efficiency while improving image clarity, and is particularly suitable for the rapid processing of large-scale remote sensing data.

[0035] Broad industrial application prospects: The method of the present invention is not only applicable to conventional remote sensing image processing, but can also be applied to multiple fields such as agricultural monitoring, environmental protection, and urban planning. By providing high-quality cloud-free images, the present invention can support more accurate surface analysis and decision-making, provide important data support for related industries, and has broad application prospects.

[0036] Significant improvement in processing accuracy and reliability: By combining the accurate generation of cloud masks with the maximum value synthesis method, the present invention significantly improves the processing accuracy and reliability of multi-temporal images. Compared with the prior art, the present invention can generate high-quality composite images under more complex environmental conditions, meeting the industry's demand for high-precision remote sensing data.

[0037] In summary, the present invention has made significant technical progress in the field of cloud removal processing of multi-temporal remote sensing images, successfully solved the problems in the prior art such as cloud interference, low processing efficiency and incomplete noise elimination, and provided more advanced and effective technical means for industrial applications.

[0038] Second, current clouds and associated shadows are a major challenge in remote sensing image processing, which seriously affects the quality of remote sensing images, which are crucial for applications in the fields of climate, land cover change analysis, environmental and ecological research, etc. Although a variety of cloud removal methods have been developed, they still face some limitations.

[0039] 1) Limitations of existing cloud removal methods

[0040] Removal of specific cloud types: Many existing cloud removal methods are only applicable to specific types of clouds. For example, some methods specialize in thin clouds or thick clouds, but do not work well on other types of clouds.

[0041] Dependence on specific images or sensors: Most existing cloud removal methods are developed for certain types of images or sensors, such as Landsat images or MODIS images, which makes these methods less applicable across different sensors.

[0042] Requirement of auxiliary data: Some techniques require the use of additional auxiliary data to recover pixels occluded by clouds, which increases the application complexity.

[0043] Limitations of the Fmask algorithm: Although the Fmask algorithm has been widely used in cloud detection, it still has some limitations in detecting thin clouds, cloud edges, and shadows, especially near the threshold limit, where the confidence of cloud detection is low.

[0044] 2) Limitations of existing synthesis technology

[0045] Sensor-specific techniques: Most of the current synthesis techniques are based on a specific type of sensor. For example, techniques such as BAP, PAC, and MAX-RNB rely on Landsat images and use the Fmask algorithm to generate cloud masks, and data from other sensors are not applicable.

[0046] Limitations of decision tree methods: Before the Fmask algorithm, supervised classification methods such as decision trees were used to generate cloud masks, but these methods are often limited to specific types of images and sensors when using synthesis mechanisms.

[0047] Threshold-based cloud detection: Low- and medium-resolution images (such as AVHRR and MODIS) usually rely on spectral band threshold mechanisms to detect clouds. However, the threshold method is not globally applicable and often suffers from misjudgment or omission problems when close to the threshold.

[0048] 3) Innovation of the present invention

[0049] In view of the limitations of the above existing technologies, we propose a multi-temporal pixel cloud removal synthesis method based on classification technology and cloud mask. This method is not only flexible, simple and does not require auxiliary data, but also has the following significant features:

[0050] Strong versatility: The synthesis technology of the present invention does not depend on a specific sensor type, spatial range, data volume or cloud type, and is applicable to various high-resolution remote sensing images.

[0051] High-quality composite images: Cloud masks are generated through classification techniques, and the maximum synthesis algorithm is used to select the pixels with the highest reflectivity from each band to generate high-quality composite images without clouds or noise.

[0052] Effective removal of clouds, shadows and noise: Our method can not only effectively remove clouds, but also remove related shadows and noise at the same time to improve image quality.

[0053] No auxiliary data required: Compared with other methods, our algorithm does not require the use of additional auxiliary data, which greatly simplifies the operation process.

[0054] 4) Maximum value synthesis algorithm

[0055] Difference from traditional methods: Traditional maximum composite (MVC) algorithms usually rely on the maximum NDVI value to select the best pixels, while our algorithm generates the best composite image by selecting the pixels with the highest reflectivity, avoiding the interference of cloud, noise and polluted pixels.

[0056] Higher cloud detection and removal accuracy: Compared with the Fmask algorithm and threshold-based methods, the present invention adopts SVM and U-net classification technology to more accurately identify clouds and remove clouds and related shadows and noise through a synthesis process.

[0057] The technical solution of the present invention solves the problem of cloud interference in the process of multi-temporal remote sensing image processing by introducing a pixel-level synthesis technology based on cloud masks, and realizes efficient and cloudless remote sensing image synthesis, which has broad application prospects in the fields of climate change monitoring and ecological research. Third, the technical solution of the present invention solves the technical problem of loss of surface information caused by cloud interference in the process of multi-temporal remote sensing image processing in the prior art in industrial applications, and has achieved significant technical progress. When dealing with cloud occlusion, the existing multi-temporal image synthesis method is often unable to accurately identify the cloud area, resulting in the obscuration of surface information, affecting the subsequent analysis and monitoring of surface changes. The present invention combines the classification technology of support vector machine (SVM) and U-net network to effectively identify and classify cloud areas and generate accurate cloud masks, thereby greatly improving the accuracy of cloud detection.

[0058] In addition, the existing technology easily leads to reduced image quality during the cloud masking process, especially when the clouds in the multi-temporal image are dense, the synthesized image information is lost. The present invention uses innovative multi-temporal cloud masking and maximum synthesis technology to filter out the best pixel value from the multi-temporal image, thereby generating the best cloud-free composite image. This method not only effectively removes cloud interference, but also retains the surface detail information to the greatest extent, improving the image quality and data utilization.

[0059] Significant technological advances are reflected in:

[0060] 1) Improve cloud detection accuracy: By combining the deep learning model of SVM and U-net, the present invention can achieve more accurate cloud layer detection and classification, reduce misclassification and missed classification, and ensure that the generated cloud mask layer is more reliable.

[0061] 2) Optimize data synthesis effect: By adopting the maximum synthesis method and selecting the optimal value of each pixel, the generated composite image has higher clarity and better cloud removal effect, which effectively improves the practicality of the image.

[0062] 3) Enhanced data applicability: The method of the present invention is applicable to different types of remote sensing data, especially MODIS data, and has high flexibility and application prospects in a wide range of remote sensing monitoring and analysis applications.

[0063] 4) Reduce data loss: Compared with traditional methods, the present invention can remove cloud interference while retaining more surface information, significantly improving the integrity and quality of the image.

[0064] Therefore, the present invention not only solves the problem of cloud interference in remote sensing image processing, but also improves the accuracy and efficiency of image synthesis, and has important technical value and industrial promotion prospects in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a flow chart of a multi-temporal pixel cloud removal synthesis method based on cloud mask using classification technology provided by an embodiment of the present invention;

[0066] Figure 2 It is a module diagram of a multi-temporal pixel cloud removal synthesis system based on cloud mask using classification technology provided by an embodiment of the present invention;

[0067] Figure 3 It is the comparison result between the product of the present invention provided by the embodiment of the present invention and MOD13A;

[0068] Figure 4 It is the result of land cover classification using three synthetic products and three different models provided by the embodiment of the present invention;

[0069] Figure 5 Confusion matrices generated using our best composite product (MaxComp-1) (column 1) and the reference composite product (MOD13A1) (column 2) by using three different classifiers. (a) Input: MaxComp-1, model: RF; (b) Input: MOD13A1, model: RF; (c) Input: MaxComp-1, model: SVM; (d) Input: MOD13A1, model: SVM; (e) Input: MaxComp-1, model: U-net; (f) Input: MOD13A1, model: U-net. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0071] like Figure 1As shown, an embodiment of the present invention provides a multi-temporal pixel cloud removal synthesis method based on a cloud mask using a classification technology, and the method specifically includes:

[0072] S1: Use SVM and U-net to classify all daily images and generate two sets of classification grids;

[0073] S2: Convert the classified raster into a cloud mask layer;

[0074] S3: Use the mask layer to mask the corresponding MODIS every day;

[0075] S3: Use the maximum method to produce the best composite product.

[0076] The multi-temporal pixel cloud removal synthesis method based on cloud mask provided in the embodiment of the present invention utilizes classification technology, and the specific working principle is as follows:

[0077] 1. Classification and grid generation: First, all daily images are classified using support vector machines (SVM) and U-net neural network models. SVM achieves preliminary classification of clouds and other elements in the image by learning the decision boundary in the feature space; while U-net performs more detailed segmentation and classification of images through its encoder-decoder architecture. Two sets of classification grids are generated through these two technologies, which respectively identify the areas of clouds, ground and other categories to ensure the accuracy of the classification results.

[0078] 2. Raster conversion to cloud mask layer: Convert the raster data generated by classification to a cloud mask layer. The cloud mask layer is a binary image where "1" represents pixels detected as clouds and "0" represents ground or non-cloud pixels. This step converts the classification results of the original image into a format suitable for cloud removal operations, so that masking operations can be performed in subsequent steps to filter out cloud interference.

[0079] 3. Cloud masking and daily image processing: For the MODIS image data acquired every day, the corresponding images are masked using the generated cloud mask layer. The masking process excludes pixels identified as clouds by shielding them, retaining only clear non-cloud pixels, ensuring that the de-clouded images retain the highest quality visible information.

[0080] 4. Generate the best composite product: After masking, the multi-temporal images are processed using the maximum value synthesis method. This method selects the maximum value of each pixel in all time series images to generate the best cloud-free composite product. This method can effectively eliminate the influence of clouds while retaining the complete information of the surface, achieving high-quality multi-temporal pixel synthesis.

[0081] This method makes full use of machine learning technology for cloud detection, and through reasonable masking and synthesis strategies, ensures that cloud interference in multi-temporal images is effectively removed to generate high-quality composite image products.

[0082] The multi-temporal pixel cloud removal synthesis method based on cloud mask provided by the present invention uses classification technology to process multi-temporal satellite images to eliminate cloud interference and generate clear composite images. The specific working principle includes the following steps:

[0083] First, S1: Use SVM and U-net to classify daily images and generate classification grids. Support vector machine (SVM) and U-net are two classic image classification techniques. SVM can effectively distinguish between cloud and surface information by finding the optimal hyperplane to classify pixels in the image. U-net, as a convolutional neural network (CNN), can accurately locate cloud boundaries and generate cloud masks by performing multi-layer convolution and deconvolution on the image. These two methods classify the satellite images acquired daily and generate two sets of classification grids, where each grid cell corresponds to a classification result, such as cloud or surface.

[0084] Next, S2: convert the classified raster into a cloud mask layer. After the classification is completed, the cloud area in the raster is marked and converted into a cloud mask layer. These cloud mask layers are binary images, where the area with a value of 1 represents the cloud layer, and the area with a value of 0 represents the cloudless surface. By converting the classified raster into a cloud mask layer, the distribution of the cloud layer can be intuitively represented, which is convenient for the masking operation of the cloud layer in the subsequent processing steps.

[0085] Then, S3: use the mask layer to mask the daily MODIS image. MODIS (Moderate Resolution Imaging Spectroradiometer) is a commonly used medium-resolution imaging spectrometer, which is widely used in surface and atmospheric observations. In this step, the generated cloud mask layer is used as a mask layer and applied to the daily MODIS image frame by frame. This operation is equivalent to covering the image with a transparent film, so that the cloud area is blocked and only the surface information is retained, thereby avoiding the interference of clouds on the surface observation results.

[0086] Finally, S4: Use the maximum method to produce the best composite product. In the multi-temporal images, each day's image has been processed by cloud masking, but there are still a small amount of cloud shadows or noise. For this reason, the maximum synthesis method is used to comprehensively process these images, that is, at the same pixel position in multiple images, the one with the largest pixel value is selected as the final composite pixel value. Since cloudless surfaces are usually more reflective than clouds, maximum synthesis can effectively remove cloud shadows and noise, and finally generate a cloudless, clear surface composite image suitable for further surface analysis and remote sensing applications.

[0087] Through the above steps, the present invention realizes cloud removal processing of multi-temporal satellite images, significantly improves the quality of surface observation data, and provides more reliable data support for subsequent environmental monitoring and geographic analysis.

[0088] Embodiments of the present invention propose an alternative satellite image synthesis strategy to circumvent the challenge of obtaining cloud-free scenes, especially for extensive areas with known persistent cloud cover.

[0089] A large study area was systematically selected, including Central, East and North Africa and part of the Middle East. The region is characterized by a range of climatic conditions, including tropical rainforests known for persistent cloud cover, arid / desert and equatorial climates, as well as various land cover features. We then acquired a total of 432 MOD09GAL2 daily MODIS datasets, including seven bands, over three seasons. Subsequently, we collected representative reference samples (training and test datasets), and the target area was classified into three categories: cloud, land (all uncontaminated pixels) and no data. Using the reference samples, three robust ML classifiers, RF, SVM and U-net, were trained and tested to select the best classifier and evaluate the impact of the algorithm on the final composite product. Therefore, SVM and U-net were selected and used to classify all daily images, and two sets of classified rasters were generated.

[0090] The S2 converts the classified raster into a cloud mask layer by mapping the values ​​of cloud and no-data pixels to -0.999999.

[0091] The S3, by using a special function 'np.where' in Python, masks the corresponding MODIS for each day using a mask layer, and the resulting mask image has a value of -0.999999 pixel, indicating a turbidity / no data value, if the pixel is masked off, otherwise the pixel keeps its actual reflectance value.

[0092] After masking clouds and no-data pixels, we generated 16-day composite products using three methods: maximum, median, and mean methods, and visually compared the products to select the best method. The product comparison showed that the maximum method produced the best product.

[0093] Said S4, generates two sets of maximum composite products MaxComp-1 and MaxComp-2, which are associated with two sets of cloud mask layers generated using SVM and U-net models, respectively.

[0094] like Figure 2 As shown, an embodiment of the present invention provides a multi-temporal pixel cloud removal synthesis system based on cloud mask using classification technology, which specifically includes:

[0095] Input module, used to collect input data, divide training sets and test sets, and select models;

[0096] Processing module for generating 16-day composite products;

[0097] Evaluation module, used to evaluate product performance.

[0098] Two sets of maximum composite products, MaxComp-1 and MaxComp-2, were generated, associated with two sets of cloud mask layers generated using SVM and U-net models, respectively, to evaluate the quality of the products for land cover mapping and compare the results with the MOD13A1v061L3 16-day composite product provided by NASA. To make a fair comparison, we compared with all products obtained during the same period containing the same type and number (5) of bands: MVC-NDVI, Band 1 (red), Band 2 (near infrared), Band 3 (blue) and Band 7 (MIR) ( Figure 3 ).

[0099] like Figure 4As shown, in terms of land cover classification accuracy, the composite product of the present invention produced the best classification accuracy among the three classifiers of RF, SVM and U-net, which means the effectiveness of the technology of the present invention in removing cloud, noise and / or polluted pixels. In particular, the MaxComp-1 product of the present invention produced the best classification results among the three classifiers with the smallest accuracy difference. It is worth noting that when using the MaxComp-1 product, the RF and SVM classifiers showed a 5% accuracy change, which is significantly lower than the 16% and 28% accuracy difference between the two classifiers when the Max-Comp-2 and reference MOD13A1 products were multiplied. In addition, the largest change between OA and kappa scores (11%) and the smallest k (36%) were observed when the reference MOD13A1 was processed using SVM, while the second smallest k (51%) was observed when processing MaxComp-2. These differences in accuracy strongly indicate that these two products contain more polluted pixels than Max-Comp-1 and therefore have stronger heterogeneity. Furthermore, visual inspection shows that the MaxComp-1 product is more consistent both spectrally and radiometrically, with less noise and fewer artifacts / salt and pepper effects, which reaffirms the highest quality of this product and the efficacy of the method of the present invention in generating high-quality composite products. Furthermore, the highest quality composite product, MaxComp-1, obtained using mask layers derived from daily MOD09GA images classified by SVM shows that SVM is the best model for cloud mask generation, i.e., classification of clouds, land (uncontaminated pixels), and no-data pixels, compared to RF and U-net.

[0100] As shown in Table 1 and Figure 5 The land cover classification results of the three different classifiers are shown in Figure 2. Our composite products MaxComp-1 and MaxComp-2 have significantly higher accuracy (5-28%) than the reference MOD13A116-day composite product. Although our two products outperform the reference composite product in all classifiers, there are significant differences in accuracy within the same classifier and between the three different classifiers, which indicates differences in the quality and / or characteristics of the input datasets and classifiers.

[0101] When applying the RF classifier, our composite products (MaxComp-1 and MaxComp-2) achieved a maximum accuracy (OA) of 0.80 and a kappa coefficient / score (k) of 0.76, which are 5% and 6% higher than the best accuracy achieved by the reference product, respectively.

[0102] However, when using the SVM classifier, significant performance differences were found between the products. MaxComp-1 had an OA of 0.75 and a k of 0.70, which are 28% and 34% higher than the OA and k values ​​of the reference composite data, respectively, while MaxComp-2 had an OA and k that were 16% and 19% lower than MaxComp-1, respectively. Similarly, when employing the U-net model, MaxComp-1 outperformed both MaxComp-2 and the reference composite data, although the change in accuracy was not as significant as when using SVM. MaxComp-1 had an OA = 0.73 and a k = 0.68, which are 8% and 11% higher than the OA and k of the reference composite data, respectively, but 3% and 5% higher than the OA and k of MaxComp-2, respectively.

[0103] The results show that MaxComp-1 achieves the highest classification accuracy among all three classifiers, demonstrating the quality of the product. In addition, the quantitative evaluation results are strengthened by a qualitative analysis based on visual inspection of the composite images from the three seasons. Figure 3 As shown, the MaxComp-1 composite image is more homogeneous, i.e., less noisy, than the MaxComp-2 and reference composite products, which can be clearly seen after zooming in. In contrast, the MaxComp-2 and reference composite products show significant heterogeneity, which is due to the prevalence of nosy pixels and, in some places, salt and pepper (see Figure 3 b, c, e, f, h and i).

[0104] In addition, the changes in the quality of the input dataset are reflected in the Figure 5 For example, when using MaxComp-1 as input, the land cover maps generated by the three classifiers (RF, SVM, and U-net) show minor changes (see Figure 4 a, d, g), where all eight classes are predicted. However, the land cover maps generated by MaxComp-2 and the reference composite product (MOD13A1) show significant differences between the classifiers, including missing classes (see Figure 4 b, c, e, f, h, and i). In particular, when the SVM classifier was used for MaxComp-2 classification, only 7 out of 8 classes were predicted and 1 class (water body) was misclassified as bare sparse vegetation (see Figure 4 e). Similarly, when the reference composite product (MOD13A1) was classified using the SVM model, only 6 out of 8 classes were predicted, i.e. 2 classes (built wetlands and herbaceous wetlands) were completely missed (see Figure 4 f).

[0105] Table 1.Land cover classification performance of our compositeproducts and Reference composite product(MOD13A1) on three models.

[0106]

[0107] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0108] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A multi-temporal pixel cloud removal synthesis method based on cloud mask using classification technology, characterized in that: The method specifically includes: Support vector machine (SVM) and U-net are used to classify the satellite images acquired daily and generate two sets of classified grids, where each grid cell corresponds to a classification result; The classified raster is converted into a cloud mask layer, the cloud area is marked, and a binary image is generated. The area with a value of 1 represents the cloud layer, and the area with a value of 0 represents the cloudless surface. The generated cloud mask layer is used as a mask layer and applied to the daily MODIS image to block the cloud area; The maximum value synthesis method is used to fuse the multi-temporal images. The pixel with the largest value at the same pixel position is selected as the final composite pixel value to generate a cloud-free composite image.

2. The multi-temporal pixel declouding synthesis method based on cloud mask according to claim 1, characterized in that: The SVM classifies the pixels in the image by finding the optimal hyperplane to distinguish between cloud and surface information; the U-net accurately locates the cloud boundary of the image and generates a cloud mask through multi-layer convolution and deconvolution processing.

3. The multi-temporal pixel declouding synthesis method based on cloud mask according to claim 1, characterized in that: The cloud mask layer is generated by mapping the values ​​of cloud and no-data pixels in the classified raster to -0.999999, and is used to identify the masked pixels in the masking operation.

4. The multi-temporal pixel declouding synthesis method based on cloud mask according to claim 1, characterized in that: The MODIS image is pixel masked using the 'np.where' function in Python. The masked pixel values ​​are set to -0.999999, indicating turbid or data-free pixels. Unmasked pixels retain their actual reflectance values.

5. The multi-temporal pixel cloud removal synthesis method based on cloud mask according to claim 1, characterized in that: The maximum value synthesis method is used in combination with the cloud mask layer generated by SVM and U-net classification to generate two sets of maximum value composite products MaxComp-1 and MaxComp-2, respectively, which are used to evaluate the land cover classification accuracy.

6. The multi-temporal pixel declouding synthesis method based on cloud mask according to claim 1, characterized in that: The maximum composite product generated by the described method was evaluated for its effectiveness in removing cloud, noise, and polluted pixels by comparison with the MOD13A1v061 L3 16-day composite product. The generated MaxComp-1 product was more consistent in spectral and radiometric measurements, with less noise, fewer artifacts, and the highest land cover classification accuracy.

7. A multi-temporal pixel cloud removal synthesis system based on cloud mask using classification technology as described in claims 1-6, characterized in that: The system specifically includes: The input module is used to collect input multi-temporal satellite image data, divide it into training set and test set, and select support vector machine (SVM) and U-net model for image classification; A processing module is used to perform classification operations, generate classification grids, and convert them into cloud mask layers, and further apply the cloud mask layers as mask layers to MODIS images to block cloud areas; The synthesis module is used to perform maximum synthesis on the masked image to generate a cloud-free composite image product.

8. The system of claim 1, wherein: The processing module further comprises: The classification submodule uses SVM and U-net to classify the satellite images acquired daily and generate classified grids containing cloud and surface information; The conversion submodule converts the generated classified raster into a cloud mask layer, marks the cloud area, and uses it for the masking operation of the MODIS image.

9. The system according to claim 1, characterized in that The synthesis module further comprises: The pixel fusion submodule fuses the multi-temporal images through the maximum synthesis method, selects the largest pixel value at the same pixel position as the final composite pixel value, and generates a cloud-free composite image; The evaluation submodule is used to evaluate the generated maximum composite product, compare it with the reference product, and analyze its effect in removing clouds, noise and polluted pixels.

10. The system according to claim 1, wherein: The input module performs pixel masking in an automated manner by working with a specific function 'np.where' integrated in Python to map the values ​​of cloud and no-data pixels to -0.999999, and performs classification, conversion, and masking operations in the processing module.

Images