Weak supervision long time sequence land cover mapping method based on remote sensing image

Abstract

The invention discloses a remote sensing image-based weak supervision long time sequence land cover mapping method, which comprises the following steps of: acquiring a low-resolution product, displaying the low-resolution product in a high-resolution image form, collecting training data from the high-resolution image, training a classifier through the training data, and mapping the low-resolution product and the high-resolution image. Predicting the land cover category of the target area through the trained classifier to obtain a land cover category label of the high-resolution image of the target area; correcting a label containing noise to obtain an optimized land cover category label; performing detail verification and time sequence consistency check on the optimized land cover category label to obtain a determined land cover mapping result; according to the method, model training is carried out by using the limited high-resolution image original land cover labels, manual labeling time and cost are saved, the labels containing noise are optimized, detail verification and time sequence consistency check are carried out on the labels obtained after optimization, and the accuracy of land cover category classification can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image information processing in computer vision, and in particular relates to a weakly supervised long-term surface coverage mapping method based on remote sensing images. Background Art

[0002] Surface cover categories include vegetation, cultivated land, lakes, impermeable layers, ice and snow or snow-capped mountains, various buildings such as roads and houses, etc. Surface cover classification has always been the core and focus of global change and modern geological research. It can not only provide a data basis for the rational use of land, but also provide an important basis for land use planning.

[0003] Most of the existing surface cover databases are surface cover category maps using low-resolution images. With the development of aerospace technology in recent years, more and more remote sensing data sources have been available, and the spatial, spectral, radiation, and temporal resolutions of remote sensing images have also become higher and higher. However, the existing high-resolution image surface cover mapping methods are too complicated.

[0004] In the existing method of high-resolution image surface cover mapping through artificial intelligence, it is necessary to annotate a large number of labels of high-resolution image surface cover categories, and then collect training data from the high-resolution images to train the classification model of the surface cover categories. The annotation cost is high and the procedure is cumbersome. Therefore, the present invention proposes a method for high-resolution image surface cover mapping by combining low-resolution image surface cover data and weak supervision learning to improve the above problems. Summary of the invention

[0005] The present invention aims to provide a weakly supervised long-term land cover mapping method based on remote sensing images to determine the land cover categories of high-resolution images.

[0006] To achieve the above objectives, the technical solution of the present invention is:

[0007] A weakly supervised long-term land cover mapping method based on remote sensing images includes the following steps:

[0008] S1: Obtain a low-resolution product, display the low-resolution product as a high-resolution image, collect training data from the high-resolution image, train a classifier using the training data, predict the land cover category of the target area using the trained classifier, and obtain a land cover category label of the high-resolution image of the target area;

[0009] S2: Correct the noisy land cover category labels in step S1 to obtain optimized land cover category labels;

[0010] S3: Use the HRNet model to verify the optimized land cover category labels in step 2 to obtain the verified land cover mapping results Where 0, 1…n represent different land cover categories. is the ground cover category of the tth image block;

[0011] S4: Using a time series filtering method to perform a time series consistency check on the land cover mapping result of step S3 to obtain a confirmed land cover mapping result, specifically including:

[0012] Let the sliding time series window size T w =3, step size is 1, from The image block starts to slide backwards, and the temporal consistency Prob of the central pixel of the sliding temporal window is calculated. i value,

[0013]

[0014] Where L i and L j are the land cover category labels of the central pixel and other pixels in the time series window respectively; con(·) is the consistency function. If L j =L i If i0 is equal to 1, it returns 1, otherwise it returns 0; i0 and i w are the indices of the first and last pixels in the sliding window;

[0015] If Prob i >0.5, the center pixel remains unchanged, and the surface coverage category of the original image block remains unchanged; if Prob i If <0.5, calculate the Prob of other pixels in the original sliding window i value, if other pixels satisfy Prob i >0.5, the land cover label of the central pixel is corrected to the land cover category label corresponding to the other pixels. If the other pixels do not meet Prob i >0.5, the center pixel value remains unchanged, the surface coverage category of the original image block remains unchanged, and the sliding window continues to slide forward. When the sliding window reaches the end of the time series, let T w =5 and then from The image block begins to continue the correction process of this step.

[0016] Furthermore, the S1 step includes:

[0017] S1.1: Obtain a low-resolution product, which is historical remote sensing image data of a certain area. The entire image of the low-resolution product has been divided into multiple image blocks, and the surface cover category label of each image block has been defined. The low-resolution product is upsampled by the nearest neighbor interpolation method. After upsampling, the low-resolution product is generated into a high-resolution image, and the low-resolution product is matched with the corresponding high-resolution image. Then, the label corresponding to the low-resolution product, i.e., the surface cover category, is assigned to the high-resolution image, and training data is collected from the high-resolution image. The training data includes a feature X and a label Y, wherein the feature X refers to a pixel spectral value selected from the high-resolution image, and the label Y refers to the surface cover category corresponding to the pixel spectral value. Each feature X has a corresponding label Y;

[0018] S1.2: using the data collected in step S1.1 to train a classifier, the classifier is used to predict the label Y corresponding to the surface cover category of the image block according to the pixel spectral value X of the image block;

[0019] S1.3: Obtain the pixel spectral values ​​of the high-resolution image blocks of the target area, and use the classifier trained in step S1.2 to predict the surface cover category of the high-resolution image of the target area based on the pixel spectral values ​​to obtain the surface cover label of the high-resolution image of the target area.

[0020] Furthermore, collecting training data in the S1.1 step includes using sparse sampling and class balanced sampling to collect training data from high-resolution images with defined labels, wherein the sparse sampling is to extract data from the center point of each window on the entire image of the high-resolution image according to a set fixed interval through a sliding window, and the data includes feature X and label Y, and the setting of the sliding window size and interval is equal to the upsampling multiple of the label, and the class balanced sampling is to make the number of samples corresponding to each surface cover category appearing in the extracted data the same.

[0021] Furthermore, the classifier in step S1.2 is a random forest classifier model, each pixel spectral value corresponds to a surface cover category, and the random forest classifier model is used to predict the surface cover category of the image block based on the pixel spectral value of the image block.

[0022] Furthermore, the step S2 includes:

[0023] S2.1: using the high-resolution image and the surface cover category labels corresponding to the high-resolution image to train a convolutional neural network (CNN) model, wherein the CNN model is a U-Net or FC-DenseNet structure, and the CNN model is used for probability estimation of surface cover categories;

[0024] S2.2: Use the convolutional neural network (CNN) model trained in step S2.1 to estimate the category probability distribution of the surface cover label corresponding to each pixel. The category with the highest probability is used as the surface cover category predicted by the convolutional neural network (CNN) model. Use the adaptive noise correction module to find the noisy label, and use the category predicted by the convolutional neural network (CNN) model to replace the current surface cover category containing the noisy label to obtain the optimized surface cover category label.

[0025] Furthermore, the HRNet model in step S3 includes an encoder and a decoder. The encoder extracts image features, i.e., pixel spectral values, from high resolution to low resolution, and the decoder restores features from low resolution to high resolution. Each image feature, i.e., pixel spectral value, corresponds to a surface cover category label. The HRNet model extracts image features of the image block and predicts the surface cover category label of the image block based on the image features. When the surface cover category label of the image block predicted by the HRNet model is inconsistent with the surface cover category label of the image block after optimization in step S2, the surface cover category label predicted by the HRNet model is used to replace the existing surface cover category label of the image block.

[0026] The present invention also designs a weakly supervised long-time series land cover mapping system based on remote sensing images, comprising a data acquisition unit, a land cover category label correction unit, a land cover category label verification unit, and a land cover category label time series consistency check unit;

[0027] The data acquisition unit is used to acquire low-resolution products, display the low-resolution products as high-resolution images, collect training data from the high-resolution images, train a classifier through the training data, predict the land cover category of the target area through the trained classifier, and obtain the land cover category label of the high-resolution image of the target area;

[0028] The ground cover category label correction unit is used to correct the ground cover category label containing noise to obtain an optimized ground cover category label;

[0029] The land cover category label verification unit verifies the optimized land cover category label using the HRNet model to obtain a verified land cover mapping result;

[0030] The land cover category label temporal consistency check unit uses temporal filtering to perform a temporal consistency check on the verified land cover mapping result to obtain a confirmed land cover mapping result.

[0031] The present invention further provides a readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by a processor.

[0032] The present invention also provides an electronic device, comprising:

[0033] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the method as described in any one of claims 1 to 7.

[0034] The beneficial effects of the present invention are:

[0035] 1. Low-resolution images are used as historical data, and their surface coverage category labels have been defined. The present invention converts low-resolution images into high-resolution images by upsampling, and then assigns the surface coverage category labels of the low-resolution images to the high-resolution images, thereby saving a lot of high-resolution image labeling costs.

[0036] 2. Weakly supervised learning uses limited, noisy data to train model parameters. The weakly supervised learning method can complete model parameter training with less manual labeling time and cost. The present invention uses limited high-resolution image original surface cover labels for model training to obtain a classifier model, saving manual labeling time and cost. For the noisy surface cover category labels predicted by the classifier model, the present invention further optimizes the noisy labels, verifies the optimized labels, and performs a time consistency check on the verified labels, thereby improving the accuracy of surface cover category classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of the method of the present invention.

[0038] Figure 2 It is a flow chart of the method for achieving super-resolution by upsampling a low-resolution product of the present invention.

[0039] Figure 3 It is a schematic diagram of the noise label automatic correction method of the present invention.

[0040] Figure 4 It is an overall flow chart of the surface cover mapping method based on deep learning of the present invention.

[0041] Figure 5 This is the high-resolution land cover mapping model structure of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the invention clearer, the present invention is further described below with reference to the accompanying drawings.

[0043] A weakly supervised long-term land cover mapping method based on remote sensing images includes the following steps:

[0044] S1: Obtain a low-resolution product, display the low-resolution product as a high-resolution image, extract training data from the high-resolution image, train a classifier using the training data, predict the surface cover category of the target area using the trained classifier, and obtain a surface cover label of the target area.

[0045] The original labels from low-resolution products are matched with high-resolution images using the nearest neighbor interpolation algorithm. The local information of the original rough labels and the spatial information of the high-resolution images are used for sampling, training and prediction to introduce additional spatial detail information, thereby improving the fineness of the labels in spatial resolution.

[0046] The step 1 specifically comprises the following steps:

[0047] S1.1: Obtain a low-resolution product, and after upsampling the low-resolution product through the nearest neighbor interpolation method, generate the low-resolution product into high-resolution image data, first match the low-resolution product with the corresponding high-resolution image, and then assign the label corresponding to the low-resolution product, that is, the surface cover category, to the high-resolution image, and extract training data from the high-resolution image.

[0048] The low-resolution product is the historical remote sensing image data of a certain area. The entire image of the low-resolution product has been divided into multiple image blocks. The surface cover category label of each image block has been defined. The surface cover category includes vegetation, cultivated land, lakes, impermeable layers, ice and snow, and various buildings such as roads and houses. The low-resolution product is upsampled by the nearest neighbor interpolation method. The main purpose of upsampling is to enlarge the original image so that the original image can be displayed on a display device with a higher resolution. After the low-resolution product is upsampled and displayed at a higher resolution, each image block of the low-resolution product is matched with each image block of the generated high-resolution image data.

[0049] After each image block of the low-resolution product is matched with each image block of the corresponding high-resolution image, for the one-to-one matching low-resolution product image blocks and high-resolution image blocks, the surface coverage category of the low-resolution product is the surface coverage category of the high-resolution image. Since the low-resolution surface coverage category, that is, the label, has been defined, the label corresponding to the low-resolution product is assigned to the high-resolution image, and the label of the surface coverage category of the high-resolution image is determined.

[0050] The pixels and ground cover categories in the high-resolution image have been determined, and sparse sampling and class-balanced sampling are used to collect training data from the high-resolution image with defined labels. The training data includes feature X and label Y. The feature X refers to the spectral value of a pixel randomly selected from the high-resolution image, and the label Y refers to the ground cover category corresponding to the pixel. Feature X corresponds to feature Y one by one, and each feature X has a corresponding label Y, that is, the spectral value of each pixel corresponds to a ground cover category.

[0051] like Figure 2 As shown, feature X is the spectral value of a randomly selected pixel from a high-resolution image, and label Y is the land cover category of the corresponding pixel in the high-resolution image.

[0052] Sparse sampling is to extract the data of each window center point according to the set fixed interval on the whole image of the high-resolution image by sliding the window, and the data includes feature X and label Y. The size and interval of the sliding window are set to be approximately equal to the upsampling multiple of the label. The data extracted by sparse sampling comes from the enlarged image obtained after upsampling the low-resolution product, that is, the center position of the high-resolution image, which can reduce the noise label problem caused by the increase of image space details and make the extracted data as accurate as possible.

[0053] Class balanced sampling is to make the number of samples corresponding to each category in the extracted data the same. The category refers to the surface cover category. Because in the case of extremely unbalanced sample categories, the model prediction results are often biased. The model will tend to predict areas with higher uncertainty as classes with a larger number of samples, which will lead to more serious class imbalance problems in subsequent noise label optimization, and even cause the loss of certain categories. Therefore, the number of samples of each category appearing in the image block is set to K. If the number of samples of this category exceeds K in sparse sampling, K samples are randomly selected from it. If it is less than K, the pixels of this category on the entire high-resolution image are resampled until K samples are supplemented, where the size of K is related to the size of the image block.

[0054] S1.2: Train a classifier using the data collected in step S1.1. The classifier may be a random forest classifier model.

[0055] Since each pixel spectral value corresponds to a land cover category, the classifier can predict the land cover category of the image block according to the pixel spectral values ​​of the image block.

[0056] S1.3: Obtain the pixel spectral values ​​of the high-resolution image blocks of the target area, and use the classifier trained in step S1.2 to predict the surface cover category of the high-resolution image of the target area based on the pixel spectral values ​​to obtain the high-resolution surface cover label of the target area.

[0057] In the present invention, the target area refers to the area where surface cover mapping is to be performed, and a high-resolution remote sensing image of the target area can be obtained. The pixel spectral values ​​in the high-resolution remote sensing image block are known, and the spectral value of each pixel corresponds to a surface cover category. The classifier predicts the label of the surface cover category of the high-resolution remote sensing image block according to the spectral value of the pixel.

[0058] Step 2: Correct the labels of the land cover categories containing noise in step 1 to obtain optimized land cover category labels. This includes the following sub-steps:

[0059] S2.1: Use high-resolution images and initial land cover category labels to train a convolutional neural network (CNN) model. The training process is the same as the usual semantic segmentation model training. Use the cross entropy loss function to obtain an initial CNN model for the next step of probability estimation. Various commonly used semantic segmentation models can be used as CNN structures, such as U-Net and FC-DenseNet. The CNN structure used here is similar to the high-resolution network structure. In order to prevent the CNN model from overfitting to noisy labels, a larger learning rate and early stopping strategy are used here.

[0060] The low-resolution product is displayed as a high-resolution image after upsampling. A convolutional neural network (CNN) model is trained using the high-resolution image obtained after the upsampling and the surface cover category label corresponding to the high-resolution image. Various commonly used semantic segmentation models can be used as the CNN structure. The CNN model can be a U-Net or FC-DenseNet model. The CNN structure used in the present invention is similar to the high-resolution network structure.

[0061] The CNN model training process is the same as the usual semantic segmentation model training, using the cross entropy loss function to obtain an initial CNN model for the next step of probability estimation. The cross entropy loss function is one of the commonly used loss functions in deep learning, and is applicable to multi-classification problems. The smaller the value of the cross entropy loss function, the smaller the difference between the model's predicted results and the true results, and the better the performance of the model. In the present invention, the cross entropy loss function is used to optimize the model parameters to improve the accuracy of the classification model. The formula of the cross entropy loss function used in the present invention is: CE(p,q)=-∑p(x)log(q(x)), where p(x) is the probability distribution of the true result, q(x) is the probability distribution of the predicted result, and the smaller the CE(p,q) value, the smaller the difference between the model's predicted results and the true results.

[0062] In order to prevent the CNN model from overfitting to noisy labels, a larger learning rate and early stopping strategy can be used in CNN model training. The early stopping strategy is a strategy to prevent the CNN model from overfitting due to overly strict classification criteria during the probability estimation process. By monitoring the validation set error during training, the training is terminated early when the validation error starts to rise.

[0063] S2.2: The training framework of the second stage is as follows Figure 3 and Figure 4 As shown, the convolutional neural network CNN model trained in step S2.1 is used as the initial model to estimate the category probability distribution of the surface cover label corresponding to each pixel, and the category with the highest probability is used as the surface cover category predicted by the convolutional neural network CNN model. Then, the uncertainty estimation module is used to calculate the uncertainty value for each pixel on the map, and each image can obtain an uncertainty map of corresponding size. Next, the adaptive noise correction module is used to find out the part that may be the wrong label, and the category predicted by the convolutional neural network CNN model is used to replace the current category of this part of the label. In order to reduce the storage pressure of the intermediate results and improve the efficiency of training, the present invention simultaneously updates the model parameters and the correction labels in each iteration of the training, instead of uniformly predicting and correcting the labels after each training cycle. The present invention uses a collaborative noise correction loss function, which includes a cross entropy loss function between the model prediction result and the original noise label and a cross entropy loss function between the model prediction result and the label updated in the previous training cycle. When the value of the collaborative noise correction loss function is the smallest, the parameters that the model can learn are optimal, so that the correct part of the original label can be used, and the more accurate label after the update can also be benefited.

[0064] Step 3: Use the HRNet model to verify the optimized land cover category labels in step 2 to obtain the verified land cover mapping results Where 0, 1…n represent different land cover categories. is the ground cover category of the tth image block.

[0065] According to the reliance of high-resolution remote sensing applications on spatial information, the deep learning semantic segmentation high-resolution network model is improved in its ability to represent high-resolution features by maintaining and fusing different resolution features to improve prediction accuracy, and the instance normalization method is used to improve versatility and robustness. The HRNet model increases the weight of small-scale and complex targets in the loss function to alleviate the problem of sample imbalance, and uses adaptive histogram equalization to improve the precision of the results of impermeable layers, which refer to areas with very low permeability such as roads, buildings or rock formations. Specifically, it includes the following sub-steps:

[0066] S3.1: In order to improve the model's ability to represent features of different resolutions, the present invention selects the HRNet model structure as the semantic segmentation model of the high-resolution land cover mapping method. The HRNet model includes an encoder and a decoder. The encoder extracts image features, i.e., pixel spectral values, from high resolution to low resolution, and the decoder performs feature recovery from low resolution to high resolution, and uses jump links to combine features of different resolutions in the encoder. It is then fused with other resolution features to improve the model's ability to represent features of different resolutions and further improve the accuracy of the results. Each image feature, i.e., pixel spectral value, corresponds to a label for a land cover category. The HRNet model extracts image features of the image block, and predicts the label of the land cover category of the image block based on the image features. When the land cover category label of the image block predicted by the HRNet model is inconsistent with the land cover category label of the image block after optimization in step S2, the land cover category label predicted by the HRNet model replaces the existing land cover category label of the image block. Figure 5As shown in the figure, HRNet can be divided into four stages, represented by different colors. Each branch (i.e., each row) represents a different resolution. First, the input image is downsampled through a 3×3 convolutional layer with a stride of 2. After downsampling, a thumbnail of the corresponding image can be generated. The thumbnail can reduce the image features and thus reduce the amount of calculation. Therefore, downsampling can effectively improve the efficiency of model training. Then, the first stage contains four residual units (represented by green rectangles), each of which consists of a bottleneck module with a width of 64, which is the same as the basic block in ResNet-50. The horizontal arrow in the transformation block (represented by a cyan rectangle) represents a 3×3 convolutional layer with a stride of 1. The downward diagonal arrow in the transformation block (represented by a pink rectangle) represents a 3×3 convolutional layer with a stride of 2, which expands the number of input channels to twice and downsamples. The second, third, and fourth stages contain 1, 4, and 3 multi-resolution blocks, respectively. The multi-resolution block (represented by a black dashed box) is used to extract and fuse features of different resolutions. Each multi-resolution block consists of several single-resolution convolution blocks and a multi-resolution fusion block. Each single-resolution convolution block (represented by a yellow rectangle) contains 4 residual blocks, each of which consists of two 3×3 convolutional layers. Each multi-resolution fusion block (represented by a purple rectangle) unifies and fuses all resolution features at different resolutions using upsampling or downsampling. The transformation blocks in the second, third, and fourth stages are basically the same as those in the first stage, except that the number of channels of the output features is different. The number of channels in the four stages is C, 2C, 4C, and 8C, respectively (C is set to 24 in this study). Finally, the features of different resolutions are unified and merged through bilinear upsampling, and the Softmax layer is applied to obtain the final mapping result.

[0067] S3.2: In the initial part of the model, in order to reduce the negative impact of different image quality or spectral characteristics, the present invention uses instance normalization for normalization. Compared with batch normalization, instance normalization calculates the mean and variance of each image instead of the mean and variance of each batch of images, thereby reducing the model's dependence on different image qualities. Therefore, using an instance normalization layer instead of a batch normalization layer after the first convolutional layer can effectively improve the generalization ability of the model.

[0068] S3.3: In the implementation details of model training, considering the balance between the model receptive field and GPU memory, the size of the model input image is set to 513×513. The initial learning rate of model training is set to 0.01, and the stochastic gradient descent method is used with a momentum of 0.9. By analyzing the probability map of the model prediction results of the impermeable layer category, it is found that the response degree of this category is different in different regions. The response is high in dense areas and low in sparse areas. Sometimes the response is not distinguished enough from other categories, resulting in discontinuous road target prediction results and urban buildings being connected into pieces without details. Therefore, the local method is used to process the probability results. Different histogram equalization is used in different regions to enhance the distinguishability of the response. The response range of the equalization can be freely controlled by using adaptive parameters. According to the experiment, the size of each local area is set to 512×512.

[0069] Step 4: In order to reduce the impact of seasonal changes, image quality and classification errors, the time series filtering method is used to check the time series consistency of the multi-time resolution land cover mapping results, reduce illogical land cover category conversions, and obtain more reasonable long-term land cover mapping results. The specific sub-steps include the following:

[0070] Assume that the current multi-temporal resolution land cover mapping result is Let the sliding time series window size T w =3, step size is 1, from The image block starts to slide backwards, and the temporal consistency Prob of the central pixel of the sliding temporal window is calculated. i value,

[0071]

[0072] Where L i and L j are the land cover category labels of the central pixel and other pixels in the time series window respectively; con(·) is the consistency function. If L j =L i If i0 is equal to 1, it returns 1, otherwise it returns 0; i0 and i w are the indices of the first and last pixels in the sliding window;

[0073] If Prob i >0.5, the center pixel remains unchanged, and the surface coverage category of the original image block remains unchanged; if Prob i If <0.5, calculate the Prob of other pixels in the original sliding window i value, if other pixels satisfy Prob i >0.5, the land cover label of the central pixel is corrected to the land cover category label corresponding to the other pixels. If the other pixels do not meet Probi >0.5, the center pixel value remains unchanged, the surface cover category of the original image block remains unchanged, the sliding window continues to slide forward, and the pixels that have been corrected in the time series will be used to correct subsequent pixels. After the sliding window runs to the end of the time series, let T w =5 and then from The image block begins to continue the correction process of this step.

[0074] Based on the above method, the present invention also proposes a weakly supervised long-term land cover mapping system based on remote sensing images, including a data acquisition unit, a land cover category label correction unit, a land cover category label verification unit, and a land cover category label time series consistency check unit;

[0075] The data acquisition unit is used to acquire low-resolution products, display the low-resolution products as high-resolution images, collect training data from the high-resolution images, train a classifier through the training data, predict the land cover category of the target area through the trained classifier, and obtain the land cover category label of the high-resolution image of the target area;

[0076] The ground cover category label correction unit is used to correct the ground cover category label containing noise to obtain an optimized ground cover category label;

[0077] The land cover category label verification unit verifies the optimized land cover category label using the HRNet model to obtain a verified land cover mapping result;

[0078] The land cover category label time sequence consistency check unit uses time sequence filtering to perform a time sequence consistency check on the verified land cover mapping result to obtain a confirmed land cover mapping result.

[0079] The present invention further provides a readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by a processor.

[0080] The present invention further provides an electronic device, comprising:

[0081] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the method as described in any one of claims 1 to 7.

[0082] Finally, it should be noted that the contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field. The above description is only the preferred embodiment of the present invention and is not used to limit the present invention. Although the present invention is described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions recorded in the aforementioned embodiments, or to replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A weakly supervised long-term land cover mapping method based on remote sensing images, characterized in that: The following steps are involved: Obtaining a low-resolution product, displaying the low-resolution product as a high-resolution image, collecting training data from the high-resolution image, training a classifier using the training data, predicting a land cover category of a target area using the trained classifier, and obtaining a land cover category label of the high-resolution image of the target area; Correcting the surface cover category label containing noise to obtain an optimized surface cover category label; The optimized land cover category labels are verified using the HRNet model to obtain verified land cover mapping results. Where 0, 1…n represent different land cover categories. is the ground cover category of the tth image block; A time series filtering method is used to perform a time series consistency check on the land cover mapping result to obtain a definite land cover mapping result.

2. According to the method of weakly supervised long-term land cover mapping based on remote sensing images in claim 1, it is characterized in that: The method of timing consistency checking is: Let the sliding time series window size T w =3, step size is 1, from The image block starts to slide backwards, and the temporal consistency Prob of the central pixel of the sliding temporal window is calculated. i value, Where L i and L j are the land cover category labels of the central pixel and other pixels in the time series window respectively; con(·) is the consistency function. If L j =L i If i0 is equal to 1, it returns 1, otherwise it returns 0; i0 and i w are the indices of the first and last pixels in the sliding window; If Prob i >0.5, the center pixel remains unchanged, and the surface coverage category of the original image block remains unchanged; if Prob i If <0.5, calculate the Prob of other pixels in the original sliding window i value, if other pixels satisfy Prob i >0.5, the land cover label of the central pixel is corrected to the land cover category label corresponding to the other pixels. If the other pixels do not meet Prob i >0.5, the center pixel value remains unchanged, the surface coverage category of the original image block remains unchanged, and the sliding window continues to slide forward. When the sliding window reaches the end of the time series, let T w =5 and then from The image block begins to continue the correction process of this step.

3. According to the method of weakly supervised long-term land cover mapping based on remote sensing images in claim 1, it is characterized in that: The method of obtaining a low-resolution product, displaying the low-resolution product as a high-resolution image, collecting training data from the high-resolution image, training a classifier using the training data, predicting the land cover category of the target area using the trained classifier, and obtaining a land cover category label of the high-resolution image of the target area includes: Acquire a low-resolution product image, wherein the low-resolution product image is historical remote sensing image data of a certain area, the entire image of the low-resolution product image has been divided into a plurality of image blocks, the surface cover category label of each image block has been defined, upsample the low-resolution product by the nearest neighbor interpolation method, generate the low-resolution product into a high-resolution image after upsampling, match the low-resolution product with the corresponding high-resolution image, and then assign the label corresponding to the low-resolution product, i.e., the surface cover category, to the high-resolution image, collect training data from the high-resolution image, the training data includes a feature X and a label Y, wherein the feature X refers to a pixel spectral value selected from the high-resolution image, and the label Y refers to a surface cover category corresponding to the pixel spectral value, and each feature X has a corresponding label Y; Using the training data to train a classifier, the classifier is used to predict a label Y corresponding to a ground cover category of the image block according to a pixel spectral value X of the image block; The pixel spectral values ​​of the high-resolution image blocks of the target area are obtained, and the surface cover category of the high-resolution image of the target area is predicted by using the trained classifier according to the pixel spectral values ​​to obtain the surface cover label of the high-resolution image of the target area.

4. The method for weakly supervised long-term land cover mapping based on remote sensing images according to claim 3 is characterized in that: The method for collecting training data is: using sparse sampling and class balanced sampling to collect training data from high-resolution images with defined labels, wherein the sparse sampling is to extract data of the center point of each window according to a set fixed interval on the entire image of the high-resolution image through a sliding window, wherein the data includes feature X and label Y, and the setting of the sliding window size and interval is equal to the upsampling multiple of the label, and the class balanced sampling is to make the number of samples corresponding to each surface cover category appearing in the extracted data the same.

5. The method for weakly supervised long-term land cover mapping based on remote sensing images according to claim 3 is characterized in that: The classifier is a random forest classifier model, each pixel spectral value corresponds to a surface cover category, and the random forest classifier model is used to predict the surface cover category of the image block according to the pixel spectral value of the image block.

6. The method for weakly supervised long-term land cover mapping based on remote sensing images according to claim 1 is characterized in that: The optimized land cover category labels include: Using the high-resolution image and the surface cover category labels corresponding to the high-resolution image to train a convolutional neural network (CNN) model, wherein the CNN model is a U-Net or FC-DenseNet structure, and the CNN model is used for probability estimation of surface cover categories; The trained convolutional neural network (CNN) model is used to estimate the category probability distribution of the surface cover label corresponding to each pixel. The category with the highest probability is used as the surface cover category predicted by the convolutional neural network (CNN) model. The adaptive noise correction module is used to find the noisy labels, and the category predicted by the convolutional neural network (CNN) model is used to replace the current surface cover category containing the noisy label to obtain the optimized surface cover category label.

7. The method for weakly supervised long-term land cover mapping based on remote sensing images according to claim 1, characterized in that: The HRNet model includes an encoder and a decoder. The encoder extracts image features, i.e., pixel spectral values, from high resolution to low resolution, and the decoder restores features from low resolution to high resolution. Each image feature, i.e., pixel spectral value, corresponds to a label of a surface cover category. The HRNet model extracts image features of image blocks and predicts the label of the surface cover category of the image blocks based on the image features. When the surface cover category label of the image block predicted by the HRNet model is inconsistent with the surface cover category label of the image block after optimization, the surface cover category label predicted by the HRNet model replaces the existing surface cover category label of the image block.

8. A weakly supervised long-term land cover mapping system based on remote sensing images, characterized by: It includes a data acquisition unit, a surface coverage category label correction unit, a surface coverage category label verification unit, and a surface coverage category label time sequence consistency check unit; The data acquisition unit is used to acquire low-resolution products, display the low-resolution products as high-resolution images, collect training data from the high-resolution images, train a classifier through the training data, predict the land cover category of the target area through the trained classifier, and obtain the land cover category label of the high-resolution image of the target area; The ground cover category label correction unit is used to correct the ground cover category label containing noise to obtain an optimized ground cover category label; The land cover category label verification unit verifies the optimized land cover category label using the HRNet model to obtain a verified land cover mapping result; The land cover category label temporal consistency check unit uses temporal filtering to perform a temporal consistency check on the verified land cover mapping result to obtain a confirmed land cover mapping result.

9. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described in any one of claims 1 to 7.

Images