Method and device for refining land cover classification products based on image time series
By repairing image time series and using K-nearest neighbor classification, the problem of insufficient utilization of temporal information in land cover classification was solved, thereby improving classification accuracy and data utilization.
Patent Information
- Application Number
- CN202111319177.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-09
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-11-09
AI Technical Summary
The lack of effective utilization time information in existing land cover classification products leads to insufficient classification accuracy.
By acquiring remote sensing images in time series, cloud and cloud shadow noise is repaired, pixel correspondence is constructed, and K-nearest neighbor classification is performed to update the land cover classification results.
It improved the accuracy and usability of land cover classification, made full use of new data conditions, and enhanced the utilization rate of historical products.
Smart Images

Figure CN114092826B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method and device for refining land cover classification products based on image time series. Background Art
[0002] Land cover classification is the division of the earth's land surface and near-surface layers into different categories according to their specific conditions, such as cultivated land, forest land, grassland, bare land, garden land, wetland, water bodies, buildings, etc. Each category can be further subdivided. Land cover classification information reflects the natural processes and human activities on the surface. Its changes have a direct impact on surface biology, climate, hydrology and other processes. It is an important indicator and parameter reflecting changes in macro and micro ecosystems. Therefore, land cover classification has become an important component and research content in many fields such as ecological environment change, natural resource monitoring, and land and resources management.
[0003] Remote sensing images provide convenient and real observations of the earth's surface over a large area, and are the main data source for the current land cover classification task. Remote sensing images generally contain three types of information: spectral information, spatial information, and temporal information. Spectral information reflects the characteristics of land object categories in different spectral bands, spatial information reflects the characteristics of the spatial neighborhood of land object categories, and temporal information reflects the characteristics of land object categories that change over time. The expression of temporal characteristics requires a series of images from different times to form an image time series, and the laws of how different land object types change over time can be analyzed from the time series, while spectral information and spatial information can be reflected in each individual image. For a long time, due to the large-scale cloud and cloud shadow cover problems in remote sensing images and the limited observation time interval, the data requirements for the image time series are difficult to meet. Therefore, land cover classification mainly uses spectral information and spatial information, that is, individual independent images with good data quality are used as information sources.
[0004] At present, the main land cover classification product series, such as GLC_FCS 30-meter global land cover classification products, FROM-GLC series 10-meter and 30-meter global land cover classification products, Global Land 30-meter global geographic information public products, etc., have been continuously producing and releasing products. In recent years, the development of these products is mainly reflected in the introduction of artificial intelligence methods represented by deep neural networks, combined with or replacing previous classification methods such as decision trees or support vector machines, and more training samples or other auxiliary data are used in the classification process. However, the historical products in these product series still use the spectral information and spatial information of the land feature category as the main information source, and the time information is in a local and auxiliary position. Summary of the invention
[0005] The present invention provides a method and device for refining land cover classification products based on image time series, so as to solve the defect in the prior art that the land cover classification is not accurately classified by using time information, and further improve the classification accuracy by adding time information to the existing classification results.
[0006] In a first aspect, the present invention provides a method for refining land cover classification products based on image time series, comprising:
[0007] Acquire remote sensing images that present time series;
[0008] Repairing each image to be repaired in the remote sensing image using other images other than each image to be repaired in the remote sensing image to obtain a repaired remote sensing image;
[0009] The existing surface cover classification result pixels are refined and updated according to the repaired remote sensing image to obtain an updated surface cover classification result.
[0010] Further, according to a method for refining land cover classification products based on image time series provided by the present invention, the method refines and updates the existing land cover classification result pixels according to the patched remote sensing image to obtain the updated land cover classification result, specifically comprising:
[0011] Constructing a correspondence between the pixels of the patched remote sensing image and the pixels of the existing land cover classification result;
[0012] The existing surface cover classification result pixels are refined and updated according to the repaired remote sensing image and the corresponding relationship to obtain an updated surface cover classification result.
[0013] Further, according to a method for refining land cover classification products based on image time series provided by the present invention, wherein each image to be repaired in the remote sensing image is repaired using other images other than each image to be repaired in the remote sensing image to obtain a repaired remote sensing image, specifically comprising:
[0014] Marking clouds and cloud shadows on each image in the remote sensing image to obtain an image with cloud and cloud shadow pixels marked, and using the image with cloud and cloud shadow pixels marked as the image to be repaired;
[0015] Determine the similarity between other images in the remote sensing image other than each image to be repaired and the image to be repaired, and sort the similarities in descending order;
[0016] The image to be repaired is repaired in sequence according to the other images corresponding to the sorting to obtain a repaired remote sensing image.
[0017] Further, according to a method for refining land cover classification products based on image time series provided by the present invention, the image to be repaired is repaired in sequence according to the other images corresponding to the sorting to obtain a repaired remote sensing image, specifically comprising:
[0018] Dividing the image to be repaired into a plurality of first image spots according to connectivity;
[0019] Acquire a second image spot at the same position as each of the first image spots in the other images;
[0020] The first image spots are filled with the second image spots in sequence according to the arrangement to obtain a repaired remote sensing image.
[0021] Further, according to a method for refining land cover classification products based on image time series provided by the present invention, the step of constructing a correspondence between pixels of the patched remote sensing image and pixels of the existing land cover classification results specifically includes:
[0022] Aligning the pixels between the repaired remote sensing images to obtain a remote sensing image with pixel alignment in a time series;
[0023] The pixel-aligned remote sensing image is reprojected and resampled according to the projection mode and resolution of the existing land cover classification result to obtain the corresponding relationship between the pixels of the patched remote sensing image and the pixels of the existing land cover classification result.
[0024] Further, according to a method for refining land cover classification products based on image time series provided by the present invention, the method refines and updates the existing land cover classification result pixels according to the patched remote sensing image and the corresponding relationship to obtain an updated land cover classification result, specifically comprising:
[0025] Dividing the patched remote sensing image into blocks in a spatial dimension to obtain a plurality of first data blocks;
[0026] Acquire a second data block having the same coverage range as the first data block in the existing surface cover classification result image;
[0027] Performing fill value judgment on each pixel position in the first data block to obtain a valid pixel mask of the first data block;
[0028] Performing a morphological erosion operation on the second data block to obtain a category label of each pixel of the second data block;
[0029] Obtaining category labels corresponding to all valid pixels as candidate training samples according to the category labels and the valid pixel masks;
[0030] Filtering training samples from the candidate training samples;
[0031] Perform K-nearest neighbor classification on all the valid pixels according to the training samples to obtain a new category label for each valid pixel;
[0032] Replace the category of each pixel of the second data block according to the new category label of each valid pixel;
[0033] The second data block after replacement according to the category of each pixel obtains an updated surface cover classification result.
[0034] Further, according to a method for refining land cover classification products based on image time series provided by the present invention, the step of selecting training samples from the candidate training samples specifically includes:
[0035] Determine the number of valid pixels corresponding to each of the category labels in the candidate training samples;
[0036] Randomly select m valid pixels from the number of pixels as training samples.
[0037] In a second aspect, the present invention provides a device for refining land cover classification products based on image time series, comprising:
[0038] An acquisition module is used to acquire remote sensing images presenting a time series;
[0039] A patching module, used for patching each to-be-patched image in the remote sensing image by using other images other than each to-be-patched image in the remote sensing image to obtain a patched remote sensing image;
[0040] The refinement module is used to refine and update the existing surface cover classification result pixels according to the repaired remote sensing image to obtain an updated surface cover classification result.
[0041] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned methods for refining land cover classification products based on image time series are implemented.
[0042] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for refining land cover classification products based on image time series.
[0043] The present invention provides a method and device for refining a surface cover classification product based on an image time series, which obtains remote sensing images presenting a time series; repairs each image to be repaired in the remote sensing image using other images other than each image to be repaired in the remote sensing image to obtain a repaired remote sensing image; and refines and updates the existing surface cover classification result pixels according to the repaired remote sensing image to obtain an updated surface cover classification result. The present invention quickly realizes reliable updating of results by introducing new time information, thereby enhancing the utilization rate of related data and historical products. On the premise of inheriting the original results, it fully utilizes new data conditions to make the original products have accuracy and availability again. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0045] Figure 1 This is one of the flow charts of the method for refining land cover classification products based on image time series provided by the present invention;
[0046] Figure 2 It is a schematic diagram of aligning the pixels between each image in a time series with the surface cover classification result provided by the present invention;
[0047] Figure 3 It is a schematic diagram of a situation in which required data may come from different tiling products when the land cover classification product to be updated is clipped provided by the present invention;
[0048] Figure 4 It is a structural schematic diagram of a device for refining land cover classification products based on image time series provided by the present invention;
[0049] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0050] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0051] Combine the following Figure 1 A method for refining land cover classification products based on image time series of the present invention is described, comprising:
[0052] Step 100: Acquire remote sensing images showing a time series;
[0053] Specifically, with the rapid development of remote sensing observation technology and platforms in recent years, as well as in-depth research on remote sensing image processing and restoration methods, the data conditions for the application of remote sensing image time series are becoming increasingly mature. From the perspective of data sources, more and more remote sensing satellites with high temporal resolution are gradually being launched, such as my country's Gaofen series satellites and Jilin series satellites, and historical data has been accumulated for more than ten years, such as Landsat, MODIS, Sentinel and other series satellite data. From the perspective of image processing methods, various cloud detection, cloud repair, image restoration, fusion or interpolation technologies have made great progress, and data that could not be used in the past due to poor data quality can be finely repaired to make it usable, and then form an image time series.
[0054] The expression of temporal characteristics requires a series of images from different times to form an image time series, from which the changing patterns of different types of land objects over time can be analyzed, while spectral information and spatial information can be reflected in each individual image.
[0055] Step 200: repairing each image to be repaired in the remote sensing image by using other images other than the image to be repaired in the remote sensing image to obtain a repaired remote sensing image;
[0056] Specifically, under the condition that the data source is already rich enough, another major reason limiting the use of remote sensing image time series is the large number of cloud and cloud shadow pixels. These pixels obscure the true situation of the surface and become noise that has a greater impact on the results during data use. Therefore, the present invention first needs to comprehensively repair the noise values such as clouds and cloud shadows distributed in the entire time series.
[0057] Since the real situation of the surface is covered by clouds and cloud shadows, the key to repairing is to choose which information source to replace the covered information. The present invention uses image time series, that is, images acquired at different times within the same spatial range, so a natural idea is to use clean pixels in another image to replace the contaminated pixels determined to be clouds or cloud shadows.
[0058] Step 300: Refining and updating the existing surface cover classification result pixels according to the repaired remote sensing image to obtain an updated surface cover classification result.
[0059] Specifically, after repairing, the time series is used as new data to be classified, the original classification results are used as training sets for reclassification, and the new classification results are used to update the original results.
[0060] The present invention provides a method for refining a land cover classification product based on an image time series, which is performed by acquiring remote sensing images presenting a time series; repairing each image to be repaired in the remote sensing images using other images other than each image to be repaired in the remote sensing images to obtain a repaired remote sensing image; and refining and updating the existing land cover classification result pixels according to the repaired remote sensing images to obtain an updated land cover classification result. The present invention quickly realizes reliable updating of the results by introducing new time information, thereby enhancing the utilization rate of related data and historical products, and fully utilizing new data conditions on the premise of inheriting the original results, so that the original products regain accuracy and usability.
[0061] Further, according to a method for refining land cover classification products based on image time series provided by the present invention, the method refines and updates the existing land cover classification result pixels according to the patched remote sensing image to obtain the updated land cover classification result, specifically comprising:
[0062] Constructing a correspondence between the pixels of the patched remote sensing image and the pixels of the existing land cover classification result;
[0063] The existing surface cover classification result pixels are refined and updated according to the repaired remote sensing image and the corresponding relationship to obtain an updated surface cover classification result.
[0064] Specifically, the remote sensing image time series data is repaired and constructed, and then a one-to-one correspondence between the spatial positions of pixels and the existing land cover classification results is established. That is, each pixel space in the time series image corresponds to the spatial position corresponding to the pixel of the existing land cover classification result.
[0065] Finally, the repaired time series is used as a new information source to refine and update the land cover classification results. That is, the existing land cover classification results are refined according to the corresponding labels of the pixels in the time series.
[0066] Further, according to a method for refining land cover classification products based on image time series provided by the present invention, wherein each image to be repaired in the remote sensing image is repaired using other images other than each image to be repaired in the remote sensing image to obtain a repaired remote sensing image, specifically comprising:
[0067] Marking clouds and cloud shadows on each image in the remote sensing image to obtain an image with cloud and cloud shadow pixels marked, and using the image with cloud and cloud shadow pixels marked as the image to be repaired;
[0068] Determine the similarity between other images in the remote sensing image other than each image to be repaired and the image to be repaired, and sort the similarities in descending order;
[0069] The image to be repaired is repaired in sequence according to the other images corresponding to the sorting to obtain a repaired remote sensing image.
[0070] Specifically, the image time series is first repaired and constructed. The main objects of repair are the ubiquitous cloud and cloud shadow pixels. For some sensors, there may also be damaged or poor quality pixels. For such pixels, if the data is released with a pixel quality tag, they can be treated as cloud pixels and processed in the same way.
[0071] Step 1: The premise of repairing clouds and cloud shadows is to be able to accurately mark the cloud and cloud shadow pixels. Since clouds are the main source of noise in remote sensing images, many remote sensing data will be accompanied by cloud and cloud shadow pixel tags when they are released, such as Landsat data products. For such products, you can directly use the attached cloud and cloud shadow pixel tags. For products that do not provide cloud and cloud shadow pixel tags, you need to detect the clouds and cloud shadows of each image yourself to obtain a mark for whether each pixel is a cloud or a cloud shadow. The specific detection method can be determined according to the specific data situation. There are many related studies at present, and you can use a method with reliable accuracy.
[0072] Step 2: After marking the cloud and cloud shadow pixels in each image, the clouds and cloud shadows need to be repaired. Since the real situation of the surface is obscured by clouds and cloud shadows, the key to repairing is to choose which information source to replace the obscured information. The present invention uses image time series, that is, images acquired at different times within the same spatial range. Therefore, a natural idea is to use clean pixels in another image to replace the dirty pixels that are judged to be clouds or cloud shadows. In the process of replacement, the issues that need to be considered mainly include: (1) the order in which the images are used, that is, which image in the sequence of images is used first to provide information, and which ones are used later; (2) what changes need to be made to the information from other images during replacement to better adapt to the current image; (3) the judgment criteria for the completion of repair. In the process of image repair, the present invention determines the order of image use by calculating the overall similarity of each image to be repaired, that is, the image most similar to the image to be repaired is used first; when replacing pixels, a block-by-block replacement method is adopted, and for each connected pixel, the image blocks from other images are more naturally integrated into the image to be repaired by Poisson fusion; finally, the repair is stopped when there are no pixels to be repaired or the images in the time series are exhausted. All images in the time series are repaired in the above manner.
[0073] The cloud and cloud shadow marks of the current image to be repaired are morphologically expanded to evenly expand the marking range and avoid missing marks; other images in the image time series are sorted according to their similarity with the current image, and structural similarity (SSIM) is used as the basis for similarity measurement; other images are used in order from large to small similarity to perform information migration and repair on the current image.
[0074] Further, according to a method for refining land cover classification products based on image time series provided by the present invention, the image to be repaired is repaired in sequence according to the other images corresponding to the sorting to obtain a repaired remote sensing image, specifically comprising:
[0075] Dividing the image to be repaired into a plurality of first image spots according to connectivity;
[0076] Acquire a second image spot at the same position as each of the first image spots in the other images;
[0077] The first image spots are filled with the second image spots in sequence according to the arrangement to obtain a repaired remote sensing image.
[0078] Specifically, other images are used in order of similarity from large to small to perform information migration and repair on the current image. The specific method is to divide the pixels to be repaired into several spots according to connectivity, obtain the spots from the same position in another image for each spot and exclude the pixels that are also clouds or cloud shadows, and use the Poisson fusion method to migrate the spots in the other image to the image to be repaired to fill the corresponding pixels, and update the cloud and cloud shadow marks after filling. If all cloud and cloud shadow pixels in the image to be repaired are repaired, or all images in the image time series have been used for repair, the repair of the current image is terminated.
[0079] Further, according to a method for refining land cover classification products based on image time series provided by the present invention, the step of constructing a correspondence between pixels of the patched remote sensing image and pixels of the existing land cover classification results specifically includes:
[0080] Aligning the pixels between the repaired remote sensing images to obtain a remote sensing image with pixel alignment in a time series;
[0081] The pixel-aligned remote sensing image is reprojected and resampled according to the projection mode and resolution of the existing land cover classification result to obtain the corresponding relationship between the pixels of the patched remote sensing image and the pixels of the existing land cover classification result.
[0082] Specifically, the first step is to align the pixels between the images in the time series, such as Figure 2 As shown, pixel alignment is to make the pixels at the same coordinate position in different images, such as the pixels at the (x, y) position, represent the same real geographical location. To achieve pixel alignment between different images, it is mainly divided into three cases according to the different data sensors: the first case is that there is nonlinear offset between images. In this case, it is necessary to select the same reference image and perform geometric correction on the images in the sequence respectively. Geometric correction is a common technology in remote sensing image processing. It will not be described in detail in the present invention. Just select a method suitable for the current sensor data. After correction, it is also necessary to cut out the overlapping parts of the spatial coverage of all images in the sequence to ensure the integrity of the time series data and the consistency of the sequence length. The second case is that there is a linear offset between images. At this time, it is only necessary to cut out the overlapping parts of the spatial coverage of all images in the sequence to eliminate the linear offset. The third case is that the pixels between different images are aligned, and there is no offset, so no processing is required. After the pixel alignment between the image sequences, the data acquired at different times at each pixel position can be connected in series into a time series, and then the corresponding relationship with the classification result pixel is established as a whole.
[0083] Step 2: Align the image time series pixels with the classification result pixels. The main operation is to reproject and resample all the images in the image time series to keep them consistent with the map projection and spatial resolution of the classification product. Then, when reclassifying, you only need to crop the specified spatial range from the classification product. Reprojection and resampling are common operations in remote sensing image processing. Currently, there are many function libraries in different programming languages available, such as GDAL, RasterIO, etc. It should be noted that the default fill value in the data may be changed during the calculation process. For example, if the default fill value is -9999, if the new fill value is not specifically specified as -9999, the interpolation operation may cause the fill value to become a negative number between -9999 and 0, thereby affecting the subsequent judgment of the fill value. Therefore, during the reprojection and resampling process, it is necessary to specify a new fill value that is consistent with the original image.
[0084] Further, according to a method for refining land cover classification products based on image time series provided by the present invention, the method refines and updates the existing land cover classification result pixels according to the patched remote sensing image and the corresponding relationship to obtain an updated land cover classification result, specifically comprising:
[0085] Dividing the patched remote sensing image into blocks in a spatial dimension to obtain a plurality of first data blocks;
[0086] Acquire a second data block having the same coverage range as the first data block in the existing surface cover classification result image;
[0087] Performing fill value judgment on each pixel position in the first data block to obtain a valid pixel mask of the first data block;
[0088] Performing a morphological erosion operation on the second data block to obtain a category label of each pixel of the second data block;
[0089] Obtaining category labels corresponding to all valid pixels as candidate training samples according to the category labels and the valid pixel masks;
[0090] Filtering training samples from the candidate training samples;
[0091] Perform K-nearest neighbor classification on all the valid pixels according to the training samples to obtain a new category label for each valid pixel;
[0092] Replace the category of each pixel of the second data block according to the new category label of each valid pixel;
[0093] The second data block after replacement according to the category of each pixel obtains an updated surface cover classification result.
[0094] Specifically, the first step is to divide the image time series into blocks in the spatial dimension. Considering the limitation of computer memory for subsequent processing, the width and height of the blocks are generally set to 1000 pixels * 1000 pixels. The block division process is carried out in sequence from the width and height dimensions. For example, for an image with a width of 7300 and a height of 6900, first determine all the starting coordinates of the image blocks in the width direction, such as 1, 1001, 2001, etc., and then determine the starting coordinates of all image blocks in the height direction, such as 1, 1001, 2001, etc., and then arrange and combine the starting coordinates in the width and height directions to obtain the upper left corner starting coordinates of all blocks, such as (1, 1), (1, 1001), (1, 2001), etc. For the last block division in the width and height directions, the remaining parts that are less than 1.5 times the width and height are merged into one block. The size of the last block is 1300 in width and 900 in height, and the corresponding upper left corner coordinates are (6001, 6001).
[0095] Step 2: For each piece of data, cut out the part with the same spatial coverage as the current block data in the land cover classification product to be updated. Since land cover classification products are generally tiled, for example, each product covers a longitude and latitude range of 10 degrees * 10 degrees, it may happen that the required data comes from different tile products during cutting. The specific situation is as follows Figure 3 The following three types are shown. The first is that the image block is completely located in a certain product. The second is that the image block spans two adjacent products in the longitude or latitude direction. The third is that the image block spans the intersection of four adjacent products. In the latter two cases, it is necessary to calculate the parts of the image block located in different products separately, and then crop them and merge them. After the previous reprojection and resampling process, the map projection and spatial resolution used by the classified product and the image block are exactly the same, so only linear coordinate operations are involved during cropping. After cropping, when the spatial range is completely consistent, a one-to-one correspondence is naturally established between pixels with the same coordinates.
[0096] Step 3: There will be fill values in the data from some satellite sensors, indicating that no data is collected here, such as Landsat data, Gaofen-1 data, etc. Therefore, for each piece of data, it is necessary to exclude these pixel positions without data before using them. The fill values of different sensors or different types of data are generally known. Since the data block here is a time series data block, all images in the sequence must be judged for each pixel position. If it is judged as a fill value in a certain image, this pixel is also considered an invalid pixel.
[0097] Step 4: For the classified product data blocks cropped in step 2, a morphological erosion operation needs to be performed on each land cover category before use to extract the pixels with higher classification credibility in each category. Morphological erosion is a common operation in image processing, and there are related implemented function libraries in various programming languages.
[0098] Step 5: Intersect the category label after morphological corrosion in the fourth step with the effective pixel mask in the third step to obtain the category label corresponding to all effective data as candidate training samples. In the existing classification method, the number of training samples in each category will affect the tendency of the classification result. Therefore, the number of training samples in each category in the present invention will refer to the existing classification products after morphological corrosion.
[0099] Step 6: For each block of data, use the training samples obtained in the fifth step to perform K-nearest neighbor classification on all valid pixels. The value of K is generally 3. If the block is large, the K value can be set to 5 or 7 depending on the situation. The time series data similarity measurement method used in classification is Euclidean distance. The nearest neighbor classifier has related implemented function libraries that can be used in various programming languages. After classification, a new category label for each valid pixel in the data block is obtained.
[0100] Step 7: After all the block data have been classified as above, merge all the block results. When merging, first create a blank complete image. The image range, projection, spatial resolution, and data format are consistent with the original classification product. Then fill all the block results into the corresponding positions to finally obtain a complete updated classification product.
[0101] Further, according to a method for refining land cover classification products based on image time series provided by the present invention, the step of selecting training samples from the candidate training samples specifically includes:
[0102] Determine the number of valid pixels corresponding to each of the category labels in the candidate training samples;
[0103] Randomly select m valid pixels from the number of pixels as training samples.
[0104] Specifically, the specific method is to count the number of valid data in each category, denoted as Ni, i represents the i-th category, and the number of training samples Si selected for each category is the e-th root of Ni, that is, Then, Si are randomly selected from the valid data of each category to form the training sample set of the current data block.
[0105] Combination Figure 4 As shown, the present invention provides a device for refining land cover classification products based on image time series, comprising:
[0106] An acquisition module 41 is used to acquire remote sensing images presenting a time series;
[0107] A repair module 42 is used to repair each image to be repaired in the remote sensing image by using other images other than each image to be repaired in the remote sensing image to obtain a repaired remote sensing image;
[0108] The refinement module 43 is used to refine and update the existing surface cover classification result pixels according to the repaired remote sensing image to obtain an updated surface cover classification result.
[0109] Since the device provided in the embodiment of the present invention can be used to execute the method described in the above embodiment, its working principle and beneficial effects are similar, so they are not described in detail here. For specific contents, please refer to the introduction of the above embodiment.
[0110] The present invention provides a device for refining land cover classification products based on image time series, which obtains remote sensing images presenting a time series; repairs each image to be repaired in the remote sensing image using other images other than each image to be repaired in the remote sensing image to obtain a repaired remote sensing image; and refines and updates the existing land cover classification result pixels according to the repaired remote sensing image to obtain an updated land cover classification result. The present invention quickly realizes reliable update of results by introducing new time information, enhances the utilization rate of related data and historical products, and fully utilizes new data conditions on the premise of inheriting the original results, so that the original products regain accuracy and usability.
[0111] Furthermore, according to a device for refining land cover classification products based on image time series provided by the present invention, the refining module 43 is specifically used for:
[0112] Constructing a correspondence between the pixels of the patched remote sensing image and the pixels of the existing land cover classification result;
[0113] The existing surface cover classification result pixels are refined and updated according to the repaired remote sensing image and the corresponding relationship to obtain an updated surface cover classification result.
[0114] Furthermore, according to a device for refining land cover classification products based on image time series provided by the present invention, the patching module 42 is specifically used for:
[0115] Marking clouds and cloud shadows on each image in the remote sensing image to obtain an image with cloud and cloud shadow pixels marked, and using the image with cloud and cloud shadow pixels marked as the image to be repaired;
[0116] Determine the similarity between other images in the remote sensing image other than each image to be repaired and the image to be repaired, and sort the similarities in descending order;
[0117] The image to be repaired is repaired in sequence according to the other images corresponding to the sorting to obtain a repaired remote sensing image.
[0118] Furthermore, according to the device for refining land cover classification products based on image time series provided by the present invention, the patching module 42 is further specifically used for:
[0119] Dividing the image to be repaired into a plurality of first image spots according to connectivity;
[0120] Acquire a second image spot at the same position as each of the first image spots in the other images;
[0121] The first image spots are filled with the second image spots in sequence according to the arrangement to obtain a repaired remote sensing image.
[0122] Furthermore, according to a device for refining land cover classification products based on image time series provided by the present invention, the refining module 43 is further specifically used for:
[0123] Aligning the pixels between the repaired remote sensing images to obtain a remote sensing image with pixel alignment in a time series;
[0124] The pixel-aligned remote sensing image is reprojected and resampled according to the projection mode and resolution of the existing land cover classification result to obtain the corresponding relationship between the pixels of the patched remote sensing image and the pixels of the existing land cover classification result.
[0125] Furthermore, according to a device for refining land cover classification products based on image time series provided by the present invention, the refining module 43 is further specifically used for:
[0126] Dividing the patched remote sensing image into blocks in a spatial dimension to obtain a plurality of first data blocks;
[0127] Acquire a second data block having the same coverage range as the first data block in the existing surface cover classification result image;
[0128] Performing fill value judgment on each pixel position in the first data block to obtain a valid pixel mask of the first data block;
[0129] Performing a morphological erosion operation on the second data block to obtain a category label of each pixel of the second data block;
[0130] Obtaining category labels corresponding to all valid pixels as candidate training samples according to the category labels and the valid pixel masks;
[0131] Filtering training samples from the candidate training samples;
[0132] Perform K-nearest neighbor classification on all the valid pixels according to the training samples to obtain a new category label for each valid pixel;
[0133] Replace the category of each pixel of the second data block according to the new category label of each valid pixel;
[0134] The second data block after replacement according to the category of each pixel obtains an updated surface cover classification result.
[0135] Furthermore, according to a device for refining land cover classification products based on image time series provided by the present invention, the refining module 43 is further specifically used for:
[0136] Determine the number of valid pixels corresponding to each of the category labels in the candidate training samples;
[0137] Randomly select m valid pixels from the number of pixels as training samples.
[0138] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute a method for refining a land cover classification product based on an image time series, the method comprising: acquiring remote sensing images presenting a time series; repairing each image to be repaired in the remote sensing images using other images other than the images to be repaired in the remote sensing images to obtain a repaired remote sensing image; and refining and updating the existing land cover classification result pixels according to the repaired remote sensing images to obtain an updated land cover classification result.
[0139] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0140] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute a method for refining a surface cover classification product based on an image time series provided by the above-mentioned methods, the method including: acquiring a remote sensing image presenting a time series; patching each image to be patched in the remote sensing image using other images in the remote sensing image other than each image to be patched to obtain a patched remote sensing image; and refining and updating the existing surface cover classification result pixels according to the patched remote sensing image to obtain an updated surface cover classification result.
[0141] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned method for refining a surface cover classification product based on an image time series, the method comprising: acquiring remote sensing images presenting a time series; repairing each image to be repaired in the remote sensing images using other images in the remote sensing images other than the each image to be repaired, to obtain a repaired remote sensing image; and refining and updating the existing surface cover classification result pixels according to the repaired remote sensing images, to obtain an updated surface cover classification result.
[0142] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for refining land cover classification products based on image time series, characterized in that: include: Acquire remote sensing images that present time series; Repairing each image to be repaired in the remote sensing image using other images other than each image to be repaired in the remote sensing image to obtain a repaired remote sensing image; Refining and updating the existing surface cover classification result pixels according to the repaired remote sensing image to obtain an updated surface cover classification result; The refining and updating of the existing surface cover classification result pixels according to the patched remote sensing image to obtain the updated surface cover classification result specifically includes: Constructing a correspondence between the pixels of the patched remote sensing image and the pixels of the existing land cover classification result; Refining and updating the existing surface cover classification result pixels according to the repaired remote sensing image and the corresponding relationship to obtain an updated surface cover classification result; The constructing of the correspondence between the pixels of the patched remote sensing image and the pixels of the existing land cover classification result specifically includes: Aligning the pixels between the repaired remote sensing images to obtain a remote sensing image with pixel alignment in a time series; Reprojecting and resampling the pixel-aligned remote sensing image according to the projection mode and resolution of the existing land cover classification result to obtain a corresponding relationship between the pixels of the patched remote sensing image and the pixels of the existing land cover classification result; The refining and updating of the existing surface cover classification result pixels according to the patched remote sensing image and the corresponding relationship to obtain an updated surface cover classification result specifically includes: Dividing the patched remote sensing image into blocks in a spatial dimension to obtain a plurality of first data blocks; Acquire a second data block having the same coverage range as the first data block in the existing surface cover classification result image; Performing fill value judgment on each pixel position in the first data block to obtain a valid pixel mask of the first data block; Performing a morphological erosion operation on the second data block to obtain a category label of each pixel of the second data block; Obtaining category labels corresponding to all valid pixels as candidate training samples according to the category labels and the valid pixel masks; Filtering training samples from the candidate training samples; Perform K-nearest neighbor classification on all the valid pixels according to the training samples to obtain a new category label for each valid pixel; Replace the category of each pixel of the second data block according to the new category label of each valid pixel; The second data block after replacement according to the category of each pixel obtains an updated surface cover classification result.
2. The method for refining land cover classification products based on image time series according to claim 1 is characterized in that: The method of repairing each image to be repaired in the remote sensing image by using other images other than each image to be repaired in the remote sensing image to obtain a repaired remote sensing image specifically includes: Marking clouds and cloud shadows on each image in the remote sensing image to obtain an image with cloud and cloud shadow pixels marked, and using the image with cloud and cloud shadow pixels marked as the image to be repaired; Determine the similarity between other images in the remote sensing image other than each image to be repaired and the image to be repaired, and sort the similarities in descending order; The image to be repaired is repaired in sequence according to the other images corresponding to the sorting to obtain a repaired remote sensing image.
3. The method for refining land cover classification products based on image time series according to claim 2 is characterized in that: The step of repairing the image to be repaired in sequence according to the other images corresponding to the sorting to obtain the repaired remote sensing image specifically includes: Dividing the image to be repaired into a plurality of first image spots according to connectivity; Acquire a second image spot at the same position as each of the first image spots in the other images; The first image spots are filled with the second image spots in sequence according to the arrangement to obtain a repaired remote sensing image.
4. The method for refining land cover classification products based on image time series according to claim 1, characterized in that: The step of selecting a training sample from the candidate training samples specifically includes: Determine the number of valid pixels corresponding to each of the category labels in the candidate training samples; Randomly select m valid pixels from the number of pixels as training samples.
5. A device for refining land cover classification products based on image time series, characterized in that: include: An acquisition module is used to acquire remote sensing images presenting a time series; A patching module, used for patching each to-be-patched image in the remote sensing image by using other images other than each to-be-patched image in the remote sensing image to obtain a patched remote sensing image; A refinement module, used for refining and updating the existing surface cover classification result pixels according to the repaired remote sensing image to obtain an updated surface cover classification result; The refinement module is specifically used for: Constructing a correspondence between the pixels of the patched remote sensing image and the pixels of the existing land cover classification result; Refining and updating the existing surface cover classification result pixels according to the repaired remote sensing image and the corresponding relationship to obtain an updated surface cover classification result; The refinement module is also specifically used for: Aligning the pixels between the repaired remote sensing images to obtain a remote sensing image with pixel alignment in a time series; Reprojecting and resampling the pixel-aligned remote sensing image according to the projection mode and resolution of the existing land cover classification result to obtain a corresponding relationship between the pixels of the patched remote sensing image and the pixels of the existing land cover classification result; The refinement module is also specifically used for: Dividing the patched remote sensing image into blocks in a spatial dimension to obtain a plurality of first data blocks; Acquire a second data block having the same coverage range as the first data block in the existing surface cover classification result image; Performing fill value judgment on each pixel position in the first data block to obtain a valid pixel mask of the first data block; Performing a morphological erosion operation on the second data block to obtain a category label of each pixel of the second data block; Obtaining category labels corresponding to all valid pixels as candidate training samples according to the category labels and the valid pixel masks; Filtering training samples from the candidate training samples; Perform K-nearest neighbor classification on all the valid pixels according to the training samples to obtain a new category label for each valid pixel; Replace the category of each pixel of the second data block according to the new category label of each valid pixel; The second data block after replacement according to the category of each pixel obtains an updated surface cover classification result.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for refining land cover classification products based on image time series as described in any one of claims 1 to 4 are implemented.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for refining land cover classification products based on image time series as described in any one of claims 1 to 4 are implemented.
