Classification model training method and land cover classification method, device and equipment

By constructing high-quality data sets and targeted training of classification models, the classification error of medium-resolution satellite image data sets is solved, the classification accuracy of forest coverage is improved, and more efficient forest coverage information monitoring is achieved.

CN119091183APending Publication Date: 2024-12-06AEROSPACE INFORMATION RES INST CAS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411009840.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the prior art, there are classification errors in the data set composed of medium-resolution satellite images, which affects the classification accuracy of forest coverage. How to improve the training effect of classification models and the classification accuracy of forest coverage are technical problems that need to be solved urgently.

Method used

By determining the classification images corresponding to the surface image, selecting the sample images and determining the classification labels based on the classification results of the target covering, building a high-quality data set, and targeted training of the classification model to improve the training efficiency and effect of the model.

Benefits of technology

The training effect of the classification model and the classification accuracy of the target covering are improved, classification errors are reduced, and the monitoring accuracy of forest coverage information is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091183A_ABST
    Figure CN119091183A_ABST
Patent Text Reader

Abstract

The invention discloses a classification model training method and a land cover classification method. The method comprises the following steps: determining a first classification image and a second classification image corresponding to a land image; performing sample selection on the earth surface image according to the first classification image to obtain a sample image; determining a classification label corresponding to the sample image according to a classification result of the target covering in the second classification image; and constructing a data set based on the sample image and the classification label corresponding to the sample image, and training a classification model based on the data set to obtain a classification model for classifying the target covering. According to the method, the effective sample image is selected from the earth surface image according to the first classification image, and the classification label corresponding to the sample image is determined according to the second classification image, so that the high-quality data set is constructed, the classification model is trained in a targeted manner, the efficiency and effect of model training are improved, and the classification accuracy of the target covering is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to model training technology, and in particular to a classification model training method and a ground cover classification method, device, and electronic equipment. Background Art

[0002] In ecological research, the monitoring of forest cover information is crucial. In the existing technology, medium-resolution satellite images are usually used to monitor forest cover information. However, due to the limitations of data sources and modeling methods, the data sets composed of these medium-resolution satellite images usually have considerable classification errors, which seriously affects the training effect of the classification model and reduces the classification accuracy of forest cover. Therefore, how to improve the training effect of the classification model and thus improve the classification accuracy of forest cover is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present application are intended to provide a classification model training method and a ground cover classification method, device, and electronic device.

[0004] The technical solution of this application is implemented as follows:

[0005] In a first aspect, the present application provides a method for training a classification model, the method comprising:

[0006] Determine a classification image corresponding to the surface image, the classification image comprising: a first classification image and a second classification image, the first classification image comprising classification results of multiple coverings, and the second classification image comprising classification results of target coverings;

[0007] Selecting samples of the surface image according to the first classification image to obtain a sample image;

[0008] Determining a classification label corresponding to the sample image according to a classification result of the target cover in the second classification image;

[0009] A data set is constructed based on the sample image and the classification label corresponding to the sample image, and a classification model is trained based on the data set to obtain a classification model for classifying the target covering.

[0010] In a second aspect, the present application provides a method for classifying ground cover, the method comprising:

[0011] Acquire a surface image to be classified;

[0012] Inputting the surface image to be classified into a classification model for classifying target covering objects, and obtaining a classification result corresponding to the target covering object in the surface image to be classified;

[0013] The classification model for classifying target coverings is obtained based on the training method of the classification model described in the first aspect.

[0014] In a third aspect, the present application provides a training device for a classification model, the device comprising:

[0015] A determination unit, configured to determine a classification image corresponding to the surface image, the classification image comprising: a first classification image and a second classification image, the first classification image comprising classification results of a plurality of coverings, the second classification image comprising a classification result of a target covering;

[0016] A first processing unit is configured to select samples of the surface image according to the first classification image to obtain a sample image; determine a classification label corresponding to the sample image according to a classification result of the target cover in the second classification image; and construct a data set based on the sample image and the classification label corresponding to the sample image;

[0017] A training unit is used to train a classification model based on the data set to obtain a classification model for classifying the target covering.

[0018] In a fourth aspect, the present application provides a ground cover classification device, the device comprising:

[0019] An acquisition unit, used for acquiring a surface image to be classified;

[0020] A second processing unit is used to input the surface image into a classification model for classifying target covers to obtain a classification image corresponding to the target cover in the surface image;

[0021] The classification model for classifying target coverings is obtained based on the training method of the classification model described in the first aspect.

[0022] In a fifth aspect, the present application provides an electronic device, comprising: a processor and a memory configured to store a computer program that can be run on the processor,

[0023] Wherein, the processor is configured to execute the steps of the method described in the first aspect or the second aspect when running the computer program.

[0024] In an embodiment of the present application, a training method for a classification model and a classification method, device, and equipment for ground cover are provided, the method comprising: determining a classification image corresponding to a ground surface image, the classification image comprising: a first classification image and a second classification image, the first classification image comprising classification results of a plurality of covers, the second classification image comprising classification results of a target cover; performing sample selection on the ground surface image according to the first classification image to obtain a sample image; determining a classification label corresponding to the sample image according to the classification result of the target cover in the second classification image; constructing a data set based on the sample image and the classification label corresponding to the sample image, and training a classification model based on the data set to obtain a classification model for classifying the target cover. In this way, effective sample images can be selected from the ground surface image according to the first classification image, and the classification label corresponding to the sample image can be determined according to the second classification image, thereby constructing a high-quality data set, conducting targeted training on the classification model, improving the efficiency and effect of model training, and thereby improving the classification accuracy of the target cover. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A flow chart of a training method for a classification model in an embodiment of the present application;

[0026] Figure 2 This is a flow chart of the classification processing method in the embodiment of the present application;

[0027] Figure 3 A schematic diagram of the composition structure of the classification model in the embodiment of the present application;

[0028] Figure 4 This is a schematic diagram of the process of obtaining a forest coverage map in an embodiment of the present application;

[0029] Figure 5 This is a schematic diagram of the weakly supervised forest classification method in an embodiment of the present application;

[0030] Figure 6 This is a schematic diagram of the composition structure of the spectrum-space network in the embodiment of the present application;

[0031] Figure 7 This is a schematic diagram of the synthesis of multi-date classification results in the embodiment of the present application;

[0032] Figure 8 This is a schematic diagram showing the forest coverage map and its local accuracy and details in the embodiment of the present application;

[0033] Fig. 9 A schematic diagram of a flow chart of a method for classifying ground cover in an embodiment of the present application;

[0034] Fig.10This is a schematic diagram of the structure of the classification model training device in the embodiment of the present application;

[0035] Fig.11 This is a schematic diagram of the structure of the ground cover classification device in the embodiment of the present application;

[0036] Fig.12 It is a schematic diagram of the structure of the device in the embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0038] Figure 1 Schematic diagram of the training method of the classification model in the embodiment of the present application, such as Figure 1 As shown, the training method of the classification model may specifically include:

[0039] Step 101: determining a classification image corresponding to a surface image, the classification image comprising: a first classification image and a second classification image, the first classification image comprising classification results of various coverings, and the second classification image comprising classification results of a target covering;

[0040] A surface image refers to an image corresponding to a surface area, which is used to reflect the characteristics of the surface area, including but not limited to topography, vegetation cover, water distribution, land use type, urban structure, etc. The embodiment of the present application does not limit the type and number of features contained in the surface area, which can be set according to actual conditions.

[0041] The first classification image includes classification results of various coverings in the surface image, and the various coverings include target coverings. The first classification image is used to select effective sample images from the surface image, and the sample images are beneficial to improving the classification model training effect.

[0042] The second classification image includes the classification result of the target cover in the surface image as the classification label of the sample image.

[0043] The covering material may be any material form on the ground surface. Exemplarily, the covering material may include at least one of the following: crops, forests, grasslands, shrubs, wetlands, waters, tundra, artificial surfaces, bare land, ice and snow, etc.

[0044] The target cover may be any cover on the ground surface. For example, the target cover may be a forest, a wetland, or a water area.

[0045] In some embodiments, a surface image of one or more surface areas is acquired; a covering in the surface image is classified to obtain a classified image corresponding to the surface image, the classified image comprising: a first classified image and a second classified image, the first classified image comprising classification results of multiple coverings in the surface image, the second classified image comprising classification results of target coverings in the surface image;

[0046] One or more surface areas may include any geographical area of ​​any country or region on the surface of the earth. For example, the surface area may include: area a of country A, area b of country A, area c of country B, area d of country B, etc. The specific location and number of the surface areas are not limited in the present embodiment of the application and can be set according to actual conditions.

[0047] The classification process is used to identify one or more coverings in the surface image to obtain a classified image, and the classified image can be used as prior knowledge of the classification model. In some embodiments, obtaining surface images of one or more surface areas includes: obtaining surface images of one or more surface areas collected by an image acquisition device at different times / dates. Exemplarily, the image acquisition device can be a remote sensing instrument such as a panchromatic, multispectral, or hyperspectral camera, a synthetic aperture radar, and a stereo mapping camera on a satellite remote sensing platform. The surface image can be a satellite remote sensing image of one or more geographical areas. Correspondingly, this embodiment can be implemented as: obtaining satellite remote sensing images of one or more geographical areas collected by a remote sensing instrument on a satellite remote sensing platform at different times / dates.

[0048] In some embodiments, the surface image may be a medium-resolution satellite remote sensing image. In some embodiments, the cover in the satellite remote sensing images of one or more geographic areas is classified to obtain a first classified image of multiple cover in one or more satellite remote sensing images, and a second classified image of the forest.

[0049] In some embodiments, Figure 2 Flow chart of the classification processing method in the embodiment of the present application, such as Figure 2 As shown, determining the classified image corresponding to the surface image may specifically include:

[0050] Step 201: determining a plurality of initial classification images corresponding to a surface image;

[0051] The initial classification image refers to a classification image generated after classifying the cover objects in the surface image. At this time, the resolutions and cover object categories of different initial classification images may be inconsistent.

[0052] In some embodiments, the cover in the surface image is classified based on at least one surface cover classification algorithm to obtain a plurality of initial classified images of one or more surface areas;

[0053] The surface cover classification algorithm is used to classify the cover in the surface image. Exemplarily, different surface cover classification algorithms can be supervised classification methods, unsupervised classification methods, decision tree methods, synthetic aperture radar (SAR) image processing algorithms, etc. The embodiment of the present application does not limit the type of surface cover classification algorithm, and it can be set according to actual conditions.

[0054] Step 202: reclassifying and resampling the multiple initial classified images to obtain multiple preprocessed classified images with consistent categories and resolutions;

[0055] Reclassification refers to reclassifying multiple initial classified images so that the cover categories of the reclassified images are consistent with the cover categories of the reference standard classification system, that is, the cover categories of different products have a one-to-one correspondence. The initial classified images may at least include classified images obtained using one or more classification systems such as GlobeLand30, FROM-GLC 10, GLC FCS0, WorldCover, Esri Land Cover, and TreeCover.

[0056] Exemplarily, the classification system of GlobeLand30 is used as a reference standard classification system to reclassify multiple initial classified images of any surface area in one or more surface areas, and each initial classified image is remapped to the reference standard classification system according to the definition of the category of each initial classified image and the internal connection. The classification system of GlobeLand30 includes at least ten categories: farmland, forest, grassland, shrub, wetland, water body, tundra, artificial surface, bare land, ice and snow; the classification system of Esri Land Cover includes at least nine categories: water body, tree, grass, submerged vegetation, crop, dwarf tree / shrub, building, bare land, snow / ice, cloud. The mapping relationship between Esri Land Cover and GlobeLand30 is as follows: water body->water body, tree->forest, grass->grassland, submerged vegetation->wetland, crop->farmland, dwarf tree / shrub->brush, building->artificial surface, bare land->bare land, snow / ice->ice and snow. The above mapping relationship is the cover category of EsriLand Cover->cover category of GlobeLand30. Since there is no cloud category in GlobeLand30, the cloud category can be discarded in the mapping process, that is, there is no cloud category in the mapped Esri Land Cover, and the cloud pixels corresponding to the cloud category do not provide category information. Therefore, reclassification can map the initial category information of the initial classified image to the reference standard classification system without changing the original spatial position and category information of the initial classified image, and give the category information of the standard classification system to the initial classified image to obtain the reclassified classified image, so that the cover categories of the reclassified multiple classified images are consistent.

[0057] Resampling refers to resampling multiple initial classified images so that the resolutions of the resampled classified images are consistent, which is beneficial for subsequent processing of the multiple classified images. Exemplarily, a preset value is set so that the resolutions of GlobeLand30, FROM-GLC10, GLC FCS0, WorldCover, Esri Land Cover, and TreeCover after resampling are all the preset value, and the preset value can be set to 16 meters.

[0058] The preprocessed classified image refers to the classified image after reclassification and resampling of the initial classified image.

[0059] In some embodiments, a plurality of initial classified images of a first surface region are reclassified and resampled to obtain a plurality of preprocessed classified images of the first surface region having consistent categories and resolutions, wherein the first surface region is any one of the one or more surface regions;

[0060] The embodiment of the present application can reclassify and resample multiple initial classified images to obtain multiple preprocessed classified images with consistent categories and resolutions, so that the multiple preprocessed classified images are comparable.

[0061] Step 203: voting for each covering in the plurality of pre-processed classified images to obtain a voting map for each covering;

[0062] Voting is to count the number of times that the same pixel position is identified as a specific covering by multiple pre-processed classification images. The voting map includes the number of votes for each pixel position to be identified as a specific covering after counting.

[0063] In some embodiments, voting is performed for each type of cover in a plurality of preprocessed classified images of the first surface area to obtain a voting map for each type of cover;

[0064] Correspondingly, this embodiment can be implemented as follows: for the same pixel position in multiple pre-processed classified images of the first surface area, the pixel position is identified as a covered object, and when it is identified as a specific covered object, the number of votes for the specific covered object is generated. The above voting steps are traversed and performed for each covered object to obtain a voting map for each covered object.

[0065] Step 204: determining a high consistency region of each covering according to the voting map and voting threshold of each covering;

[0066] A high consistency region refers to a region in the voting map where the number of votes is greater than the voting threshold. A high consistency region is used to characterize that the probability that the coverage in the region is a specific coverage is high.

[0067] In some embodiments, a high consistency region for each covering in the first surface region is determined based on a voting map and a voting threshold for each covering in the first surface region.

[0068] In some embodiments, based on the voting map and voting threshold of each covering in the first surface area, high consistency areas for each covering in the first surface area are determined, including: based on the voting map of the target covering in the first surface area and the first voting threshold, determining high consistency areas for the target covering category; based on the voting maps of other coverings in the first surface area and the second voting threshold, determining high consistency areas for other covering categories; wherein the first voting threshold is greater than the second voting threshold.

[0069] The first voting threshold refers to the voting threshold for the target cover. The second voting threshold refers to the voting threshold for other covers. The first voting threshold is greater than the second voting threshold, which makes the standard for determining the high consistency area of ​​the target cover category more stringent, thereby improving the accuracy of the classification result of the target cover.

[0070] Other coverings include one or more coverings other than the target covering.

[0071] In some embodiments, the voting threshold of the forest category is set to be greater than the voting threshold of the non-forest category. Exemplarily, the voting threshold of the forest category is set to 4, and then the area with more than 4 votes is set as a high consistency area of ​​the forest; the voting threshold of the non-forest category is set to 3, and then the area with more than 3 votes is set as a high consistency area of ​​the non-forest. The embodiment of the present application does not limit the values ​​of the first voting threshold and the second voting threshold, which can be set according to actual conditions.

[0072] Step 205: Fusing the high consistency regions of each covering to obtain a first classified image;

[0073] In some embodiments, the high consistency regions of each covering category are fused to obtain a first classified image of the first surface area. The embodiment of the present application can obtain the classification result of each covering through the first classified image, thereby achieving classification of the covering.

[0074] Step 206: Fusing the high consistency area of ​​the non-target covering in the first classification image to obtain a second classification image;

[0075] The non-target covering may include coverings other than the target covering, and may also include an uncertain area, where the uncertain area refers to an area that does not belong to any covering category.

[0076] In some embodiments, the high consistency area of ​​non-target cover in the first classified image is fused to obtain a second classified image of the first surface area. Correspondingly, this embodiment can be implemented as: fusing the high consistency area of ​​non-forest in the first classified image to obtain a second classified image including the high consistency area of ​​non-forest and the high consistency area of ​​forest. The embodiment of the present application can obtain the classification result of the target cover through the second classified image, thereby realizing the classification of the target cover.

[0077] In some embodiments, the classification image further includes: a third classification image, the third classification image includes a high consistency area, a medium consistency area, and a low consistency area of ​​the target cover in the surface image. Classifying the cover in the surface image to obtain a classification image corresponding to the surface image also includes: when the number of votes for each pixel in the first area in the voting map of the target cover is greater than the third voting threshold, the first area is determined to be a high consistency area of ​​the target cover; when the number of votes for each pixel in the second area in the voting map of the target cover is less than or equal to the third voting threshold and greater than or equal to the fourth voting threshold, the second area is determined to be a medium consistency area of ​​the target cover; when the number of votes for each pixel in the third area in the voting map of the target cover is less than the fourth voting threshold, the third area is determined to be a low consistency area of ​​the target cover; wherein the third voting threshold is greater than or equal to the fourth voting threshold.

[0078] The embodiment of the present application obtains high consistency areas, medium consistency areas and low consistency areas of the target covering in the surface image through the third classification image, so that the probability of the covering in the surface image being the target covering can be intuitively judged based on the third classification image.

[0079] In some embodiments, a forest spatial distribution consistency map is obtained based on the voting map of the forest category and different voting thresholds. Exemplarily, the voting threshold division rule may include: setting the area with a voting number greater than 5 as a high consistency area; setting the area with a voting number less than or equal to 5 and greater than or equal to 3 as a medium consistency area; and setting the area with a voting number greater than 0 and less than 3 as a low consistency area.

[0080] Step 102: Selecting samples of the surface image according to the first classification image to obtain a sample image;

[0081] The sample image refers to a part or all of the surface image obtained by sampling the surface image, and the sample image can improve the efficiency and effect of model training. Exemplarily, the sample image can be an image including the target cover, and the sample image can also include one or more covers similar to the target cover.

[0082] In some embodiments, sample selection is performed on a surface image according to a first classification image to obtain a sample image, including: dividing the first classification image into image blocks of a preset size to obtain classification image blocks; dividing the surface image into image blocks of a preset size to obtain surface image blocks; and selecting a surface image block corresponding to a target classification image block as a sample image according to a preset sample selection rule, wherein the sample selection rule defines one or more covering features of the classification image blocks.

[0083] The cover feature refers to the cover that is easily confused with the target cover. For example, the cover feature may include water bodies, farmland, grassland, wetlands and other cover categories.

[0084] The sample selection rule is used to define the covering features of one or more classified image blocks.

[0085] The target classification image block refers to a classification image block that meets the preset sample selection rules, which contains one or more covering features of the classification image blocks defined by the preset sample selection rules. Exemplarily, when the classification image block contains one or more coverings of water bodies, farmlands, grasslands, and wetlands, the classification image block is determined to be a target classification image block.

[0086] In some embodiments, a 64x64 sliding window without overlapping areas is used to divide the first classification image and the surface image into image blocks to obtain classification image blocks and surface image blocks, and the multiple classification image blocks after division are detected one by one to see whether they contain one or more of the coverings that are easily confused with the target coverings. If the classification image block contains the coverings that are easily confused with the target coverings, the classification image block is determined to be the target classification image block, and the surface image block corresponding to the target classification image block is selected as the sample image. There may be one or more target classification image blocks, and correspondingly, there may also be one or more sample images.

[0087] The embodiment of the present application determines a target classification image block that contains a covering that is easily confused with a target covering, selects a surface image block corresponding to the target classification image block as a sample image, and trains a classification model. In this way, during the training process of the classification model, the characteristics of the covering that is easily confused with the target covering can be learned, and the misclassification of other coverings as target coverings can be avoided, thereby improving the classification accuracy of the target covering.

[0088] Step 103: Determine a classification label corresponding to the sample image according to the classification result of the target cover in the second classification image;

[0089] The classification label is used to indicate the category of the covering object corresponding to the pixel position. Exemplarily, the classification label determined according to the classification result of the target covering object may be a forest label or a non-forest label.

[0090] Step 104: constructing a data set based on the sample image and the classification label corresponding to the sample image, and training a classification model based on the data set to obtain a classification model for classifying target coverings;

[0091] The classification model refers to a model used to classify target coverings. In some embodiments, during the classification model training phase, orthorectification and cloud detection are performed on sample images, and the processed sample images are input into the classification model.

[0092] In some embodiments, a classification model is trained based on a data set to obtain a classification model for classifying target coverings, including: randomly selecting some sample images and classification labels corresponding to some sample images from the data set to construct a training set and a verification set; using the training set to perform preliminary training on the classification model to obtain a preliminary trained classification model; using the verification set to verify the preliminary trained classification model to obtain an optimized classification model; continuing to perform the step of training the optimized classification model using the data set until a verified classification model is obtained; in the process of verifying the preliminary trained classification model using the verification set, the method also includes: determining the classification result of the predicted probability image based on the predicted probability of each pixel of the predicted probability image output by the preliminary trained classification model for the target covering and a first threshold; when the predicted probability of the first pixel of the predicted probability image for the target covering is greater than the second threshold, if the classification result of the first pixel in the classification label is inconsistent with the classification result of the first pixel in the predicted probability image, dynamically correcting the classification result of the first pixel in the classification label.

[0093] The training set refers to the data set used to train the classification model.

[0094] The validation set refers to the data set used to validate the classification model.

[0095] The prediction probability image refers to an image that contains the prediction probability of each pixel.

[0096] A verified classification model refers to a classification model whose classification result of the target covering reaches a preset accuracy. Exemplarily, if the accuracy of the classification result of the target covering output by the classification model reaches a preset threshold, it means that the classification result of the target covering output by the classification model reaches the preset accuracy, and the classification model passes the verification.

[0097] The first threshold is used to determine the classification result of each pixel of the predicted probability image for the target cover. The embodiment of the present application does not limit the specific value of the first threshold, which can be set according to actual conditions.

[0098] The first pixel refers to any pixel in the prediction probability image.

[0099] The second threshold is used to determine whether to dynamically correct the classification result of the first pixel in the classification label when the classification result of the first pixel in the classification label is inconsistent with the classification result of the first pixel in the predicted probability image. The embodiment of the present application does not limit the specific value of the second threshold, which can be set according to actual conditions.

[0100] In some embodiments, the model prioritizes learning correct samples in the initial stage of training. When the model optimized by noise learning and initial training set has a certain learning ability, the predicted probability is used to dynamically correct the pseudo-labels. In order to reduce the error in sample correction caused by insufficient model classification ability, only the labels of high predicted probability areas of forest / non-forest are adjusted, while the pseudo-labels of non-high predicted probability areas are kept unchanged.

[0101] In the embodiment of the present application, the classification model is firstly trained using the training set to obtain a preliminary trained classification model; secondly, the classification label is dynamically corrected using the verification set and the predicted probability image output by the preliminary trained classification model, and the corrected classification label is input into the classification model to obtain an optimized classification model; finally, the step of training the optimized classification model using the data set is continued until a verified classification model is obtained. In this way, the classification model can be trained multiple times until the classification model passes the verification. In this way, when the classification model is used to classify the target covering, a more accurate target covering classification result can be obtained.

[0102] In an embodiment of the present application, the classification result of the first pixel in the classification label that is inconsistent with the classification result of the first pixel in the predicted probability image is dynamically corrected only when the predicted probability of the first pixel in the predicted probability image for the target cover is greater than the second threshold. In this way, the accuracy of the classification result of the first pixel in the predicted probability image can be improved. Moreover, since the classification result of the first pixel in the classification label is corrected based on the classification result of the first pixel in the predicted probability image, the accuracy of the classification result of the first pixel in the classification label is also improved.

[0103] In some embodiments, training a classification model based on a data set to obtain a classification model for classifying target cover also includes: constructing a test set based on the data set; inputting multi-date sample images of a second surface area in the test set into the verified classification model respectively to classify the target cover to obtain a multi-date predicted probability image of the second surface area; merging the multi-date predicted probability images of the second surface area to obtain a predicted classified image of the second surface area; marking special areas of the sample image using the classification result of the predicted classified image for the target cover to optimize the data set, and training the classification model based on the optimized data set to obtain a classification model for classifying the target cover; the special areas include: areas where the classification result of the predicted classified image for the target cover and the classification result of the second classified image for the target cover are inconsistent, and / or, areas predicted as target cover in the predicted classified image and predicted as uncertain by the second classified image.

[0104] The test set refers to a data set used to test the classification model. The test set can be a portion of sample images randomly selected from the data set and their corresponding classification labels, or it can be the entire data set. This embodiment of the application does not limit this and can be set according to actual conditions.

[0105] The second surface area refers to the surface area corresponding to the multi-date sample images.

[0106] A predicted classification image refers to an image that contains the predicted classification results of each pixel.

[0107] In some embodiments, the characteristics of sample selection in active learning are utilized to select informative and representative samples as training sets. In the initial sample determination stage, categories that are easily confused with forests are considered, including water bodies, farmlands, grasslands, wetlands, etc. When the model has a certain learning ability, based on the easily confused categories, priority is given to areas where the model classification may have problems, including: (1) areas that are considered to be forests by pseudo-labels but predicted to be non-forests; (2) areas that are considered to be non-forests by pseudo-labels but predicted to be forests; (3) areas where pseudo-labels are uncertain but predicted to be forests.

[0108] The samples are selected using a 64x64 sliding window with no overlapping areas, the image is divided into patches, and 5% and 5% of the patches that meet the conditions are randomly selected as the training set and validation set, respectively.

[0109] The embodiment of the present application can input the multi-date sample images in the test set into the verified classification model to obtain the multi-date prediction probability image, merge the multi-date prediction probability images to obtain the prediction classification image, use the classification result of the prediction classification image for the target covering to mark the special area of ​​the sample image to optimize the data set, and train the classification model based on the optimized data set, the special area includes: the classification result of the prediction classification image for the target covering and the classification result of the second classification image for the target covering are inconsistent, and / or, the predicted classification image predicts to be the target covering and the second classification image predicts to be the uncertain area, so that the model training can be carried out for the special area in the sample image, that is, the difficult sample area, thereby improving the efficiency and effect of the model training, and further improving the accuracy of the classification results.

[0110] In some embodiments, a multi-date cloud detection result of a multi-date sample image of a second surface area and a third classified image of the second surface area are obtained; the multi-date predicted probability images of the second surface area are merged to obtain a predicted classified image of the second surface area, including: determining coverage information of the second surface area according to the multi-date cloud detection result of the second surface area, the coverage information including the number of times each pixel in the second surface area is marked as valid; determining classification information of the second surface area according to the multi-date predicted probability image of the second surface area and the third classified image of the second surface area, the classification information including the number of times each pixel in the second surface area is marked as a target cover in the multi-date predicted probability image; determining probability information of the second surface area according to the coverage information and classification information of the second surface area, the probability information including the probability of each pixel in the second surface area being marked as a target cover; determining the predicted classified image of the second surface area according to the probability of each pixel in the second surface area being marked as a target cover and a probability threshold.

[0111] The cloud detection result is used to determine whether a pixel is a valid value area pixel, or in other words, to determine whether a pixel is a cloud pixel. When a pixel is a cloud pixel, it means that the classification result for the pixel will be interfered by the cloud, making the classification result inaccurate.

[0112] The coverage information is used to indicate the number of times each pixel of the second surface area is marked as a valid value area pixel on multiple dates. Exemplarily, the coverage information may be a valid image area mark (VIM).

[0113] The classification information is used to indicate the number of times each pixel in the second surface area is marked as a target cover. Exemplarily, the classification information may be a forest classification mark (FRM).

[0114] The probability information is used to represent the probability that each pixel in the second surface area is marked as the target cover. Exemplarily, the probability information can be represented by p, and the probability p of each pixel being marked as a forest can be calculated through VIM and FRM.

[0115] The probability threshold refers to the threshold used to determine the target cover classification result of the pixel. The probability threshold can be a fixed threshold. For example, the probability threshold can be set to 0.5, or the probability threshold can also be set to other values. The embodiment of the present application does not limit the value of the probability threshold, and it can be set according to the actual situation.

[0116] In some embodiments, a final forest cover map is synthesized based on multi-date classification results to reduce the impact of cloud detection results on classification results; the prediction probability of a single-date image is post-processed based on a threshold segmentation method that combines overall constraints of prior knowledge (forest spatial consistency distribution map) with local adaption; the post-processing is performed using a moving window with an overlap rate of 50%, and the moving window size is 500x500 pixels; the classification results of the single-date image are synthesized based on a fixed threshold, and areas with a prediction probability greater than 50% are regarded as the final forest cover results.

[0117] In some embodiments, classification information of the second surface area is determined based on a multi-date prediction probability image of the second surface area and a third classification image of the second surface area, including: obtaining high consistency areas, medium consistency areas, and low consistency areas of target cover in the second surface area based on the third classification image of the second surface area; obtaining a single-date prediction probability image of the second surface area; obtaining the prediction probability of each pixel in the second surface area for the target cover based on the single-date prediction probability image of the second surface area; obtaining the average value of the prediction probability of all pixels in the high consistency area, the medium consistency area, and the low consistency area for the target cover respectively; processing the average value of the prediction probability in the high consistency area, the medium consistency area, and the low consistency area to obtain an overall threshold and a local threshold; wherein the overall threshold is used to constrain the local threshold; obtaining the multi-date prediction probability of each pixel in the second surface area for the target cover based on the multi-date prediction probability image of the second surface area; and determining the classification information of the second surface area based on the overall threshold, the local threshold, and the multi-date prediction probability of each pixel for the target cover.

[0118] In some embodiments, satellite remote sensing images with a higher resolution than the surface images obtained in the present application may also be used to verify the classification results of the target cover in the predicted classification image of the second surface area to determine whether the classification accuracy and reliability of the target cover in the predicted classification image meet the requirements.

[0119] The embodiment of the present application can synthesize the classification results of a single date based on the multi-date cloud detection results of the second surface area, the multi-date predicted probability image of the second surface area and the third classification image of the second surface area to obtain a multi-date classification result. The multi-date classification result reduces the influence of the cloud detection results on the classification results and improves the accuracy of the classification results.

[0120] In some embodiments, the classification model is a classification model based on deep learning, such as Figure 3As shown, the classification model includes: a spectral modeling module, a spatial modeling module and a classification module; the classification model is configured to perform the following steps: the sample image is input into the spectral modeling module to extract the spectral difference information between the target covering and the non-target covering in the sample image; the sample image is input into the spatial modeling module to extract the spatial context features of the sample image; the spectral difference information and the spatial context features are fused as the input of the classification module to obtain a predicted probability image; the predicted probability image includes the predicted probability of each pixel for the target covering.

[0121] The spectral modeling module can enhance the features of the edge and transition areas of the target coverage while maintaining the original resolution by extracting the spectral differences between the target coverage and the non-target coverage in the sample image.

[0122] Spatial context features are used to characterize features based on the relative positions and structural relationships between pixels in sample images.

[0123] The spatial modeling module extracts the spatial context features of the sample image and can use the neighborhood information of pixels to infer the boundaries of the target coverage.

[0124] The classification module is used to classify the target cover using the fused spectral difference information and spatial context features as input to obtain a predicted probability image of the target cover. In some embodiments, considering the characteristics of medium-resolution multispectral images, such as weak texture, severe mixed pixels, but rich spectral information, irregular geometric boundaries of forests, and lack of sufficient texture features, a network for medium-resolution satellite image forest classification is designed; the spectral difference information and spatial context features are fused as input to the classification layer of the network to obtain the predicted probability of forest / non-forest.

[0125] In some embodiments, the spectral modeling module includes a multi-layer perceptron. Exemplarily, the multi-layer perceptron is a feed-forward neural network, which includes at least three layers of nodes: an input layer, a hidden layer, and an output layer.

[0126] In some embodiments, the spatial modeling module includes a shallow U-shaped network structure. Exemplarily, the basic unit of the shallow U-shaped network structure is a residual block, which is used to capture spatial context features; the spatial pyramid pooling module (ASPP block) refers to a pooling technology used in computer vision tasks, especially in convolutional neural networks (CNN), which is built into the spatial modeling module to expand the model receptive field and extract multi-scale feature information.

[0127] In some embodiments, the classification module includes one or more convolutional layers and one or more activation layers.

[0128] In an embodiment of the present application, a deep learning-based classification model is constructed, which includes a spectral modeling module, a spatial modeling module and a classification module. The spectral difference information between the target cover and the non-target cover in the sample image extracted by the spectral modeling module and the spatial context features of the sample image extracted by the spatial modeling module are fused and input into the classification module to obtain the predicted probability of each pixel for the target cover. In this way, the shortcomings of medium-resolution images such as weak texture and severe mixed pixels, as well as the irregular geometric boundaries of the forest and the lack of sufficient texture features are taken into account, thereby improving the accuracy of the target cover classification results.

[0129] Below, the training method of the classification model provided in the embodiment of the present application is explained through a complete example.

[0130] Forests are an important part of terrestrial ecosystems and are closely related to the global carbon cycle, hydrological cycle and biodiversity. The international community attaches great importance to the protection and monitoring of forests, but in the past three decades, the rate of forest change and degradation has continued to occur at an alarming rate. Forest monitoring is indispensable for diverse ecosystems, and forest cover information is an important input for ecosystem service assessment, natural disaster monitoring, climate change and carbon storage, as well as ecological research and related studies. The accuracy and reliability of forest cover information profoundly affect the scientific nature of relevant conclusions. Due to climate change and human activities (such as urban development, afforestation and desertification control), the forest area in a certain country has changed significantly. On the one hand, these changes have increased forest coverage and improved the climate, but on the other hand, they may also bring some ecological and environmental threats, including the reduction of natural forests, forest pests and diseases, etc. Accurate and timely drawing of forest maps is essential for the sustainable development and management of forests. Therefore, there is an urgent need to draw and update forest cover maps.

[0131] Remote sensing technology plays an increasingly important role in forest mapping due to its powerful data collection capabilities and the advantage of repeated observations in a relatively short period of time. Early forest monitoring mostly used coarse-resolution satellite data. In recent years, due to the advancement of sensor technology, medium- and high-resolution satellite images have become increasingly important in large-scale forest monitoring. These higher-resolution images can provide more accurate and detailed land cover boundary information, which is helpful for extracting information such as forest cover and plantations. Using these images, scholars have achieved a series of influential results and released a series of public data sets. These data sets provide a large amount of information on land cover and contribute rich prior knowledge and experience to subsequent forest research. However, due to the limitations of data sources and modeling methods, these data sets have considerable classification errors. These errors have affected the assessment of forests to a certain extent. In addition, inconsistencies in data quality, mapping standards, and spatiotemporal coverage of data sources often produce different results, making these data sets difficult to use for accurate, reliable, and comprehensive socioeconomic assessments.

[0132] Land cover classification is usually performed using supervised classification methods. Advances in classification methods have improved the accuracy of classification results. Traditional remote sensing-based forest classification mostly uses shallow machine learning methods, including decision trees, random forests, multi-layer perceptrons, support vector machines, and maximum likelihood classification. In particular, decision trees and random forests are widely used in forest classification due to their robustness to noise and their advantages in processing multidimensional data. Classical classification algorithms rely on artificially selected features (such as spectral features, spectral index features, texture features, etc.) as the input of the classifier. The representativeness of the input features significantly affects the accuracy of the classification.

[0133] In recent years, many deep learning methods have been used for forest monitoring, with good performance in applications such as independent tree crown extraction, forest change detection, leaf phenology monitoring, and forest tree species. However, most of the existing deep learning-based forest research focuses on high spatial resolution drone or airborne platform image data, while few studies use medium-resolution satellite images. In addition, such algorithms rely heavily on a large number of finely labeled samples, and it is challenging to obtain a large number of well-labeled samples. Public land cover products provide pixel-level annotations. Using these data as pixel-level labels is a common method in large-scale land cover classification, but there are a lot of errors in these data sets. The development of weak supervision technology reduces the need for sample labeling and proves the robustness of the model to inaccurate and incomplete labels. This shows that deep learning methods have broad application prospects in forest research.

[0134] The embodiment of the present application aims to achieve practical national-scale forest cover classification using medium-resolution remote sensing images, utilize the different surface cover distribution information of shared surface cover classification products, construct three different types of prior knowledge through voting information of different cover categories and the spatial distribution pattern of the accuracy of each product, and combine the designed weakly supervised classification framework for forests in medium-resolution satellite images to achieve large-scale forest cover classification, which acts in the direction of vegetation remote sensing, specifically involving forest classification in medium-resolution satellite images.

[0135] The embodiment of the present application utilizes the prior knowledge and weakly supervised noise learning provided by existing products, and proposes a practical medium-resolution forest classification method to realize the extraction of forest cover information from large-scale medium-resolution satellite remote sensing data. This embodiment is based on medium-resolution satellite remote sensing data, and with the support of existing products, constructs prior information such as forest distribution spatial consistency map, high-precision surface cover map and forest / non-forest pseudo-labels, and combines the deep learning weakly supervised classification algorithm of noise learning, and adopts a multi-date classification result synthesis method to realize large-scale medium-resolution forest cover information extraction, solving the current large-scale forest classification task's dependence on a large number of high-precision samples, the influence of cloud detection results and the demand for large-scale forest classification automation programs. The present invention involves six parts: satellite remote sensing image preprocessing, public product prior knowledge construction, sample location determination and update, pseudo-label dynamic correction, deep learning classification model construction and multi-date classification result synthesis.

[0136] The following is a further detailed description of the embodiment of the present application in conjunction with the accompanying drawings. The embodiment of the present application includes the following specific steps:

[0137] 1. Satellite remote sensing image preprocessing

[0138] Orthorectification and cloud detection are performed on satellite remote sensing images to obtain satellite remote sensing images with higher positioning accuracy and their cloud detection results.

[0139] 2. Prior knowledge construction based on shared land cover classification products

[0140] The shared land cover classification products have different classification schemes and resolutions. To ensure comparability between datasets, each product was first reclassified according to its classification system and resampled to a 16-meter resolution ( Figure 4(2)). The reclassification refers to the classification system of GlobeLand30, and the surface cover is classified into crops, forests, grasslands, shrubs, wetlands, waters, tundra, artificial surfaces, bare land, and ice and snow. Then a voting strategy is used to create a voting map for each category. Finally, a more accurate surface cover map, forest / non-forest pseudo-label, and forest distribution spatial consistency map are created through threshold segmentation. Specifically, according to the voting results of the forest, the area with a vote greater than 0 is divided into three levels using formula (1), thereby obtaining a forest distribution spatial consistency map (FSC).

[0141]

[0142] Among them, v f is the forest voting map. The forest distribution spatial consistency map is optimized by using morphological operations such as dilation and erosion. Then, the high consistency area (AHC) of each category is created according to formula (2). c )picture.

[0143]

[0144] Among them, v c represents the voting map for each class c. t represents the threshold of the voting map, which is 4 for forest class and 3 for non-forest class.

[0145] Fusion of high consistency area maps of different categories generates a more accurate surface coverage map (i.e., the first classification image in the embodiment of the present application, corresponding to Figure 4 (2) A higher precision LULC map). When the maximum number of votes for the forest class is 6 and the maximum number of votes for the non-forest class is 5, there is no overlapping area between the high consistency area maps of different categories. Among them, this application refers to 6 products: GlobeLand30, FROM-GLC10, GLC FCS0, WorldCover, Esri Land Cover and TreeCover, among which GlobeLand30, FROM-GLC10, GLC FCS0, WorldCover, Esri Land Cover provide coverage information of multiple categories, and TreeCover only provides forest information. That is to say, for the forest category, 6 products are referenced; and for the non-forest category, 5 products are referenced. Therefore, when all products consider the classification result of pixel P1 to be forest, the number of votes for the forest category is 6 (the highest number of votes, full marks). Similarly, when all products consider the classification result of pixel P2 to be non-forest, the number of votes for the non-forest category is 5 (the highest number of votes, full marks). In addition, the fused surface cover map also includes some areas with uncertain categories. Finally, forest / non-forest pseudo labels with uncertainty areas were created by fusing the non-forest classes in the land cover map.

[0146] According to the more accurate land cover map mentioned above, sample images for model training are obtained from satellite remote sensing images, orthorectified and cloud detected, and the processed sample images are input into the classification model.

[0147] 3. Determination and update of sample space position

[0148] Taking advantage of the characteristics of sample selection in active learning, the information content and representativeness of the sample are used as the basis for determining the sample position. In addition, it is also expected to select scenes that are easily misclassified as difficult samples to optimize the network. Therefore, the determination of the sample position includes the initial determination and optimization of the sample position ( Figure 4 (3) and Figure 5 ). In the process of training and validation region selection, a non-overlapping sliding window is applied to split all images into patches. By setting some constraints, some patches that meet the requirements are filtered out as candidate patches for training and validation areas. Then, 5% of the candidate regions are randomly selected as training and validation areas, respectively.

[0149] Initial determination of sample locations. In the early stages of the experiment, representative sample images were selected based on the more accurate surface coverage map obtained in step (ii), such as Figure 5 (1) As shown. There are three rules for sample image selection: (1) Determine whether a patch contains Figure 5 (1) A list of objects (water, forest, crop, grass, and wetland). The order is determined based on the region and the likelihood of being misclassified as forest. (2) There are at least 2 classes in the patch, and the number of pixels of the target class is greater than a minimum value (here set to 20 pixels, about 0.5 hectares). (3) Patches used for training and testing are randomly selected from those patches that meet the requirements of (1) and (2), and the number of non-forest patches does not exceed the number of forest patches.

[0150] Optimization of sample positions. The prediction results of the model trained with the initially selected samples are checked against the pseudo labels (PL) to find potential difficult sample areas and Figure 4 The three potential classification difficult areas are marked on the higher precision land cover map (PM) in (2). Figure 5As shown in (1), the first one is the lost forest, which means that the PL thinks it is a forest but is actually a non-forest area. The second one is the error forest, which means that the PL thinks it is a non-forest area but is actually a forest area. The third one is the new forest, which means that the classification result classifies the uncertain area in the PL as a forest area. (The lost / error / new forest names are just for convenience). At the same time, the proportion of objects that are easily confused with forests in the optimized samples is still retained, that is, the category of the initial position of the sample is determined.

[0151] (IV) Dynamic Correction of Pseudo-labels

[0152] After marking the potential difficult sample areas on PM, the PL is dynamically corrected using the high probability areas predicted by the model, as shown in formula (3).

[0153]

[0154] Where UPL refers to the updated PL, p is the forest probability, θ1 and θ2 are the maximum and minimum values ​​of the non-forest probability, respectively. Regions with predicted forest probability less than θ1 are updated to non-forest regions. On the contrary, regions with predicted probability greater than θ2 are updated to forest regions. In the experiments, we compared different combinations of θ1 and θ2 and set them to 0.2 and 0.8, respectively.

[0155] (V) Construction of deep learning classification model

[0156] Medium-resolution multispectral images have spectral and spatial heterogeneity and are sensitive to atmosphere and light. These images can provide spectral information, but have serious mixed pixels. Except for some artificial forests, forests are usually in patches with irregular geometric boundaries. In addition, medium-resolution images are defective in depicting spatial details. Therefore, in medium-resolution images, the spatial texture of the forest is relatively blurred. According to the characteristics of medium-resolution images and forests, a spectral-spatial network (SSNet) is designed for forest classification, which can capture both spectral and spatial contextual information. Figure 6 As shown in Figure 1, SSNet consists of a spectral modeling module (SeMB) and a spatial modeling module (SaMB). The acquired spectral and spatial features are fused and used as the input of the classification module.

[0157] SeM: This module consists of a multi-layer perceptron to mine the spectral differences between forest and non-forest features. SeM consists of three linear layers with output dimensions of 64, 128, and 256, respectively, and is designed to explore the spectrum of forest / non-forest (FNF) without being affected by the spatial background of the scene. It can not only emphasize the spectral differences of FNF and enhance the boundaries of FNF in transition and edge regions, but also compensate for the blurring and smoothing effects of SaM.

[0158] SaM: This block consists of a shallow U-shaped structure to exploit spatial context and texture. Figure 6 As shown in , SaM uses the residual structure as the basic unit, and undergoes two downsampling and two upsampling operations. The residual structure adopts the classic BottleNeck structure, as shown in Figure 6 (b) In addition, in order to enhance the feature extraction capability of SaM, the standard spatial pyramid pooling (ASPP) is also used to expand the model receptive field (such as Figure 6 (c)). There are two main factors in the design of the shallow U-shaped structure. On the one hand, in medium-resolution images, the forest lacks clear texture. On the other hand, as downsampling proceeds, deeper networks will reduce the resolution, resulting in less prominent feature details being more blurred.

[0159] (VI) Synthesis of multi-date classification results

[0160] like Figure 7 As shown in Figure 1, forest cover maps were first generated on a date-by-date basis, and then the results of different dates were fused. A similar synthesis strategy was used in this process. The image valid area marker (VIM) and forest classification marker (FRM) were used to record the coverage information and classification information of each pixel. The former is used to mark the number of valid markings of each pixel during the data synthesis process, which is closely related to the availability of the pixel in the corresponding image of the voting result, that is, the pixel is a valid value area and not a cloud pixel. The latter is used to mark the number of times each pixel is marked as a forest. Through VIM and FRM, the probability p of each pixel being marked as a forest is obtained, as shown in formula (4).

[0161]

[0162] The fixed threshold segmentation method is used to determine the category of the pixel. As shown in formula (5), the probability of being marked as forest is greater than 0.5 and is regarded as a forest pixel, otherwise it is regarded as a non-forest pixel.

[0163]

[0164] Adaptive thresholding is used to segment forest / non-forest for a single date. When segmenting the probability map of a single date image, a threshold segmentation method combining global constraint and local adaptation (GCLA) based on prior knowledge of existing forest products is used. Based on the obtained forest spatial distribution consistency map, the implementation process of GCLA threshold segmentation is designed.

[0165] The specific calculation method of the overall threshold and the local threshold is as follows: according to the forest spatial distribution consistency map, obtain the high consistency area, medium consistency area and low consistency area of ​​the forest; obtain the probability p of each pixel being marked as a forest; respectively calculate the average value of the probability p of all pixels being marked as a forest in the high consistency area, medium consistency area and low consistency area; process the average values ​​obtained in the high consistency area, medium consistency area and low consistency area to obtain the overall threshold and the local threshold.

[0166] For the single scene classification probability map, we finally get VIM s ∈[0, 4] and FRM s ∈[0, 4]. According to formulas (4)-(5), the forest labeling probability p of the single date classification image in the specified area is obtained s ∈[0, 4] and the segmentation result f s ∈{0, 1}. It should be noted that when the pixel Pix i,j (i and j represent row and column numbers respectively) forest labeling probability P i,j >1 and part of P i,j = 1, it is an abnormal situation. In this case, all date images are identified as cloud pixels (i.e. ) and some cloud detection results are wrong. The reason why the image valid area appears When The reason for this is that the VIM is optimized by re-marking the 0 marks in the VIM as 1. The main purpose of doing this is to avoid the interference of incorrect cloud detection results on the classification results. Because the tolerance is very high during cloud detection, the cloud pixel range may be larger than the actual situation or the cloud detection result may be wrong, and the algorithm compensates for the interference of the cloud detection result error and achieves correct classification. In addition, this also maximizes the utilization rate of each image, especially in low latitudes, coastal areas and other areas that are more affected by clouds.

[0167] The voting scheme is used to fuse multi-date forest / non-forest data. We fuse the classification results of single images to obtain the forest distribution data of the target area. The classification results of each image in the target area are obtained by segmenting the single-date image. Using the forest segmentation results of the single-date image, we use a voting strategy to obtain the forest result mark FRM of the target grid area. grid∈[0, C] (C is the number of images). At the same time, we obtain the image valid area mark VIM based on the voting of the cloud mask results grid ∈[0,C]. VIM grid The regions marked as 0 in are re-marked as 1 to avoid abnormal situations. Finally, the synthetic forest result f is obtained according to formulas (4)-(5): grid ∈{0, 1}.

[0168] The present application embodiment is implemented using the Linux platform of NVIDIA TITAN V GPU, and its program is developed based on the PyTorch framework and Python programming language. According to the process of obtaining forest cover map ( Figure 4 ), a forest classification program written in Python is used, and the forest classification result can be obtained by running the program. The medium-resolution remote sensing image forest cover classification program written in the embodiment includes a remote sensing image preprocessing module, an auxiliary data acquisition and preprocessing module, a weakly supervised forest classification framework module and a forest classification result synthesis module. In the remote sensing image preprocessing module, the remote sensing image and the reference image base map are first read, the matching points are calculated, and the remote sensing image is orthorectified in combination with the digital elevation model; then, based on the corrected image, the sum of the red, green and blue bands is calculated, and the cloud detection result is obtained by the threshold segmentation method. In the auxiliary data acquisition and preprocessing module, the above method is used to obtain the surface cover map, forest / non-forest pseudo-label and forest distribution spatial consistency map. In the weakly supervised forest classification framework module, the initial sample positions are first screened using the land cover map, and the classification model is trained with samples with noise; when the model reaches a certain learning ability, the pseudo-labels are dynamically adjusted using the model's predicted probabilities, and the updated pseudo-labels are continued to be used to optimize the model; when the model training is completed, if the validation set meets the conditions, the model's predicted probabilities will be used to mark special areas, and these areas will be considered as difficult samples and given priority in sample position optimization. The optimized training set will participate in the training of the model. In the forest classification result synthesis module, the forest coverage results of the single-view image are segmented according to the calculated overall threshold and local threshold; then the forest coverage results of multiple dates are fused to obtain the final forest coverage map.

[0169] Figure 8 (a) is a forest coverage map obtained based on the embodiment of the present application; Figure 8 (b) is the local accuracy of the forest cover map; Figure 8 (ce) Figure 8 From the detailed display diagram of area (ce) in (a), it can be seen that the classification result of the embodiment of the present application has high accuracy and accurately describes forests of different sizes.

[0170] Fig. 9 Schematic diagram of the process of the method for classifying ground cover in the embodiment of the present application, such as Fig. 9 As shown, the classification methods of ground cover can specifically include:

[0171] Step 901: Acquire a surface image to be classified;

[0172] In some embodiments, obtaining a surface image to be classified includes: obtaining a surface image to be classified collected by an image acquisition device. Exemplarily, the image acquisition device may be a remote sensing instrument such as a panchromatic, multispectral, or hyperspectral camera, a synthetic aperture radar, and a stereo mapping camera on a satellite remote sensing platform. The surface image may be a satellite remote sensing image of one or more geographical areas.

[0173] In some embodiments, a satellite remote sensing image collected by a remote sensing instrument on a satellite remote sensing platform is obtained. Exemplarily, the surface image may be a medium-resolution satellite remote sensing image.

[0174] In some embodiments, the method may further include: performing orthorectification and cloud detection processing on the sample image, and inputting the processed sample image into the classification model.

[0175] Step 902: inputting the surface image to be classified into a classification model for classifying target covers to obtain a predicted probability of the target covers in the surface image to be classified;

[0176] Step 903: Determine the classification result corresponding to the target cover in the surface image to be classified based on the predicted probability of the target cover.

[0177] In some embodiments, the classification model is a deep learning-based classification model, and the classification model includes: a spectral modeling module, a spatial modeling module and a classification module; the classification model is configured to perform the following steps: the sample image is input into the spectral modeling module to extract the spectral difference information between the target covering and the non-target covering in the sample image; the sample image is input into the spatial modeling module to extract the spatial context features of the sample image; the spectral difference information and the spatial context features are fused as the input of the classification module to obtain a predicted probability image; the predicted probability image includes the predicted probability of each pixel for the target covering.

[0178] By adopting the above technical solution and using the trained classification model provided in the embodiment of the present application to classify the target covering objects in the surface image, the classification accuracy of the target covering objects can be improved.

[0179] The present application also provides a training device for a classification model. Fig.10 As shown, the training device 100 of the classification model includes:

[0180] A determination unit 1001 is used to determine a classification image corresponding to a surface image, the classification image comprising: a first classification image and a second classification image, the first classification image comprising classification results of multiple coverings, and the second classification image comprising classification results of a target covering;

[0181] The first processing unit 1002 is used to select samples of the surface image according to the first classification image to obtain a sample image; determine the classification label corresponding to the sample image according to the classification result of the target cover in the second classification image; and construct a data set based on the sample image and the classification label corresponding to the sample image;

[0182] The training unit 1003 is used to train the classification model based on the data set to obtain a classification model for classifying target coverings.

[0183] In some embodiments, the determination unit 1001 is specifically used to: determine multiple initial classification images corresponding to the surface image; reclassify and resample the multiple initial classification images to obtain multiple preprocessed classification images with consistent categories and resolutions; vote for each cover in the multiple preprocessed classification images to obtain a voting map for each cover; determine a high consistency area for each cover based on the voting map and voting threshold for each cover; fuse the high consistency areas for each cover to obtain a first classification image; fuse the high consistency areas of non-target covers in the first classification image to obtain a second classification image.

[0184] In some embodiments, the determination unit 1001 is also used to: determine the high consistency area of ​​the target covering according to the voting map of the target covering and the first voting threshold; determine the high consistency areas of other coverings according to the voting maps of other coverings and the second voting threshold; wherein the first voting threshold is greater than the second voting threshold.

[0185] In some embodiments, the classification image also includes: a third classification image, the third classification image includes a high consistency area, a medium consistency area and a low consistency area of ​​the target cover; the determination unit 1001 is also used to: when the number of votes for each pixel in the first area in the voting map of the target cover is greater than the third voting threshold, determine the first area as a high consistency area of ​​the target cover; when the number of votes for each pixel in the second area in the voting map of the target cover is less than or equal to the third voting threshold and greater than or equal to the fourth voting threshold, determine the second area as a medium consistency area of ​​the target cover; when the number of votes for each pixel in the third area in the voting map of the target cover is less than the fourth voting threshold, determine the third area as a low consistency area of ​​the target cover; wherein the third voting threshold is greater than or equal to the fourth voting threshold.

[0186] In some embodiments, the first processing unit 1002 is specifically used to: divide the first classification image into image blocks of a preset size to obtain classification image blocks; divide the surface image into image blocks of a preset size to obtain surface image blocks; and select the surface image blocks corresponding to the target classification image blocks as sample images according to preset sample selection rules, wherein the sample selection rules define the covering features of one or more classification image blocks.

[0187] In some embodiments, the training unit 1003 is specifically used to: randomly select some sample images and classification labels corresponding to some sample images from the data set to construct a training set and a verification set; use the training set to perform preliminary training on the classification model to obtain a preliminary trained classification model; use the verification set to verify the preliminary trained classification model to obtain an optimized classification model; continue to perform the step of training the optimized classification model using the data set until a verified classification model is obtained; in the process of verifying the preliminary trained classification model using the verification set, the method also includes: determining the classification result of the predicted probability image based on the predicted probability of each pixel of the predicted probability image output by the preliminary trained classification model for the target cover and a first threshold; when the predicted probability of the first pixel in the predicted probability image for the target cover is greater than the second threshold, if the classification result of the first pixel in the classification label is inconsistent with the classification result of the first pixel in the predicted probability image, dynamically correct the classification result of the first pixel in the classification label.

[0188] In some embodiments, the training unit 1003 is also used to: construct a test set based on the data set; input the multi-date sample images in the test set into the verified classification model respectively to classify the target covering to obtain a multi-date predicted probability image; merge the multi-date predicted probability images to obtain a predicted classification image; use the classification result of the predicted classification image for the target covering to mark special areas of the sample image to optimize the data set, and train the classification model based on the optimized data set to obtain a classification model for classifying the target covering; the special areas include: areas where the classification results of the predicted classification image for the target covering and the classification results of the second classification image for the target covering are inconsistent, and / or, areas predicted as target covering in the predicted classification image and predicted as uncertain areas by the second classification image.

[0189] In some embodiments, the determination unit 1001 is further used to: determine the multi-date cloud detection results of the multi-date sample image, and the third classification image;

[0190] The training unit 1003 is also used to determine coverage information based on multi-date cloud detection results, the coverage information including the number of times each pixel is marked as valid; determine classification information based on the multi-date predicted probability image and the third classification image, the classification information including the number of times each pixel is marked as a target cover in the multi-date predicted probability image; determine probability information based on the coverage information and the classification information, the probability information including the probability of each pixel being marked as a target cover; determine a predicted classification image based on the probability of each pixel being marked as a target cover and a probability threshold.

[0191] In some embodiments, the classification model is a deep learning-based classification model, and the classification model includes: a spectral modeling module, a spatial modeling module and a classification module; the classification model is configured to perform the following steps: through the spectral modeling module, based on the sample image, obtain the spectral difference information between the target covering and the non-target covering in the sample image; through the spatial modeling module, based on the sample image, obtain the spatial context features of the sample image; after fusing the spectral difference information and the spatial context features, use them as the input of the classification module to obtain a predicted probability image; the predicted probability image includes the predicted probability of each pixel for the target covering.

[0192] By using the above-mentioned device, effective sample images are selected from the surface images according to the first classification image, and the classification labels corresponding to the sample images are determined according to the second classification image, so as to construct a high-quality data set and conduct targeted training on the classification model, thereby improving the efficiency and effect of model training, and further improving the classification accuracy of the target cover.

[0193] The present application also provides a device for classifying ground cover, such as Fig.11 As shown, the ground cover classification device 110 includes:

[0194] An acquisition unit 1101 is used to acquire a surface image to be classified;

[0195] The second processing unit 1102 is used to input the surface image into the classification model for classifying target covers to obtain a classified image corresponding to the target covers in the surface image; the classification model for classifying target covers is obtained based on the training method of the classification model of any one of the embodiments of the present application.

[0196] By adopting the above-mentioned device and utilizing the trained classification model provided in the embodiment of the present application to classify the target covering objects in the surface image, the classification accuracy of the target covering objects can be improved.

[0197] The present application also provides an electronic device, such as Fig.12 As shown, the electronic device 120 includes: a processor 1201 and a memory 1202 configured to store a computer program that can be run on the processor;

[0198] The processor 1201 is configured to execute the method steps in the aforementioned embodiment when running a computer program.

[0199] Of course, in practical applications, Fig.12 As shown, the various components in the electronic device are coupled together through a bus system 1203. It can be understood that the bus system 1203 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1203 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are marked as bus system 1203 in the figure.

[0200] In practical applications, the processor may be at least one of an application-specific integrated circuit (ASIC), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the functions of the processor may also be other, and the embodiments of the present application are not specifically limited.

[0201] The above-mentioned memory can be a volatile memory (volatile memory), such as a random access memory (RAM) or a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD) or a combination of the above-mentioned types of memory, and provides instructions and data to the processor.

[0202] It should be understood that in the embodiments of the present application, related data such as user information is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0203] It should be understood that the terms used in the present application are only for the purpose of describing specific embodiments, but are not intended to limit the present application. The singular forms of "a", "said" and "the" used in the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items. The expressions "having", "may have", "include" and "include", or "may include" and "may include" in this application can be used to indicate the presence of corresponding features (e.g., elements such as numerical values, functions, operations or components), but the presence of additional features is not excluded.

[0204] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other and are not necessarily used to describe a specific order or sequence. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.

[0205] The technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0206] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. The above described embodiments are only exemplary, for example, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0207] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A training method for a classification model, It is characterized in that The method comprises: Determine a classification image corresponding to the surface image, the classification image comprising: a first classification image and a second classification image, the first classification image comprising classification results of multiple coverings, and the second classification image comprising classification results of target coverings; Selecting samples of the surface image according to the first classification image to obtain a sample image; Determining a classification label corresponding to the sample image according to a classification result of the target cover in the second classification image; A data set is constructed based on the sample image and the classification label corresponding to the sample image, and a classification model is trained based on the data set to obtain a classification model for classifying the target covering.

2. The method according to claim 1, It is characterized in that The step of determining the classified image corresponding to the surface image comprises: Determining a plurality of initial classification images corresponding to the surface image; Reclassifying and resampling the multiple initial classified images to obtain multiple preprocessed classified images with consistent categories and resolutions; Voting for each covering in the plurality of preprocessed classified images to obtain a voting map for each covering; Determine a high consistency region for each covering according to the voting map and voting threshold for each covering; fusing the high consistency regions of each covering to obtain the first classification image; The highly consistent regions of non-target covering objects in the first classified images are fused to obtain the second classified images.

3. The method according to claim 2, It is characterized in that Determining the high consistency area of ​​each covering according to the voting map and voting threshold of each covering includes: Determining a high consistency region of the target coverage according to the voting map of the target coverage and a first voting threshold; Determine a high consistency region of the other coverings according to the voting map of the other coverings and the second voting threshold; Wherein, the first voting threshold is greater than the second voting threshold.

4. The method according to claim 2, It is characterized in that The classification image further includes: a third classification image, the third classification image including a high consistency region, a medium consistency region, and a low consistency region of the target cover; The step of determining the classified image corresponding to the surface image further includes: When the number of votes for each pixel in the first area in the voting map of the target cover is greater than a third voting threshold, determining that the first area is a high consistency area of ​​the target cover; When the number of votes for each pixel in the second area of ​​the voting map of the target cover is less than or equal to the third voting threshold and greater than or equal to the fourth voting threshold, the second area is determined to be the medium consistency area of ​​the target cover; When the number of votes for each pixel in the third area of ​​the voting map of the target cover is less than the fourth voting threshold, the third area is determined to be a low consistency area of ​​the target cover; Wherein, the third voting threshold is greater than or equal to the fourth voting threshold.

5. The method according to claim 1, It is characterized in that The step of selecting samples of the surface image according to the first classification image to obtain a sample image includes: Dividing the first classified image into image blocks of a preset size to obtain classified image blocks; dividing the surface image into image blocks of a preset size to obtain surface image blocks; According to a preset sample selection rule, a surface image block corresponding to the target classification image block is selected as a sample image, wherein the sample selection rule defines one or more covering features of the classification image blocks.

6. The method according to claim 1, It is characterized in that The step of training a classification model based on the data set to obtain a classification model for classifying the target covering includes: Randomly select some sample images and classification labels corresponding to the some sample images from the data set to construct a training set and a validation set; Performing preliminary training on the classification model using the training set to obtain a preliminary trained classification model; Using the validation set to validate the initially trained classification model to obtain an optimized classification model; Continue to perform the step of training the optimized classification model using the data set until a verified classification model is obtained; In the process of verifying the preliminarily trained classification model using the verification set, the method further includes: Determine a classification result of the predicted probability image according to the predicted probability of each pixel of the predicted probability image output by the preliminarily trained classification model for the target covered object and a first threshold; When the predicted probability of the first pixel of the predicted probability image for the target coverage is greater than a second threshold, if the classification result of the first pixel in the classification label is inconsistent with the classification result of the first pixel in the predicted probability image, the classification result of the first pixel in the classification label is dynamically corrected.

7. The method according to claim 6, It is characterized in that The training of the classification model based on the data set to obtain the classification model for classifying the target covering also includes: Construct a test set based on the data set; Inputting the multi-date sample images in the test set into the verified classification model to classify the target cover, and obtaining a multi-date prediction probability image; Merging the multi-date prediction probability images to obtain a prediction classification image; Using the classification result of the predicted classification image for the target covering, marking the special area of ​​the sample image to optimize the data set, and training the classification model based on the optimized data set to obtain the classification model for classifying the target covering; The special area includes: an area where the classification result of the predicted classification image for the target covering is inconsistent with the classification result of the second classification image for the target covering, and / or an area predicted as the target covering in the predicted classification image and predicted as an uncertain area by the second classification image.

8. The method according to claim 4, It is characterized in that The method further comprises: Determine the multi-date cloud detection results of the multi-date sample images, and the third classified image; The step of merging the multi-date prediction probability images to obtain a prediction classification image includes: Determining coverage information according to the multi-date cloud detection results, the coverage information including the number of times each pixel is marked as valid; Determining classification information based on the multi-date prediction probability image and the third classification image, the classification information including the number of times each pixel is marked as the target cover in the multi-date prediction probability image; Determine probability information according to the coverage information and the classification information, wherein the probability information includes a probability that each pixel is marked as the target coverage object; The predicted classification image is determined based on the probability of each pixel being marked as the target coverage and a probability threshold.

9. The method according to any one of claims 1 to 8, It is characterized in that The classification model is a classification model based on deep learning, and the classification model includes: a spectral modeling module, a spatial modeling module and a classification module; The classification model is configured to perform the following steps: Obtaining, by the spectral modeling module, spectral difference information between the target covering and the non-target covering in the sample image based on the sample image; Obtaining, by the spatial modeling module, spatial context features of the sample image based on the sample image; The spectral difference information and the spatial context features are fused and used as the input of the classification module to obtain a predicted probability image; the predicted probability image includes the predicted probability of each pixel for the target cover.

10. A method for classifying ground cover. It is characterized in that The method comprises: Acquire a surface image to be classified; Inputting the surface image to be classified into a classification model for classifying target covering objects, and obtaining a classification result corresponding to the target covering object in the surface image to be classified; The classification model for classifying target coverings is obtained based on the training method of the classification model according to any one of claims 1 to 9.

11. A training device for a classification model, It is characterized in that The device comprises: A determination unit, configured to determine a classification image corresponding to the surface image, the classification image comprising: a first classification image and a second classification image, the first classification image comprising classification results of a plurality of coverings, the second classification image comprising a classification result of a target covering; A first processing unit is configured to select samples of the surface image according to the first classification image to obtain a sample image; determine a classification label corresponding to the sample image according to a classification result of the target cover in the second classification image; and construct a data set based on the sample image and the classification label corresponding to the sample image; A training unit is used to train a classification model based on the data set to obtain a classification model for classifying the target covering.

12. A device for classifying ground cover, It is characterized in that The device comprises: An acquisition unit, used for acquiring a surface image to be classified; A second processing unit is used to input the surface image into a classification model for classifying target covers to obtain a classification image corresponding to the target cover in the surface image; The classification model for classifying target coverings is obtained based on the training method of the classification model according to any one of claims 1 to 11.

13. An electronic device, It is characterized in that The electronic device comprises: a processor and a memory configured to store a computer program capable of running on the processor, Wherein, the processor is configured to implement the steps of the method described in any one of claims 1 to 9, or implement the steps of the method described in claim 10 when running the computer program.