Satellite image-based cross-domain object classification fine-tuning sample automatic selection method

Abstract

The application relates to a satellite image-based cross-domain ground object classification fine-tuning sample automatic selection method, which comprises the following steps: obtaining an initial interpretation model, to-be-interpreted images and medium-resolution ground surface cover classification results; obtaining an interpretation result and an interpretation category probability distribution; constructing a high-value sample selection framework; selecting candidate images from the to-be-interpreted images as a to-be-labeled sample set according to a sample screening strategy, artificially labeling the candidate samples to obtain a fine-tuning sample set; fine-tuning the initial interpretation model according to the fine-tuning sample set and model evaluation, and obtaining an optimal interpretation model; and obtaining a final fine-tuning sample set after a screening criterion is met. On the basis of reasonably utilizing an existing model, the application constructs diversified, representative and differentiated fine-tuning samples through ground surface cover prior knowledge, fully utilizes the prior knowledge, reduces the input of manpower and material resources, and can better complete a current satellite remote sensing image ground object classification task.

Description

Technical Field

[0001] This invention relates to the field of satellite remote sensing image processing technology, and in particular to an automated selection method for cross-domain ground feature classification fine-tuning samples based on satellite imagery. Background Technology

[0002] In applications of land cover, land classification, and land use, satellite remote sensing imagery plays a crucial role as an important source of information. When interpreting satellite remote sensing imagery, it is typically necessary to first build a predictive model using a sufficient training sample library, and then use this model to interpret the target satellite remote sensing imagery. However, when interpreting new satellite remote sensing imagery, the previously built model is often inapplicable because the previously acquired training samples may be outdated and no longer meet the requirement of the identically distributed assumption. In such cases, continuing to use these samples for predictive learning may affect the accuracy of the results.

[0003] Multiple factors, such as atmospheric absorption and scattering, sensor calibration, solar altitude angle, azimuth angle, phenological phases, and data processing, can affect the spectral data of ground objects acquired by satellite sensors. These data change over time, causing the spectral values ​​of the training samples to follow different probability statistical distributions than those of the current image data. Therefore, the trained model is not suitable for this situation. To meet the needs of current image interpretation tasks, it is necessary to re-label samples and fine-tune the existing model.

[0004] In intelligent interpretation of satellite remote sensing imagery, sample selection is a crucial step. How to select sample sets with minimal time, manpower, and material resources to achieve high-precision interpretation is a problem that needs to be solved in current intelligent remote sensing interpretation applications. Currently, there are two methods for sample selection: one is based on manual screening, and the other is through multi-model comparison for differential selection. However, existing sample selection methods have some shortcomings:

[0005] (1) Insufficient data diversity: When selecting samples, the various situations and changes in satellite remote sensing images may not be fully considered, resulting in the selected samples being too similar or not comprehensive enough, and failing to cover various details and features in the images.

[0006] (2) Lack of dynamic adjustment: Satellite remote sensing images change over time, but existing sample selection methods often do not take this time change into account, which may result in the selected samples being unsuitable for subsequent satellite remote sensing images.

[0007] (3) Insufficient sample representativeness: The sample selection method, which mainly relies on manual selection, often ignores the effect of existing models, resulting in samples that are not representative.

[0008] (4) Low computational efficiency: Due to the large amount of data in satellite remote sensing images and the high computational complexity, existing automated sample selection methods require comparison of multiple models, which significantly increases computational complexity. Summary of the Invention

[0009] The present invention aims to solve the technical problems in the prior art by providing an automated selection method for cross-domain ground feature classification fine-tuning samples based on satellite imagery.

[0010] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0011] An automated selection method for cross-domain land cover classification fine-tuning samples based on satellite imagery includes the following steps:

[0012] S1: Obtain the initial interpretation model, the image to be interpreted, and the medium-resolution land cover classification results;

[0013] S2: Input the image to be interpreted into the initial interpretation model to obtain the interpretation results and the probability distribution of each interpretation category;

[0014] S3: Construct a high-value sample selection framework, which includes sample value measurement and calculation, sample screening strategy and labeling, model fine-tuning and evaluation, and screening criteria;

[0015] S4: Select candidate images from the images to be interpreted as the sample set to be labeled according to the sample selection strategy, and manually label the candidate samples to obtain the fine-tuned sample set;

[0016] S5: Fine-tune and evaluate the initial interpretation model based on the fine-tuning sample set to obtain the optimal interpretation model;

[0017] S6: Based on the screening criteria, repeat steps S3-S5 until the screening criteria are met, and then obtain the final fine-tuned sample set.

[0018] In the above technical solution, step S3 specifically includes:

[0019] S31: Construct a sample value measurement system and calculate the sample value;

[0020] S32: Construct a sample selection strategy.

[0021] In the above technical solution, step S31 specifically includes:

[0022] A single slice image is denoted as Image, which is the smallest unit of measurement for a sample. Uncertainty is calculated by the information entropy of the model output probability. The land scene division is determined by the proportion of each land feature category in the medium-resolution land cover. The task foreground division is determined by the matching degree between the medium-resolution land cover scene and the interpretation target type. The calculation formulas are as follows:

[0023]

[0024]

[0025]

[0026]

[0027] In the formula, i represents the row number of the pixel in the image, and j represents the column number of the pixel in the image. This represents the mode category in the calculated land cover reclassification image. This represents the number of pixels in the image that satisfy the given conditions. This represents a list of interpretation task categories. H(Image) represents the frequency of positive samples in Image; S(Image) represents the model uncertainty; T(Image) represents the terrain scene partitioning; and T(Image) represents the task foreground partitioning. In the value range, 1 represents the task prospect, and 0 represents the task background.

[0028] In the above technical solution, step S32 specifically includes:

[0029] (1) Calculate H(Image), S(Image) and T(Image) for all images in Sample_Pool; H(Image) represents model uncertainty, S(Image) represents the land scene division, and T(Image) represents the task foreground division;

[0030] (2) Group Sample_Pool, first according to The values ​​are divided into two groups, among which A value of 1 is called the task foreground, denoted as Sample_Pool_foreground, and A value of 0 indicates a task background group, denoted as Sample_Pool_background; subsequently, based on... The surface cover scene category is further divided into Sample_Pool_foreground and Sample_Pool_background groups, denoted as Sample_Pool_foreground_Class and Sample_Pool_background_Class; finally, each group is further classified according to... Sort and group the data. First, sort the data in descending order. Then, divide the data into three groups based on the ranking. The top 5%-50% are designated as the high uncertainty group, denoted as Sample_Pool_foreground_Class_Hard or Sample_Pool_background_Class_Hard. The 50%-72.5% group is designated as the medium uncertainty group, denoted as Sample_Pool_foreground_Class_Middle or Sample_Pool_background_Class_Middle. The 72.5%-95% group is designated as the low uncertainty group, denoted as Sample_Pool_foreground_Class_Easy or Sample_Pool_background_Class_Easy.

[0031] (3) Determine the total number of samples and the number of labeled samples in each category pool. The total number of samples for model fine-tuning needs to be determined based on the difficulty of the interpretation task and the size of the region. The division ratio of each category pool needs to be determined based on the interpretation task. First, allocate the number of samples in Sample_Pool_foreground and Sample_Pool_background. Then, allocate the number of category pools in Sample_Pool_foreground and Sample_Pool_background equally. Finally, allocate the number of Sample_Pool_foreground_Class_Hard, Sample_Pool_foreground_Class_Middle, and Sample_Pool_foreground_Class_Easy.

[0032] (4) Finally, sampling is carried out according to each sample pool and the number of samples. The sampling principle is random sampling.

[0033] In the above technical solution, step S4 specifically includes:

[0034] S41: Sample extraction and sample labeling: Random sampling is performed in each sample pool according to the sample pool grouping and the number of samples to obtain the fine-tuned sample set to be labeled;

[0035] S42: Then, the samples are manually labeled to obtain the final fine-tuned sample set.

[0036] In the above technical solution, step S5 specifically includes:

[0037] S51: Divide the fine-tuning sample set into a training sample set and a validation sample set according to a certain ratio;

[0038] S52: Fine-tune the initial model using the training sample set, and use the initial model weights as pre-training weights;

[0039] S53: After fine-tuning, use the validation sample set to evaluate the accuracy of the model.

[0040] The present invention has the following beneficial effects:

[0041] The present invention provides an automated selection method for cross-domain land cover classification fine-tuning samples based on satellite imagery. By utilizing model uncertainty, land cover scene measurement, and interpretation task scene measurement, a fine-tuning sample selection framework is constructed. This method automatically selects more representative, diverse, and homogeneous samples, improving the quality of the fine-tuning sample set. It can significantly reduce the interference of human subjective factors and train a model that is more suitable for the current interpretation task.

[0042] The automatic selection method for cross-domain land cover classification fine-tuning samples based on satellite imagery of the present invention can construct diverse, representative and differentiated fine-tuning samples by making reasonable use of existing models and prior knowledge of land cover. It makes full use of prior knowledge, reduces the input of human and material resources, and can better complete the current land cover classification task based on satellite remote sensing imagery. Attached Figure Description

[0043] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0044] Figure 1 This is a schematic diagram of the automatic sample set mining process for fine-tuning. Detailed Implementation

[0045] The present invention will now be described in detail with reference to the accompanying drawings.

[0046] The present invention provides an automated selection method for cross-domain land cover classification fine-tuning samples based on satellite imagery, comprising the following steps:

[0047] First, an initial interpretation model, the image to be interpreted, and medium-resolution land cover classification results are obtained. Then, the image to be interpreted is input into the initial interpretation model to obtain the interpretation results and the probability distribution of each interpretation category. After that, a high-value sample selection framework is constructed, including high-value sample measurement calculation, sample screening and labeling, model fine-tuning and evaluation, and screening criteria. Through the high-value sample selection framework, iterative selection of the sample set and model evolution are realized to obtain a model fine-tuning sample set suitable for the interpretation task.

[0048] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0049] The present invention provides an automated selection method for cross-domain land cover classification fine-tuning samples based on satellite imagery, comprising the following steps:

[0050] S1: Obtain the initial interpretation model, the image to be interpreted, and the medium-resolution land cover classification results;

[0051] First, understand the current decoding task. The decoding task category list is recorded as follows: The process involves acquiring an initial interpretation model and the image to be interpreted. Specifically, this can be understood as a task of extracting land features based on semantic segmentation. Land features can include surface entities such as farmland, forest, water bodies, and buildings. The initial interpretation model refers to a deep learning semantic segmentation model trained on previously labeled data. Model examples can include conventional semantic segmentation models such as UNet, DeepLabV3, and HRNet. The image to be interpreted is divided and cropped according to a 512 or 1024 grid to obtain a potential sample pool (Sample_Pool). Medium-resolution land cover results can be selected from publicly available global 10-meter and 30-meter land cover data. In this embodiment, the 2020 30-meter global land cover data released by Liu Liangyun, researcher at the Aerospace Information Research Institute of the Chinese Academy of Sciences, is selected as the basic data. Considering the high-resolution land cover classification task, it is reclassified into 10 categories: background, farmland, forest, shrubs, sparse vegetation, grassland, wetlands, impermeable surfaces, bare land, water bodies, and permanent glaciers and snow cover. The reclassified land cover image is denoted as CLASS.

[0052] S2: Input the image to be interpreted into the initial interpretation model to obtain the interpretation results and the probability distribution of each interpretation category;

[0053] According to the interpretation task, all images in the potential sample pool Sample_Pool are input into the initial interpretation model to obtain the interpretation results and the probability distribution of each category. The probability distribution of each category is denoted as P(x), where P(x) represents the probability interpretation of a certain value of the random variable X.

[0054] S3: Construct a high-value sample selection framework, which includes sample value measurement and calculation, sample screening strategy and labeling, model fine-tuning and evaluation, and screening criteria;

[0055] S31: Construct a sample value measurement system and calculate the sample value. A single slice image is used as the smallest unit of measurement for the sample, denoted as Image. In this embodiment, it can refer to an image with a width and height of 512 or 1024 pixels, denoted as W. The sample value measurement system consists of model uncertainty H (Image), surface scene division S (Image), and task foreground division T (Image). Uncertainty is calculated by the information entropy of the model output probability. Surface scene division is determined by the proportion of each surface element category in the medium-resolution land cover. Task foreground division is determined by the matching degree between the medium-resolution land cover scene and the interpretation target type. The calculation formula is as follows:

[0056]

[0057]

[0058]

[0059]

[0060] In the formula, i represents the row number of the pixel in the image, and j represents the column number of the pixel in the image. This represents the mode category in the calculated land cover reclassification image. This represents the number of pixels in the image that satisfy the given conditions. This represents a list of interpretation task categories. This represents the frequency of positive samples in the Image, specifically the frequency of the number of land cover categories that match the interpretation task categories. In the text, 1 represents the task's prospect, and 0 represents the task's background. The larger the value, the greater the uncertainty of the model for the sample.

[0061] S32: Construct a sample selection strategy. The principles for sample selection are: samples with higher uncertainty should be selected; the terrain scene should be uniformly divided; and the ratio of foreground to background samples should be appropriate. The specific steps are as follows:

[0062] (1) Calculate and obtain H(Image), S(Image), and T(Image) from all images in Sample_Pool;

[0063] (2) Group Sample_Pool, first according to The values ​​are divided into two groups, among which A value of 1 is called the task foreground (denoted as Sample_Pool_foreground) and A value of 0 indicates a task background group (denoted as Sample_Pool_background). Then, based on... The surface cover scene category is further divided into Sample_Pool_foreground and Sample_Pool_background groups, denoted as Sample_Pool_foreground_Class and Sample_Pool_background_Class. Finally, each group is classified according to... The data is sorted and grouped. First, it is sorted in descending order. Then, it is divided into three groups according to the ranking order. The top 5%-50% are designated as the high uncertainty group (denoted as Sample_Pool_foreground_Class_Hard or Sample_Pool_background_Class_Hard), 50%-72.5% are designated as the medium uncertainty group (denoted as Sample_Pool_foreground_Class_Middle or Sample_Pool_background_Class_Middle), and 72.5%-95% are designated as the low uncertainty group (denoted as Sample_Pool_foreground_Class_Easy or Sample_Pool_background_Class_Easy).

[0064] To avoid the impact of extreme outliers, 0-5% and 95%-100% are excluded; endpoint descriptions: 5% belongs to (5%-50%), 50% belongs to (50%-72.5%), 72.5% belongs to (72.5%-95%), and 95% is excluded; specific outlier and endpoint settings can be set with reference to actual task experience.

[0065] (3) Determine the total number of samples and the number of labeled samples in each category pool. The total number of samples for model fine-tuning needs to be determined based on the difficulty of the interpretation task and the size of the region. In this embodiment, a total of 500 or 1000 images are recommended. The division ratio of each category pool needs to be determined based on the interpretation task. First, the number of samples in Sample_Pool_foreground and Sample_Pool_background is allocated. In this embodiment, a ratio of 9:1 is recommended for the number of samples in Sample_Pool_foreground and Sample_Pool_background. Then, the number of category pools in Sample_Pool_foreground and Sample_Pool_background is allocated equally, for example, the number of samples in each Sample_Pool_foreground_Class is equal. Finally, Sample_Pool_foreground_Class_Hard, Sample_Pool_foreground_Class_Middle, and Sample_Pool_foreground_Class_Easy are allocated with a sample allocation ratio of 2:1:1.

[0066] (4) Finally, sampling is carried out according to each sample pool and the number of samples. The sampling principle is random sampling.

[0067] S4: Select candidate images from the images to be interpreted as the sample set to be labeled according to the sample selection strategy, and manually label the candidate samples to obtain the fine-tuned sample set;

[0068] S41: Sample extraction and labeling. Random sampling is performed in each sample pool according to the sample pool grouping and the number of samples to obtain the fine-tuned sample set to be labeled.

[0069] S42: Then, the samples are manually labeled to obtain the final fine-tuned sample set.

[0070] S5: Fine-tune and evaluate the initial interpretation model based on the fine-tuning sample set to obtain the optimal interpretation model;

[0071] S51: Divide the fine-tuning sample set into training sample set and validation sample set according to a certain ratio. In this embodiment, the ratio is set to 8:2.

[0072] S52: Fine-tune the initial model using the training sample set. The fine-tuning method is to use the initial model weights as pre-training weights.

[0073] S53: After fine-tuning, the model's accuracy is evaluated using the validation sample set. In this embodiment, the F1 score is used for model evaluation. First, three concepts are defined: TP: positive samples predicted as positive by the model; FP: negative samples predicted as positive by the model; FN: positive samples predicted as negative by the model. The expression for calculating the F1 score is as follows:

[0074]

[0075]

[0076]

[0077] Here, precision represents the actual correctness of the model's predictions among positive samples, and recall represents the percentage of positive samples in the true set that were correctly predicted by the model. The F1 score is obtained from precision and recall, and the F1 score is generally between [0, 1].

[0078] S6: Based on the screening criteria, repeat steps S3-S5 until the screening criteria are met, and then obtain the final fine-tuned sample set.

[0079] The selection criteria can be set as the F1 score, an evaluation metric for model fine-tuning, or it can be based on visual judgment to determine whether the interpretation task has met the interpretation requirements. The specific F1 score needs to be set according to the specific task. Repeat steps S3-S5 iteratively based on the selection criteria until the iteration criteria are met, then stop and obtain the final fine-tuning sample set.

[0080] The present invention provides an automated selection method for cross-domain land cover classification fine-tuning samples based on satellite imagery. By utilizing model uncertainty, land cover scene measurement, and interpretation task scene measurement, a fine-tuning sample selection framework is constructed. This method automatically selects more representative, diverse, and homogeneous samples, improving the quality of the fine-tuning sample set. It can significantly reduce the interference of human subjective factors and train a model that is more suitable for the current interpretation task.

[0081] The automatic selection method for cross-domain land cover classification fine-tuning samples based on satellite imagery of the present invention can construct diverse, representative and differentiated fine-tuning samples by making reasonable use of existing models and prior knowledge of land cover. It makes full use of prior knowledge, reduces the input of human and material resources, and can better complete the current land cover classification task based on satellite remote sensing imagery.

[0082] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. An automated selection method for cross-domain land cover classification fine-tuning samples based on satellite imagery, characterized in that, Includes the following steps: S1: Obtain the initial interpretation model, the image to be interpreted, and the medium-resolution land cover classification results; S2: Input the image to be interpreted into the initial interpretation model to obtain the interpretation results and the probability distribution of each interpretation category; S3: Construct a high-value sample selection framework, which includes sample value measurement and calculation, sample screening strategy and labeling, model fine-tuning and evaluation, and screening criteria; S4: Select candidate images from the images to be interpreted as the sample set to be labeled according to the sample selection strategy, and manually label the candidate samples to obtain the fine-tuned sample set; S5: Fine-tune and evaluate the initial interpretation model based on the fine-tuning sample set to obtain the optimal interpretation model; S6: Based on the screening criteria, repeat steps S3-S5 until the screening criteria are met, and then obtain the final fine-tuned sample set. Step S3 specifically includes: S31: Construct a sample value measurement system and calculate the sample value; S32: Construct a sample selection strategy; Step S31 specifically includes: A single slice image is denoted as Image, which is the smallest unit of measurement for a sample. Uncertainty is calculated by the information entropy of the model output probability. The land scene division is determined by the proportion of each land feature category in the medium-resolution land cover. The task foreground division is determined by the matching degree between the medium-resolution land cover scene and the interpretation target type. The calculation formulas are as follows: In the formula, i represents the row number of the pixel in the image, and j represents the column number of the pixel in the image. This represents the mode category in the calculated land cover reclassification image. This represents the number of pixels in the image that satisfy the given conditions. This represents a list of interpretation task categories. H(Image) represents the frequency of positive samples in Image; S(Image) represents the model uncertainty; T(Image) represents the terrain scene partitioning; and T(Image) represents the task foreground partitioning. In the value range, 1 represents the task foreground and 0 represents the task background; Step S32 specifically includes: (1) Calculate H(Image), S(Image) and T(Image) for all images in Sample_Pool; H(Image) represents model uncertainty, S(Image) represents the land scene division, and T(Image) represents the task foreground division; (2) Group Sample_Pool, first according to The values ​​are divided into two groups, among which A value of 1 is called the task foreground, denoted as Sample_Pool_foreground, and A value of 0 indicates a task background group, denoted as Sample_Pool_background; subsequently, based on... The surface cover scene category is further divided into Sample_Pool_foreground and Sample_Pool_background, denoted as Sample_Pool_foreground_Class and Sample_Pool_background_Class; finally, each group is further classified according to... Sort and group the data. First, sort the data in descending order. Then, divide the data into three groups based on the ranking. The top 5%-50% are designated as the high uncertainty group, denoted as Sample_Pool_foreground_Class_Hard or Sample_Pool_background_Class_Hard. The 50%-72.5% group is designated as the medium uncertainty group, denoted as Sample_Pool_foreground_Class_Middle or Sample_Pool_background_Class_Middle. The 72.5%-95% group is designated as the low uncertainty group, denoted as Sample_Pool_foreground_Class_Easy or Sample_Pool_background_Class_Easy. (3) Determine the total number of samples and the number of labeled samples in each category pool. The total number of samples for model fine-tuning needs to be determined based on the difficulty of the interpretation task and the size of the region. The division ratio of each category pool needs to be determined based on the interpretation task. First, allocate the number of samples in Sample_Pool_foreground and Sample_Pool_background. Then, allocate the number of category pools in Sample_Pool_foreground and Sample_Pool_background equally. Finally, allocate the number of Sample_Pool_foreground_Class_Hard, Sample_Pool_foreground_Class_Middle, and Sample_Pool_foreground_Class_Easy. (4) Finally, sampling is carried out according to each sample pool and the number of samples. The sampling principle is random sampling.

2. The automated selection method for cross-domain land cover classification fine-tuning samples based on satellite imagery according to claim 1, characterized in that, Step S4 specifically includes: S41: Sample extraction and sample labeling: Random sampling is performed in each sample pool according to the sample pool grouping and the number of samples to obtain the fine-tuned sample set to be labeled; S42: Then, the samples are manually labeled to obtain the final fine-tuned sample set.

3. The automated selection method for cross-domain land cover classification fine-tuning samples based on satellite imagery according to claim 1, characterized in that, Step S5 specifically includes: S51: Divide the fine-tuning sample set into a training sample set and a validation sample set according to a certain ratio; S52: Fine-tune the initial model using the training sample set, and use the initial model weights as pre-training weights; S53: After fine-tuning, use the validation sample set to evaluate the accuracy of the model.

Images