Land cover classification model training method, land cover classification method and device

Abstract

The application discloses a kind of ground cover classification model training method, ground cover classification method and device, the ground cover classification model training method includes: obtaining training set;Sample image is input into ground cover classification initial model and is extracted to obtain feature image;First feature is obtained by carrying out first type feature processing to feature image, and first classification result is predicted using first feature;Second feature is obtained by carrying out second type feature processing to feature image, and third feature is obtained by extracting the deep feature of first feature, second feature and third feature are fused, and second classification result is predicted using the feature after fusion;Total loss of first classification result and second classification result is calculated using annotation information and is carried out back propagation, to adjust the parameters of ground cover classification initial model.By the above mode, the multilevel feature of high-resolution remote sensing image can be fully utilized, and the ground cover classification accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a land cover classification model training method, land cover classification method and apparatus. Background Technology

[0002] The formation and development of land cover types are influenced not only by geographical factors but also by human activities. Accurate land cover classification can, on the one hand, allow for the statistical analysis of the distribution of natural resources within a region, and on the other hand, it can also provide insights into the changes in natural resources over time, thereby reflecting the impact of human activities on the distribution of natural resources and aiding in decision-making.

[0003] Traditional land cover type surveys often rely on field investigations, requiring significant human resources. In recent years, with the development of remote sensing technology, the importance of land cover type surveys based on remote sensing imagery has become increasingly prominent, and land cover classification using high-resolution remote sensing imagery is a crucial component. Currently, most commonly used high-resolution remote sensing imagery land cover classification methods require artificial design of features for specific land cover types, failing to fully utilize the multi-band characteristics of high-resolution remote sensing imagery and thus possessing certain limitations. Summary of the Invention

[0004] The main technical problem solved by this invention is to provide a land cover classification model training method, land cover classification method and device, which can make full use of the multi-level features of high-resolution remote sensing images to improve the accuracy of land cover classification.

[0005] To address the aforementioned technical problems, one technical solution adopted by this invention is: providing a land cover classification model training method, which includes: acquiring a training set, the training set including sample images and corresponding annotation information; inputting the sample images into an initial land cover classification model for feature extraction to obtain feature images; performing a first type of feature processing on the feature images to obtain a first feature, and using the first feature to predict a first classification result; performing a second type of feature processing on the feature images to obtain a second feature, and extracting deep features from the first feature to obtain a third feature; fusing the second and third features, and using the fused feature to predict a second classification result; calculating the total loss of the first and second classification results using the annotation information and performing backpropagation to adjust the parameters of the initial land cover classification model.

[0006] The first type of feature processing is the first convolution processing, the second type of feature processing is the second convolution processing, and the extraction of deep features of the first feature to obtain the third feature includes: processing the first feature through a pixel attention mechanism to obtain the third feature.

[0007] The process of fusing the second and third features and using the fused features to predict the second classification result includes: multiplying the second and third features by a dot product and using the product result to predict the second classification result.

[0008] The annotation information includes sub-category annotations and major category annotations. The calculation of the total loss of the first classification result and the second classification result using the annotation information includes: calculating the first loss by comparing the sub-category annotations and the first classification result; calculating the second loss by comparing the major category annotations and the second classification result; calculating the weighted loss between the first classification result and the second classification result; and weightedly fusing the first loss, the second loss, and the weighted loss to obtain the total loss.

[0009] The calculation of the weighted loss between the first classification result and the second classification result includes: comparing the sub-category label and the corresponding second classification result to calculate the third loss; and weighting and fusing the third loss and the second loss to obtain the weighted loss.

[0010] The weighted fusion of the first loss, the second loss, and the weighted loss to obtain the total loss includes: setting weight values ​​for the first loss, the second loss, and the weighted loss, and performing regularization on each weight value; the total loss is the sum of the weighted value of the first loss, the second loss, and the weighted loss and the regularization result of the weight values.

[0011] The total loss is: Where loss is the total loss, loss1 is the first loss, loss2 is the second loss, loss3 is the weighted loss, and w1, w2, and w3 are the weight values.

[0012] The process involves inputting sample images into an initial land cover classification model for feature extraction. The resulting feature images include: extracting RGB three-channel images from the sample images to obtain RGB images; extracting features from the RGB images to obtain RGB feature images at multiple scales; extracting features from the sample images and fusing them with RGB feature images of the same scale to obtain a downsampled feature image at that scale; downsampling the fused downsampled feature image and fusing it with the next scale RGB feature image to obtain a small-scale downsampled feature image; upsampling the small-scale downsampled feature image and fusing it with the downsampled feature image of the same scale to obtain an upsampled feature image at that scale; upsampling the fused upsampled feature image and fusing it with the previous scale downsampled feature image to obtain the final feature image.

[0013] The process of extracting features from RGB images to obtain RGB feature images at multiple scales includes: extracting features from RGB images to obtain RGB feature images at four scales; and extracting features from sample images includes: passing the sample images sequentially through a multi-head attention mechanism and a multilayer perceptron for feature extraction.

[0014] The method further includes: obtaining a validation set, which includes validation images and corresponding validation labeled images; inputting the validation images into the initial land cover classification model; using the validation labeled images to obtain the accuracy of the initial land cover classification model; and selecting the initial land cover classification model with the highest accuracy as the land cover classification model.

[0015] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is to provide a land cover classification method, the method comprising: acquiring an image to be classified; inputting the image to be classified into a land cover classification model to obtain a land cover classification result, wherein the land cover classification model is trained by using the above-mentioned land cover classification model training method to induce land cover classification.

[0016] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is to provide a data device, which includes a processor for executing the above-mentioned land cover classification model training method or land cover classification method.

[0017] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is to provide a computer-readable storage medium for storing instruction / program data, which can be executed to implement the above-mentioned land cover classification model training method or land cover classification method.

[0018] The beneficial effects of this invention are as follows: Unlike existing technologies, the land cover classification model of this invention uses multi-level decoding and prediction. By mining the correspondence between different levels, it first uses low-level features to predict classification, then processes the low-level features to obtain high-level features, and then uses the high-level features to predict classification. Compared with the method of directly using the output of the previous level as the input of the next level, it can make reasonable use of the multi-band and multi-level information of high-resolution remote sensing images, making it easier to mine the hierarchical relationship between land covers and improve the accuracy of land cover classification. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating one implementation method of the land cover classification model training method of this application;

[0020] Figure 2 This is a flowchart illustrating another implementation of the land cover classification model training method of this application;

[0021] Figure 3 This is a schematic diagram of the initial model decoding and prediction module for land cover classification in this application;

[0022] Figure 4This is a schematic diagram of the weighted loss between the first and second classification results of this application;

[0023] Figure 5 This is a flowchart illustrating another implementation of the land cover classification model training method of this application;

[0024] Figure 6 This is a schematic diagram of an image of a copy of this application;

[0025] Figure 7 A schematic diagram of the small category labeled image and the large category labeled image of a sample image in this application;

[0026] Figure 8 This is a schematic diagram of an RGB image of a copy of this application;

[0027] Figure 9 This is a schematic diagram of the structure of the initial model for land cover classification in this application;

[0028] Figure 10 This is a flowchart illustrating one implementation method of the land cover classification method of this application;

[0029] Figure 11 This is a schematic diagram of the structure of the land cover classification model training device in the embodiments of this application;

[0030] Figure 12 This is a schematic diagram of the structure of the data device in the embodiments of this application;

[0031] Figure 13 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and effects of the present invention clearer and more explicit, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0033] Compared to conventional imagery, high-resolution remote sensing imagery adds a near-infrared band and exhibits clear hierarchical relationships between land cover categories, such as forest and grassland cover being categorized into arbor forests, shrub forests, and mixed arbor-shrub forests. Therefore, this application proposes a land cover classification model with multi-level decoding and prediction, making reasonable use of the multi-band and multi-level information of high-resolution remote sensing imagery. Simultaneously, this invention proposes a land cover classification model training method. After feature extraction from sample images, it mines the correspondence between different levels. First, it uses low-level features for classification prediction, then processes these low-level features to obtain high-level features, and finally uses these high-level features for classification prediction. Compared to directly using the output of the previous level as the input of the next level, this method can reasonably utilize the multi-band and multi-level information of high-resolution remote sensing imagery, more easily mine the hierarchical relationships between land covers, and improve the accuracy of land cover classification.

[0034] Please see Figure 1 , Figure 1 This is a flowchart illustrating one implementation method of the land cover classification model training method of this application. It should be noted that if substantially the same result is obtained, this embodiment is not necessarily identical. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, this embodiment includes:

[0035] S110: Obtain the training set.

[0036] Land cover images are acquired using high-resolution remote sensing imagery and used as sample images. Land cover features in the sample images are classified and labeled to obtain labeled images as corresponding annotation information. The sample images and their corresponding annotation information are then used as the training set.

[0037] S130: Input the sample image into the initial land cover classification model for feature extraction to obtain the feature image.

[0038] Feature extraction is performed on the sample images, using networks including but not limited to ResNet, Inception, and Transformer. Rich semantic features are obtained through deep networks to produce feature images.

[0039] S150: Perform first-class feature processing on the feature image to obtain the first feature, and use the first feature to predict the first classification result.

[0040] The feature image is processed in two dimensions to obtain prediction results in different dimensions. First, the feature image is processed in the first category to obtain the first classification result in a low dimension.

[0041] S170: Perform second-type feature processing on the feature image to obtain the second feature, extract the deep features of the first feature to obtain the third feature, fuse the second feature and the third feature, and use the fused feature to predict the second classification result.

[0042] Deep feature extraction is performed on the first feature obtained in step S150 to obtain the third feature. At the same time, the feature image is processed by the second type of feature to obtain the second feature. The second feature and the third feature are fused to predict and obtain a high-dimensional second classification result.

[0043] S190: Calculate the total loss of the first and second classification results using the annotation information and perform backpropagation to adjust the parameters of the initial land cover classification model.

[0044] The first and second classification results are compared with the corresponding annotation information, and the loss is calculated separately. The total loss of the two classification results is used for backpropagation to adjust the parameters of the initial land cover classification model.

[0045] In this implementation, after extracting features from the sample images, the correspondence between different levels is explored. First, low-level features are used to predict classification. Then, the low-level features are processed to obtain high-level features, and then the high-level features are used to predict classification. Compared with the method of directly using the output of the previous level as the input of the next level, this method can make reasonable use of the multi-band and multi-level information of high-resolution remote sensing images, making it easier to explore the hierarchical relationship between land cover and improve the accuracy of land cover classification.

[0046] In one embodiment, a land cover classification model with a multi-level decoding and prediction module is proposed. (See also...) Figure 2 , Figure 2 This is a flowchart illustrating another implementation of the land cover classification model training method of this application. It should be noted that if substantially the same result is obtained, this embodiment is not necessarily identical. Figure 2 The illustrated process sequence is limited. For example... Figure 2 As shown, this embodiment includes:

[0047] S210: Obtain the training set.

[0048] High-resolution remote sensing imagery of land cover is acquired and preprocessed. The preprocessed land cover images are then used as sample images. Land cover features in the sample images are classified and labeled, resulting in labeled images as corresponding annotation information. The sample images and their corresponding annotation information are then used as the training set.

[0049] S230: Input the sample image into the initial land cover classification model for feature extraction to obtain the feature image.

[0050] S250: Perform a first convolution on the feature image to obtain the first feature, and use the first feature to predict the first classification result.

[0051] Please see Figure 3 , Figure 3 This is a schematic diagram of the decoding and prediction module of the initial model for land cover classification in this application. The decoding and prediction module performs two-dimensional feature processing on the feature image to obtain prediction results in different dimensions. First, the feature image undergoes a first convolution process to predict and obtain a low-dimensional first classification result.

[0052] S270: Perform a second convolution on the feature image to obtain the second feature, process the first feature through a pixel attention mechanism to obtain the third feature, multiply the second feature and the third feature by a dot product, and use the dot product result to predict the second classification result.

[0053] Deep feature extraction is performed on the first feature obtained in step S250, and the third feature is obtained after processing by the pixel attention mechanism. At the same time, the feature image is processed by the second convolution to obtain the second feature. The second feature and the third feature are multiplied and fused to predict and obtain a high-dimensional second classification result.

[0054] S290: Calculate the total loss of the first and second classification results using the annotation information and perform backpropagation to adjust the parameters of the initial land cover classification model.

[0055] The annotation information includes sub-category annotations and major category annotations. The first loss is calculated by comparing the major category annotations with the first classification result; the second loss is calculated by comparing the sub-category annotations with the second classification result; and the weighted loss between the first and second classification results is calculated. The first loss, second loss, and weighted loss are then combined to obtain the total loss.

[0056] Specifically, the cross-entropy loss (loss1) for the smaller category result is calculated, the cross-entropy loss (loss2) for the larger category result is calculated, and the weighted loss (loss3) between the larger and smaller category results is calculated. Please refer to [link / reference]. Figure 4 , Figure 4This is a schematic diagram illustrating the weighted loss between the first and second classification results of this application. A first classification result has one or more second classification results. Therefore, by comparing the first and second classification results, the loss of the second classification result is transformed into a third loss for the corresponding first classification result. This third loss is then used as a weight to weight and fuse with the first loss of the corresponding first classification result, thereby constraining the hierarchical correspondence and obtaining the weighted loss. Here, the first classification result is the class classification result, and the second classification result is the subclass result. In one specific implementation, if a major category A contains 0 to n subclasses, when the subclass result is 0 and the major category result is A, the loss of subclass 0 is transformed into the loss of subclass 0 corresponding to major category A and fused with the loss of major category A.

[0057] When weighted and fused the first loss, second loss, and weighted loss, weight values ​​are assigned to the first loss, second loss, and weighted loss, and each weight value is regularized separately. The total loss is the sum of the weighted values ​​of the first loss, second loss, and weighted loss, and the regularized results of the weight values. In the embodiments of this application, unlike the method of manually designing the proportions of the three output losses, three learnable parameters w1, w2, and w3 are used to automatically adjust the proportions among the three losses. To avoid the three learnable parameters being too small, which would cause the backpropagation of the loss function to fail, regularization constraints are applied to the three learnable parameters. The overall loss is as follows:

[0058]

[0059] Backpropagation is performed using the loss function described above to adjust the parameters of the initial land cover classification model.

[0060] In this implementation, after feature extraction from the sample images, the correspondence between different levels is mined. Low-level features are used to predict the major land cover category, and then the low-level features are processed through a pixel attention module to obtain high-level features. These high-level features are then used to predict the minor land cover category. Compared to methods that directly use the output of the previous level as the input of the next level, this approach can effectively utilize the multi-band and multi-level information of high-resolution remote sensing images, making it easier to mine the hierarchical relationships between land covers. Furthermore, this application uses an automatically adjusting loss function to propose a weighted loss, thereby constraining the hierarchical correspondence. Learnable parameters are used to automatically learn the proportions of the three losses to improve the accuracy of land cover classification.

[0061] In one embodiment, a land cover classification model with a dual-input feature aggregation module and a multi-level decoding prediction module is proposed. See also... Figure 5 , Figure 5This is a flowchart illustrating another implementation of the land cover classification model training method of this application. It should be noted that if substantially the same result is obtained, this embodiment is not necessarily identical. Figure 5 The illustrated process sequence is limited. For example... Figure 5 As shown, this embodiment includes:

[0062] S510: Obtain the training set.

[0063] This paper acquires high-resolution remote sensing images of land cover and preprocesses them. Specifically, preprocessing operations include radiometric calibration, atmospheric correction, geometric correction, orthorectification, and linear stretching. Because high-resolution remote sensing images are easily affected by the atmosphere, illumination, and the sensor itself during acquisition, errors exist between the measured values ​​and the image's spectral reflectance. Therefore, radiometric calibration and a series of corrections are necessary to eliminate these distortions. Since the pixel values ​​of high-resolution remote sensing images are mostly greater than 255, the pixel values ​​need to be stretched to 0-255 to load pre-trained weights, resulting in remote sensing images used for training. The data stretching methods used in this application include, but are not limited to, maximum and minimum value normalization and 2% linear stretching.

[0064] Please refer to the section on using preprocessed high-resolution remote sensing images as sample images. Figure 6 , Figure 6 This is a schematic diagram of a sample image from this application. The sample image is then divided and labeled. First, based on national geographic monitoring, the Third National Land Survey, and other land cover classification standards, and considering practical application needs, a hierarchical interpretation standard is created and visually interpreted. During visual interpretation, only the sub-category levels of land cover classification need to be interpreted, and sub-category labels are created. Then, according to the correspondence between major and minor categories in the hierarchical interpretation standard, the minor category labels are merged to obtain the major category labels, where the minor and major category labels are the corresponding labeled images. For details, please refer to [link to relevant documentation]. Figure 7 , Figure 7 This is a schematic diagram illustrating the small-category labeled image and the large-category labeled image of a sample image for this application. Wherein, Figure 7 The left image is a small category labeled image, and the right image is a large category labeled image. As can be seen from the images, the small category labeled image has more labeled categories and richer details than the large category labeled image.

[0065] High-resolution remote sensing images have high resolution, and current computing devices struggle to handle the input of the entire image, necessitating cropping. Referring to the resolution commonly used in current mainstream semantic segmentation models, the cropped size is 512×512. During cropping, to minimize the loss of cropping edge information, overlapping sliding window cropping is employed, retaining half the size of the previous cropped image at each stage. Sliding window cropping is performed on sample images, small-class labeled images, and large-class labeled images separately. The cropped sample images and their corresponding small-class and large-class labeled images are then divided into training and validation sets according to a reasonable ratio. The sample images used in the validation set can also serve as validation images. The training set includes sample images and their corresponding annotation information, while the validation set includes validation images and their corresponding annotation information.

[0066] S520: Extracts the RGB three-channel image from the sample image to obtain the RGB image.

[0067] The acquired sample images are high-resolution four-channel remote sensing images, i.e., RGBN images, where the four channels are the RGB channels and the near-infrared channel. The RGB three-channel image is extracted from the sample image to obtain the RGB image. Please refer to [link to relevant documentation]. Figure 8 , Figure 8 This is a schematic diagram of an RGB image of a sample image of this application. The RGB image is compared to... Figure 6 The RGBN image lacks a near-infrared channel, while the RGB channels have the same detail.

[0068] S530: Extracts features from RGB images to obtain RGB feature images at multiple scales.

[0069] Please see Figure 9 , Figure 9 This is a schematic diagram of the initial model for land cover classification in this application. First, features are extracted from the RGB images. The feature extraction module includes, but is not limited to, network structures such as ResNet and Inception, and can load pre-trained weights from datasets such as ImageNet to reduce problems such as poor generalization caused by insufficient samples. Downsampling is then performed to obtain RGB feature images at four scales. In one embodiment, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 feature maps of the RGB images are obtained, respectively.

[0070] S540: Extract features from the sample image and fuse it with an RGB feature image of the same scale to obtain a downsampled feature image of that scale. Then, downsample the fused downsampled feature image and fuse it with an RGB feature image of the next scale to obtain a small-scale downsampled feature image.

[0071] The RGBN image of the sample image is input into the Transformer module for feature extraction. Specifically, the RGBN image is first divided into blocks for encoding, and then processed sequentially through a multi-head attention mechanism and a multilayer perceptron for feature extraction. The pixels of adjacent 4×4 regions are mapped into a higher-dimensional representation. In one embodiment, after feature extraction, a second feature extraction is performed through a multi-head attention mechanism and a multilayer perceptron, passing the information down sequentially to obtain a fully mixed feature map.

[0072] The feature image is downsampled and fused with an RGB feature image of the same scale to obtain a downsampled feature image at that scale. This process is repeated, downsampling is performed again, and the image is fused with an RGB feature image of the next scale, ultimately resulting in a small-scale downsampled feature image. For details, please refer to [link to documentation]. Figure 9 The RGBN image after feature extraction is downsampled to obtain a 1 / 4 feature map of RGBN. The 1 / 4 feature map of RGB is then fused with the 1 / 4 feature map of RGBN to obtain a 1 / 4 downsampled feature map. The 1 / 4 downsampled feature map is then further downsampled and fused with the RGB feature map of the same scale to finally obtain a 1 / 32 downsampled feature map.

[0073] S550: Upsample the small-scale downsampled feature image and fuse it with the downsampled feature image of the same scale to obtain the upsampled feature image of that scale. Upsample the fused upsampled feature image and fuse it with the downsampled feature image of the previous scale to obtain the feature image.

[0074] Furthermore, the small-scale downsampled feature images are then passed up again to restore pixel resolution. Specifically, the 1 / 32 downsampled feature image is upsampled and fused with the 1 / 16 downsampled feature image to obtain the 1 / 16 upsampled feature image. The 1 / 16 upsampled feature image is then upsampled again and fused with the downsampled feature image of the same scale to finally obtain the 1 / 4 feature image.

[0075] S560: Perform a first convolution on the feature image to obtain the first feature, and use the first feature to predict the first classification result.

[0076] Please continue reading. Figure 9 The first convolution process is performed on the 1 / 4 feature image obtained above to obtain the first feature. The first feature is then used to predict the major category of land cover to obtain the first classification result.

[0077] S570: Perform a second convolution on the feature image to obtain the second feature, process the first feature through a pixel attention mechanism to obtain the third feature, multiply the second feature and the third feature by a dot product, and use the dot product result to predict the second classification result.

[0078] Furthermore, the feature image is subjected to a second convolution process to obtain the second feature. Then, the first feature is processed through a pixel attention mechanism to extract deeper information to obtain the third feature. The second and third features are multiplied by a dot product, and the dot product is used to predict the subcategories of land cover to obtain the second classification result.

[0079] S580: Calculate the total loss of the first and second classification results using the annotation information and perform backpropagation to adjust the parameters of the initial land cover classification model.

[0080] S590: Verify the accuracy of the initial land cover classification model and select the initial land cover classification model with the highest accuracy as the land cover classification model.

[0081] The validation images are input into the initial land cover classification model. The accuracy of the initial land cover classification model is obtained using the validation labeled images. The initial land cover classification model with the highest accuracy is selected as the land cover classification model.

[0082] Specifically, a validation set is obtained, which includes validation images and corresponding annotation information. Due to insufficient computing resources, sliding window processing is required for the validation images. In one embodiment, 128 pixels are first filled around the validation image, and then sliding window prediction is performed. The sliding window size is 512×512, and the sliding window step size is 256. The sliding window image of the validation image is input into the initial land cover classification model to obtain the land cover classification prediction result. The prediction result only retains the 256×256 pixel area in the center of the sliding window image, thereby avoiding the impact of inaccurate prediction of boundary pixels on the model result. The number of sliding windows in the input image is calculated based on the preset sliding window size and image resolution. The number of images processed in batches is dynamically adjusted according to the computing resources of the computing device. The output result is compared with the corresponding annotation information, and the model accuracy is evaluated based on multiple indicators such as MIOU, MACC, and Kappa coefficient. The initial land cover classification model with the highest accuracy is selected as the land cover classification model.

[0083] In this embodiment, the accuracy evaluation method for land cover classification in high-resolution remote sensing images differs somewhat from the accuracy evaluation during the training process, mainly in terms of resolution and the diversity of evaluation indicators. When evaluating the accuracy of a land cover classification model, prediction of the entire map is required. Directly performing full-map convolution is problematic due to insufficient computing resources and the significant difference between the predicted and training resolutions, which can significantly impact accuracy. This application employs a dilation prediction method.

[0084] This implementation proposes a dual-input feature aggregation module. One input is deep semantic features obtained from RGB imagery through network structures such as ResNet and Inception. The other input is feature maps with richer band information obtained from RGBN imagery through a Transformer module. Then, the correspondence between different levels is mined. Low-level features are used to predict land cover categories, and then the low-level features are processed through a pixel attention module to obtain high-level features. Finally, the high-level features are used to predict land cover subcategories. Compared to methods that directly use the output of the previous level as the input of the next level, this approach can effectively utilize the multi-band and multi-level information of high-resolution remote sensing imagery, making it easier to mine the hierarchical relationships between land covers. Furthermore, this application uses an automatically adjusting loss function, proposing a weighted loss to constrain hierarchical correspondences. Learnable parameters are used to automatically learn the proportions of the three losses to improve the accuracy of land cover classification.

[0085] Please see Figure 10 , Figure 10 This is a flowchart illustrating one embodiment of the land cover classification method of this application. It should be noted that if substantially the same result is obtained, this embodiment is not necessarily identical. Figure 9 The illustrated process sequence is limited. For example... Figure 10 As shown, this embodiment includes:

[0086] S1010: Obtain the image to be classified.

[0087] S1030: Input the image to be classified into the land cover classification model to obtain the land cover classification result.

[0088] In this embodiment, the land cover classification model trained above is used to process the image to be classified. By learning the features of different levels, the multi-band and multi-level information of high-resolution remote sensing images can be reasonably utilized to more easily explore the hierarchical relationship between land covers and improve the classification and recognition accuracy.

[0089] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of a land cover classification model training device according to an embodiment of this application. In this embodiment, the land cover classification model training device includes an acquisition module 111, an extraction module 112, a first processing module 113, a second processing module 114, and an adjustment module 115.

[0090] The system comprises the following modules: an acquisition module 111 for acquiring a training set; an extraction module 112 for inputting sample images into the initial land cover classification model for feature extraction to obtain feature images; a first processing module 113 for performing first-type feature processing on the feature images to obtain first features, and using the first features to predict the first classification result; a second processing module 114 for performing second-type feature processing on the feature images to obtain second features, extracting deep features from the first features to obtain third features, fusing the second and third features, and using the fused features to predict the second classification result; and an adjustment module 115 for calculating the total loss between the first and second classification results using labeled information and performing backpropagation to adjust the parameters of the initial land cover classification model. This land cover classification model training device is used to explore the correspondence between different levels. It first uses low-level features to predict classification, then processes the low-level features to obtain high-level features, and then uses the high-level features to predict classification. Compared to directly using the output of the previous level as the input of the next level, this method can reasonably utilize the multi-band and multi-level information of high-resolution remote sensing images, making it easier to explore the hierarchical relationships between land covers and improve the accuracy of land cover classification.

[0091] Please see Figure 12 , Figure 12 This is a schematic diagram of the structure of a data device according to an embodiment of this application. In this embodiment, the data device 11 includes a processor 12.

[0092] Processor 12 can also be referred to as a CPU (Central Processing Unit). Processor 12 may be an integrated circuit chip with signal processing capabilities. Processor 12 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor, or processor 12 can be any conventional processor.

[0093] The data device 11 may further include a memory (not shown) for storing instructions and data required for the processor 12 to run.

[0094] The processor 12 is used to execute instructions to implement the methods provided by any embodiment of the land cover classification model training method or land cover classification method of the present application and any non-conflicting combination thereof.

[0095] Please see Figure 13 , Figure 13This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of this application. The computer-readable storage medium 21 in this embodiment stores instruction / program data 22. When executed, this instruction / program data 22 implements the method provided by the land cover classification model training method or any embodiment of the land cover classification method, and any non-conflicting combination thereof. The instruction / program data 22 can be formed into a program file and stored in the storage medium 21 as a software product, so that a computer device (which may be a personal computer, server, or network device, etc.) or processor can execute all or part of the steps of the methods in various embodiments of this application. The aforementioned storage medium 21 includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or terminal devices such as computers, servers, mobile phones, and tablets.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0097] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0098] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for training a land cover classification model, characterized in that, The method includes: Obtain a training set, which includes sample images and corresponding annotation information; The sample images are input into the initial land cover classification model for feature extraction to obtain feature images; The feature image is processed by a first type of feature processing to obtain a first feature, and the first feature is used to predict a first classification result; The feature image is processed by a second type of feature processing to obtain a second feature, and the deep features of the first feature are extracted to obtain a third feature. The second feature and the third feature are fused together, and the fused feature is used to predict the second classification result. The total loss of the first and second classification results is calculated using the labeled information and backpropagation is performed to adjust the parameters of the initial land cover classification model; The step of using the annotation information to calculate the total loss of the first classification result and the second classification result and performing backpropagation to adjust the parameters of the initial land cover classification model includes: comparing the first classification result and the second classification result with the corresponding annotation information respectively, calculating the loss respectively, and using the total loss of the two classification results to perform backpropagation to adjust the parameters of the initial land cover classification model.

2. The land cover classification model training method according to claim 1, characterized in that, The first type of feature processing is a first convolutional process, the second type of feature processing is a second convolutional process, and the extraction of deep features from the first feature to obtain the third feature includes: The first feature is processed by a pixel attention mechanism to obtain the third feature.

3. The land cover classification model training method according to claim 1, characterized in that, The process of fusing the second feature and the third feature, and using the fused feature to predict the second classification result, includes: The second feature and the third feature are multiplied by a dot product, and the result of the dot product is used to predict the second classification result.

4. The land cover classification model training method according to claim 1, characterized in that, The annotation information includes minor category annotations and major category annotations. The calculation of the total loss of the first classification result and the second classification result using the annotation information includes: Calculate a first loss by comparing the minor category label with the first classification result; calculate a second loss by comparing the major category label with the second classification result; calculate a weighted loss between the first classification result and the second classification result; The first loss, the second loss, and the weighted loss are weighted and combined to obtain the total loss.

5. The land cover classification model training method according to claim 4, characterized in that, The calculation of the weighted loss between the first classification result and the second classification result includes: The third loss is calculated by comparing the sub-category labels with the corresponding second classification results; The weighted loss is obtained by weighted fusion of the third loss and the second loss.

6. The land cover classification model training method according to claim 4, characterized in that, The weighted fusion of the first loss, the second loss, and the weighted loss to obtain the total loss includes: Weight values ​​are set for the first loss, the second loss, and the weighted loss, and regularization is performed on each of the weight values. The total loss is the sum of the first loss, the second loss, the weighted value of the weighted loss, and the regularized result of the weight value.

7. The land cover classification model training method according to claim 6, characterized in that, The total loss is: ； Wherein, loss is the total loss, loss1 is the first loss, loss2 is the second loss, loss3 is the weighted loss, and w1, w2, and w3 are weight values.

8. The land cover classification model training method according to claim 1, characterized in that, The step of inputting the sample image into the initial land cover classification model for feature extraction to obtain the feature image includes: Extract the RGB three-channel image from the sample image to obtain the RGB image; Feature extraction is performed on the RGB image to obtain RGB feature images at multiple scales; Feature extraction is performed on the sample image, and it is fused with the RGB feature image of the same scale to obtain a downsampled feature image of that scale. The fused downsampled feature image is then downsampled and fused with the RGB feature image of the next scale to finally obtain a small-scale downsampled feature image. The small-scale downsampled feature image is upsampled and fused with the downsampled feature image of the same scale to obtain the upsampled feature image of that scale. The fused upsampled feature image is then upsampled and fused with the downsampled feature image of the previous scale to obtain the feature image.

9. The land cover classification model training method according to claim 8, characterized in that, The step of extracting features from the RGB image to obtain RGB feature images at multiple scales includes: Feature extraction is performed on the RGB image to obtain RGB feature images at four scales; The feature extraction of the sample image includes: The sample images are sequentially processed through a multi-head attention mechanism and a multilayer perceptron for feature extraction.

10. The land cover classification model training method according to claim 1, characterized in that, The method further includes: Obtain a verification set, which includes verification images and corresponding verification labeled images; The verification image is input into the initial land cover classification model. The accuracy of the initial land cover classification model is obtained using the verification labeled image. The initial land cover classification model with the highest accuracy is selected as the land cover classification model.

11. A method for classifying land cover, characterized in that, The method includes: Obtain the image to be classified; The image to be classified is input into the land cover classification model to obtain the land cover classification result. The land cover classification model is trained by using the land cover classification model training method according to any one of claims 1-8 to enable the land cover classification model.

12. A data device, characterized in that, Includes a processor, the processor being configured to execute instructions to implement the land cover classification model training method as described in any one of claims 1-10 or the land cover classification method as described in claim 11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store instruction / program data that can be executed to implement the land cover classification model training method as described in any one of claims 1-10 or the land cover classification method as described in claim 11.

Images