Remote sensing image land cover classification method based on pseudo-label and class dictionary learning
By employing pseudo-labels and category dictionary learning methods, and utilizing deep features and SVM classifiers, the problems of insufficient training samples and in-class/out-of-class discrepancies in land cover classification of high-resolution remote sensing images are solved, achieving higher classification accuracy and reliability.
Patent Information
- Application Number
- CN202211727361.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-30
- Publication Date
- 2025-11-07
- Estimated Expiration
- 2042-12-30
AI Technical Summary
Existing technologies for land cover classification of high-resolution remote sensing images suffer from problems such as insufficient training sample quantity, large intra-class differences, and high inter-class similarity, resulting in insufficient classification accuracy and reliability.
We employ a pseudo-label and category dictionary learning approach. We extract deep features through a pre-trained convolutional neural network, select reliable pseudo-label data, and combine category dictionary learning to reduce reconstruction errors and dictionary changes during the iteration process, thereby obtaining domain-invariant feature representations. Finally, we use an SVM classifier for classification.
It effectively improves the accuracy and reliability of land cover classification in high-resolution remote sensing images, suppresses the negative impact of insufficient training samples and in-class and out-of-class differences on classification, and enhances the generalization ability of the classification model.
Smart Images
Figure CN115965867B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The image processing technology relates to the field of remote sensing image ground cover classification based on pseudo-label and category dictionary learning. BACKGROUND
[0002] Ground cover refers to the comprehensive body of the ground elements covered by natural products and artificial buildings, including ground vegetation, soil, lakes, marshes, and various buildings such as roads and houses. Ground cover is an important forcing factor of global environmental change and has received extensive attention in recent years.
[0003] With the development of remote sensing technology and sensors, the resolution of remote sensing images is getting higher and higher, which provides feasibility for ground cover mapping. Ground cover classification is an important link in ground cover mapping, which determines the quality of ground cover mapping. At present, the methods for remote sensing image classification mainly include pixel-based method, object-oriented method and scene-oriented method.
[0004] Pixel is the basic unit of remote sensing image, and using the statistical information of pixel for remote sensing image classification is the most simple and effective method. However, in high-resolution remote sensing images, the area of a single pixel corresponding to the ground object is small. Therefore, using pixel-based method for remote sensing image ground cover classification will produce a large number of pepper noise, which will reduce the accuracy of ground cover classification.
[0005] In order to overcome the shortcomings of pixel-based method, object-oriented classification method has emerged. The object-oriented method first divides the remote sensing image into independent objects with homogeneous spectrum and continuous space, and then classifies the objects. However, the size of the object obtained by image segmentation depends on the segmentation scale parameter of the image. Larger segmentation scale will lead to low segmentation of ground objects, and smaller segmentation scale will lead to over-segmentation of ground objects.
[0006] Remote sensing image scene classification can make the most intuitive understanding of the whole remote sensing image, which can greatly facilitate other field workers such as urban construction planners to make correct decisions or plans, so remote sensing image scene classification has become an important task in the field of intelligent remote sensing information processing. However, there are still some problems in the scene classification task of remote sensing image, such as:
[0007] 1) Insufficient number of training samples will lead to underfitting of the scene classification model, resulting in insufficient generalization ability of the scene classification model.
[0008] 2) High intra-class difference and inter-class similarity of high-resolution remote sensing image scene will increase the difficulty of remote sensing image scene classification. SUMMARY
[0009] To solve the existing technical problems, the application provides a remote sensing image ground cover classification method based on pseudo-label and category dictionary learning.
[0010] The specific content of the application is as follows: the remote sensing image ground cover classification method based on pseudo-label and category dictionary learning comprises the following steps:
[0011] Step (10), assuming that the source domain is existing sample data and the target domain is remote sensing image to be classified, deep layer features of the source domain and the target domain image are extracted by using a pre-trained convolutional neural network, and reliable pseudo-label data of the target domain is screened by calculating the distance of the target domain and the source domain image in the feature space;
[0012] Step (20), the source domain label data and the reliable pseudo-label data of the target domain form a joint training sample, and a source domain category dictionary is obtained by using the joint training sample combined with a category dictionary learning method;
[0013] Step (30), combined with the reliable pseudo-label data of the target domain, a transition category dictionary and a final target domain category dictionary are obtained by reducing the reconstruction error of the target domain and avoiding sudden changes of the dictionary in the iteration process;
[0014] Step (40), combined with the source domain category dictionary, the transition category dictionary and the target domain category dictionary, domain-invariant feature expression of the source domain and the target domain is obtained;
[0015] Step (50), the domain-invariant feature expression of the joint training sample is used to train an SVM classifier, and the ground cover category of the target domain image is predicted by using the trained SVM classification model.
[0016] Further, the reliable pseudo-label acquisition of step (10) comprises the following specific steps:
[0017] (10a) for the ground cover categories existing in the remote sensing image to be classified, existing sample data is collected;
[0018] (10b) assuming that the source domain is existing sample data and the target domain is remote sensing image to be classified, deep layer features of the source domain and the target domain image are extracted by using a pre-trained network;
[0019] (10c) the center of each category of the source domain in the feature space is calculated according to the following formula:
[0020]
[0021] wherein represents the deep layer feature corresponding to the i-th image of the source domain, represents the number of samples of the source domain corresponding to the category k, and respectively represent the i-th image of the source domain and the ground cover category corresponding thereto;
[0022] (10d) Calculate the Euclidean distance between the t-th feature of the target domain and the k-th class center in the source domain in the feature space according to the following formula:
[0023]
[0024] (10e) Convert the Euclidean distance into the probability that the t-th feature of the target domain belongs to the k-th class according to the following formula:
[0025]
[0026] where C is the number of land cover classes;
[0027] (10f) Calculate the pseudo label of the t-th feature of the target domain according to the following formula
[0028]
[0029] (10g) Set a threshold, and select the target domain image with a classification probability higher than a certain threshold as a reliable pseudo label of the target domain.
[0030] Further, the source domain class dictionary calculation method of step (20) is as follows:
[0031] (20a) Joint all the sample data of the source domain and the reliable pseudo label data of the target domain after screening to obtain a joint training sample set;
[0032] (20b) Based on the joint training sample set, learn the dictionary of each class of the source domain according to the following formula:
[0033]
[0034]
[0035] where Y i represents all features of the i-th class, D i represents the i-th class dictionary to be learned, X i,i represents the feature expression obtained from D i , and a is a parameter for balancing feature recognition ability and reconstruction ability, X j,i represents the feature expression obtained from D j , and T0 represents the upper limit of the number of non-zero features.
[0036] (20c) Concatenate the dictionary of each class to obtain the final source domain class dictionary D s ={D1,D2,...,D C}.
[0037] Further, the transition class visual dictionary learning method of step (30) comprises the following steps:
[0038] (30a) According to the source domain category dictionary, the target domain reliable pseudo label is updated in combination with (10b)-(10g);
[0039] (30b) The transition category visual dictionary is calculated in combination with the target domain reliable pseudo label according to the following formula
[0040]
[0041] Y t indicates the target domain feature, indicates Y t The feature expression obtained on k indicates the number of iterations, indicates the dictionary obtained in the kth iteration of the ith category.
[0042] (30c) Repeat the iteration until convergence, and the final transition category visual dictionary is the target domain category visual dictionary.
[0043] Further, step (50) comprises:
[0044] (50a) The SVM classifier is trained in combination with the domain-invariant feature expression of the source domain sample label and the target domain reliable pseudo label, and the land cover classification model is obtained.
[0045] (50b) The land cover classification prediction result is obtained by using the land cover classification model to predict the target domain image.
[0046] The present application is aimed at the deficiencies of the prior art, based on the pseudo label and category dictionary learning of remote sensing image transfer land cover classification method, which suppresses the negative influence of insufficient training samples, large intra-class difference and high inter-class similarity on high-resolution remote sensing image land cover classification, and effectively improves the accuracy and reliability of remote sensing image land cover classification in the case of missing training samples. BRIEF DESCRIPTION OF DRAWINGS
[0047] The specific embodiments of the present application will be further illustrated below in combination with the drawings.
[0048] Figure 1 It is a flowchart of the remote sensing image transfer land cover classification method based on pseudo label and category dictionary learning.
[0049] Figure 2 It is a source domain sample data;
[0050] Figure 3 It is a target domain image to be classified;
[0051] Figure 4 It is the target domain reliable pseudo label result obtained by the remote sensing image transfer land cover classification method based on pseudo label and category dictionary learning.
[0052] Figure 5 The classification result of the remote sensing image transfer ground cover classification method based on pseudo label and category dictionary learning on the target domain image. DETAILED DESCRIPTION
[0053] The application will be further described in detail below with reference to the accompanying drawings. In conjunction with the drawings, the specific steps of the application are as follows: Figure 1
[0054] (10) Assuming that the source domain is existing sample data and the target domain is remote sensing image to be classified, deep features are extracted for the source domain and target domain images by using a pre-trained convolutional neural network, and reliable pseudo label data of the target domain is screened by calculating the distance of the target domain and the source domain images in the feature space.
[0055] (10a) For the ground cover categories existing in the remote sensing image to be classified, existing sample data is collected.
[0056] (10b) Assuming that the source domain is existing sample data and the target domain is remote sensing image to be classified, deep features are extracted for the source domain and target domain images by using a pre-trained network.
[0057] Figure 2 The source domain sample data of 8 different ground cover categories is shown, Figure 3 The target domain image to be classified is shown.
[0058] (10c) The center of each category of the source domain in the feature space is calculated according to the following formula.
[0059]
[0060] Wherein represents the deep feature corresponding to the i-th image of the source domain, represents the number of source domain samples corresponding to the k-th category, and respectively represent the i-th image of the source domain and the ground cover category corresponding thereto.
[0061] (10d) The Euclidean distance of the t-th feature of the target domain and the k-th category center of the source domain in the feature space is calculated according to the following formula.
[0062]
[0063] (10e) The Euclidean distance is converted into the probability that the t-th feature of the target domain belongs to the k-th category according to the following formula.
[0064]
[0065] Wherein C is the number of ground cover categories
[0066] (10f) Calculate the pseudo label of the t-th feature of the target domain according to the following formula
[0067]
[0068] (10g) Set a threshold, and the target domain image with a classification probability higher than the threshold is selected as the reliable pseudo label of the target domain.
[0069] Figure 4 The target domain reliable pseudo label obtained by the remote sensing image transfer ground cover classification method based on pseudo label and class dictionary learning is shown.
[0070] (20) The source domain label data and the target domain reliable pseudo label data form a joint training sample, and the source domain class dictionary is obtained by combining the class dictionary learning method with the joint training sample.
[0071] (20a) Jointly obtain all sample data of the source domain and the filtered target domain reliable pseudo label data to obtain a joint training sample set
[0072] (20b) Based on the joint training sample set, learn the dictionary of each class of the source domain according to the following formula.
[0073]
[0074]
[0075] where Y i represents all features of the i-th class, D i represents the i-th class dictionary to be learned, X i,i represents the feature expression obtained from D i , and a is a parameter for balancing feature recognition ability and reconstruction ability. X j,i represents the feature expression obtained from D j , and T0 represents the upper limit of the number of non-zero features.
[0076] (20c) Concatenate each class dictionary to obtain the final source domain class dictionary D s ={D1,D2,...,D C}.
[0077] (30) By combining the target domain reliable pseudo label data, reducing the reconstruction error of the target domain, and avoiding sudden changes in the dictionary during the iteration process, the transition class dictionary and the final target domain class dictionary are obtained.
[0078] (30a) According to the source domain class dictionary, update the target domain reliable pseudo label combined with (10b)-(10g).
[0079] (30b) Calculate the transition category visual dictionary according to the following formula, combined with reliable pseudo-labels in the target domain.
[0080]
[0081] Y t Represents the features of the target domain. Y represents t exist The feature representation obtained above, where k represents the number of iterations, Let represent the dictionary obtained in the k-th iteration for the i-th category.
[0082] (30c) Repeat the iteration until convergence, and the final transition category visual dictionary is the target domain category visual dictionary.
[0083] (40) By combining the source domain category dictionary, the transition category dictionary and the target domain category dictionary, the domain-invariant feature representations of the source domain and the target domain are obtained.
[0084] (40a) According to the following formula, the domain-invariant feature representation of the source domain image is obtained by combining the source domain category dictionary, the transition category dictionary and the target domain category dictionary.
[0085]
[0086] (40b) According to the following formula, the domain-invariant feature representation of the target domain image is obtained by combining the source domain category dictionary, the transition category dictionary and the target domain category dictionary.
[0087]
[0088] (50) Use the domain-invariant feature representation of the joint training samples to train the SVM classifier, and use the trained SVM classification model to predict the land cover category of the target domain image.
[0089] (50a) Train an SVM classifier by combining the domain-invariant feature representations of source domain sample labels and target domain reliable pseudo-labels to obtain a land cover classification model.
[0090] (50b) Use the land cover classification model to predict the target domain image and obtain the land cover classification results.
[0091] Figure 5 The results of the remote sensing image transfer land cover classification method based on label and category dictionary learning are shown on the land cover classification results of the target domain image.
[0092] The application realizes a remote sensing image transfer ground cover classification method based on pseudo-label and category dictionary learning, and can provide core technical support for application fields such as national land resource investigation, environmental monitoring, agricultural and forestry monitoring census and disaster warning evaluation. Compared with the existing remote sensing image transfer ground cover classification method, the method has the remarkable advantages that the negative influence of insufficient training samples, large intra-class difference and high inter-class similarity on high-resolution remote sensing image ground cover classification can be effectively inhibited, and the ground cover classification precision and reliability of the remote sensing image can be effectively improved in the case of missing training samples.
[0093] In the above description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the above description is merely a preferred embodiment of the present application, and the present application can be carried out in many different ways other than those described above, and therefore the present application is not limited by the specific implementation disclosed above. Meanwhile, any person skilled in the art can make many possible changes and modifications to the technical solutions of the present application, or modify equivalent embodiments, by using the methods and technical contents disclosed above, without departing from the scope of the technical solutions of the present application. Any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application, without departing from the content of the technical solutions of the present application, shall still fall within the scope of protection of the technical solutions of the present application.
Claims
1. A remote sensing image land cover classification method based on pseudo-label and class dictionary learning, characterized in that: Comprising the following steps: Step (10), assuming that the source domain is existing sample data and the target domain is remote sensing images to be classified, a pre-trained convolutional neural network is used to extract deep features of the source domain and target domain images, and reliable pseudo-label data of the target domain is screened by calculating the distance between the target domain and the source domain images in the feature space; Step (20), the source domain label data and the reliable pseudo-label data of the target domain form a joint training sample, and a source domain category dictionary is obtained by using the joint training sample combined with a category dictionary learning method; Step (30), combined with the reliable pseudo-label data of the target domain, a transition category dictionary is obtained by reducing the reconstruction error of the target domain and avoiding sudden changes of the dictionary in the iteration process, and the iteration is repeated until convergence, and the final transition category dictionary is the target domain category dictionary; Step (40), combined with the source domain category dictionary, the transition category dictionary and the target domain category dictionary, domain-invariant feature expression of the source domain and the target domain is obtained; Step (50), the domain-invariant feature expression of the joint training sample is used to train an SVM classifier, and the trained SVM classification model is used to predict the land cover category of the target domain image.
2. The pseudo-label and class dictionary learning based remote sensing image land cover classification method according to claim 1, characterized in that, The reliable pseudo-label acquisition of step (10) comprises the following specific steps: (10a) Collect existing sample data for the land cover categories existing in the remote sensing images to be classified; (10b) Assuming that the source domain is existing sample data and the target domain is remote sensing images to be classified, a pre-trained network is used to extract deep features of the source domain and target domain images; (10c) The center of each category of the source domain in the feature space is calculated according to the following formula: ， wherein denotes the deep features of the lth image in the source domain, denotes the number of source domain samples corresponding to class i, and denotes the lth image in the source domain and its corresponding land cover class, respectively. (10d) Calculate the Euclidean distance between the i-th feature of the target domain and the i-th class center in the source domain in the feature space according to the following formula: D = (xi - ci)2 ， (10e) Convert the Euclidean distance into the probability that the feature belongs to the i-th class according to the following formula: P(i | x) = e- d(i) / Σ e- d(j) ， Wherein C is the number of land cover categories; (10f) The pseudo labels for the features of the target domain are calculated according to the following formula Ytarget= Ysource ， (10g) Set a threshold, and the target domain images with a classification probability higher than a certain threshold are selected as reliable pseudo-labels of the target domain.
3. The pseudo-label and class dictionary learning based remote sensing image land cover classification method according to claim 2, characterized in that, The source domain category dictionary calculation method of step (20) is as follows: (20a) Joint the source domain all sample data and the filtered reliable pseudo-label data of the target domain to obtain a joint training sample set; (20b) Based on the joint training sample set, the dictionary of each category of the source domain is learned according to the following formula: ， ， wherein, represents all features of the i-th class, represents a dictionary of the i-th class to be learned, represents a feature expression obtained from , is a parameter for balancing feature recognition ability and reconstruction ability, represents a feature expression obtained from , represents an upper limit of the non-zero number in the feature expression; (20c) concatenating the dictionaries of each category to obtain a final source domain category dictionary .
4. The pseudo-label and class dictionary learning based remote sensing image land cover classification method according to claim 3, characterized in that, The transition category visual dictionary learning method of step (30) comprises the following steps: (30a) According to the source domain category dictionary, the reliable pseudo-label of the target domain is updated combined with (10b)-(10g); (30b) According to the following formula, the transition category visual dictionary is calculated combined with the reliable pseudo-label of the target domain ， denote target domain features, denote In the feature representation obtained on, k denotes the iteration number, denote the dictionary obtained in the i-th class in the k-th iteration; (30c) Repeat the iteration until convergence, and the final transition category visual dictionary is the target domain category visual dictionary. 5.The pseudo-label and class dictionary learning based remote sensing image land cover classification method of claim 1, wherein, Step (50) comprises: (50a) Combine the domain-invariant feature expression of the source domain sample label and the reliable pseudo-label of the target domain to train an SVM classifier, and obtain a land cover classification model; (50b) The land cover classification model is used to predict the target domain image to obtain a land cover classification prediction result.
Citation Information
Patent Citations
Dictionary creation and domain self-adaption-based open world target detection training method
CN115099358A
Industrial system fault diagnosis method and system based on sub-field adaptation dictionary learning
CN115129029A