An Open Set Cross-Domain Hyperspectral Image Classification Method and System Based on Critical Reflective Learning

Abstract

This application discloses an open-set cross-domain hyperspectral image classification method and network based on critical reflective learning, belonging to the field of computer vision technology. The method includes: extracting features from source and target domain data to obtain source and target domain embedding features; inputting the source and target domain embedding features into a teacher network and a student network, respectively, to obtain teacher and student network outputs; expanding the dimension of the teacher network output to match the dimension of the student network output to obtain a modified teacher network, while simultaneously controlling the student network to learn the modified teacher network's output to obtain a modified student network; and inputting the target domain embedding features into the modified student network to obtain the category prediction result for the target domain data. This application introduces the teacher-student approach to the open-set cross-domain hyperspectral image classification task, using the teacher network to guide the student network in mastering the decision boundaries of known / unknown categories.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, specifically relating to an open-set cross-domain hyperspectral image classification method and system based on critical reflective learning. Background Technology

[0002] In recent years, deep learning-based HSI (Hyperspectral Imaging) classification methods have achieved impressive performance in fields such as remote sensing interpretation, environmental monitoring, and land use planning. However, these methods typically assume that training and testing data follow the principle of independent and identically distributed characteristics. In real-world scenarios, due to differences in environment, equipment, and other factors during HSI acquisition, there is often a significant data distribution discrepancy between the training (source domain) and testing (target domain). In such cases, these methods often fail to achieve satisfactory performance.

[0003] Domain adaptation can mitigate the distributional differences between different HSIs through cross-domain distribution adaptation, ensuring that models trained on source domain data maintain good generalization performance in the target domain. However, it should be noted that the effectiveness of existing domain adaptation methods largely relies on the assumption of closed sets where the source and target domains have the same classes, making it difficult to adapt to scenarios where the target domain contains open sets with unknown classes. In this case, due to the lack of an unknown class identification mechanism, such methods may incorrectly align known and unknown classes during cross-domain distribution adaptation, thus inducing negative transfer.

[0004] Therefore, open-set cross-domain hyperspectral image classification has emerged, aiming to extend cross-domain scenes from closed sets to open sets, enabling models to identify unknown classes in the target domain. Existing methods often rely on threshold-based solutions when identifying known classes. Specifically, researchers calculate distance metrics between samples, such as Euclidean or cosine distance, and compare them with a preset threshold to determine the sample category. The threshold is usually determined based on prototype calculations of known class samples in the source domain data or estimations of output probabilities. However, although known and unknown classes can be distinguished in the feature space by setting appropriate thresholds, such methods often face challenges in generalization performance limitations in complex scenes with blurred inter-class boundaries.

[0005] Therefore, overcoming the limitations of threshold-based solutions and developing an unknown class recognition mechanism capable of learning complex boundary relationships between unknown and known classes is a key problem to be solved. The difficulty lies in the fact that, since the unknown class is unknown during training, the model lacks supervisory information about the unknown class, posing a significant challenge to unknown class recognition. Summary of the Invention

[0006] Purpose of the invention: In view of the problems existing in the above-mentioned background technology, this application develops an open set cross-domain hyperspectral image classification method and system based on critical reflective learning.

[0007] Technical Solution: In a first aspect, embodiments of this application provide an open-set cross-domain hyperspectral image classification method based on critical reflective learning, comprising:

[0008] Feature extraction is performed on the source domain data and the target domain data respectively to obtain the source domain embedding features and the target domain embedding features;

[0009] The source domain embedding features and the target domain embedding features are input into the teacher network to obtain the teacher network output; the source domain embedding features and the target domain embedding features are input into the student network to obtain the student network output.

[0010] In response to the fact that the dimension of the teacher network output is smaller than the dimension of the student network output, the dimension of the teacher network output is expanded so that the dimension of the teacher network output is equal to the dimension of the student network output, thereby obtaining a modified teacher network; the student network is then controlled to learn the output results of the modified teacher network, thereby obtaining a modified student network.

[0011] The target domain embedding features are input into the modified student network to obtain the category prediction results for the target domain data.

[0012] In some embodiments, the step of expanding the dimensions of the teacher network output includes:

[0013] The entropy value of the probability distribution output by the teacher network after classifying the embedded features of the target domain is obtained based on the entropy-assisted model.

[0014] The determination factor is obtained based on the entropy value;

[0015] The decision factor is used as a dimension to expand the predicted output of the teacher network, thereby expanding the output dimension of the teacher network.

[0016] In some embodiments, the formula for representing the determination factor includes:

[0017]

[0018] Wherein, ω is the determination factor, used to characterize the target domain data x. t The probability of belonging to an unknown category; H(TE(Z(x) t The teacher network TE embeds the feature Z(x) into the target domain. t The entropy value of the probability distribution output after classification:

[0019]

[0020] Where i represents the known categories of the data, i = 1, 2, ..., K, and K is the number of known categories; p(y i |x t The target domain data x output by the teacher network TE is... t The probability of belonging to a known category i; y i The predicted label is for the known category i.

[0021] In some embodiments, the step of expanding the output dimension of the teacher network includes:

[0022] Based on the determination factor and the teacher network output, the adjusted probability distribution of the known categories is obtained by element-wise multiplication;

[0023] The output dimension of the teacher network is expanded based on the adjusted probability distribution of the decision factor and the known categories, thereby constructing a new output distribution for the teacher network:

[0024] q(y i |x t ) = p known (y i |x t :ω), i=1,2,…,K;

[0025] Where q(y) i |x t ) represents the new output distribution of the teacher network; p known (y i |x t )=p(y i |x t )⊙(1-ω),i=1,2,…,K is the adjusted probability distribution of the known categories, p(y i |x t The target domain data x output by the teacher network TE is... t The probability of belonging to a known category i is used to characterize the teacher network output, ⊙ is the element-wise product, and ω is the decision factor used to characterize the target domain data x. t The probability of belonging to an unknown category, 1-ω, is used to characterize the target domain data x. t The probability of belonging to a known category, where i is the known category of the data, i = 1, 2, ..., K, and K is the number of known categories.

[0026] In some embodiments, the step of controlling the student network to learn the output of the modified teacher network and obtaining the modified student network includes:

[0027] Obtain the minimized output of the student network;

[0028] The modified student network is obtained by controlling the student network to learn based on the KL divergence between the minimized output and the output of the modified teacher network.

[0029] In some embodiments, the open-set cross-domain hyperspectral image classification method based on critical reflective learning further includes, prior to inputting the target domain embedding features into the modified student network:

[0030] Obtain the known category data from the prediction results of the target domain output by the corrected student network, and perform subdomain alignment with the known categories in the source domain data;

[0031] Determine the similarity between source domain embedded features, and distinguish between unknown category data in the target domain data and known category data in the source domain data based on the similarity.

[0032] In some embodiments, the step of obtaining known category data from the prediction results of the target domain output by the corrected student network and performing subdomain alignment with the known categories in the source domain data includes:

[0033] Align the source and target domains as a whole through domain adversarial analysis;

[0034] Based on the data category, the source domain data and the target domain data are divided into multiple subdomains, and the number of subdomains in the source domain data, the number of subdomains in the target domain data, and the data category are the same.

[0035] The modified student network outputs the prediction results of the target domain data, filters out known category samples, and performs LMMD subdomain alignment with the known categories of the source domain data.

[0036] In some embodiments, the step of determining the similarity between source domain embedding features and distinguishing unknown category data in the target domain and known category data in the source domain data based on the similarity includes:

[0037] The similarity matrix between the domain-invariant features of the source domain data is determined based on the Gaussian kernel function.

[0038] The contrast loss function is determined based on the similarity matrix;

[0039] The contrastive loss function is used to determine the distinction between unknown data in the target domain and known data in the source domain.

[0040] In some embodiments, the modified student network is further guided to identify error-prone samples from the modified teacher network and learn from the error-prone samples to optimize the modified student network.

[0041] Based on the average value of the entropy, the target domain data is divided into known categories and unknown categories, and the centers of the known categories and unknown categories are determined.

[0042] Obtain the Euclidean distance from each sample in the target domain data to the known class center and the unknown class center, and concatenate them to obtain the distance set from the sample to each class center. Then, convert the distance set into the confidence score of the corresponding class through an activation function.

[0043] Obtain the difference between the confidence level and the predicted probability of the corrected teacher network, determine the samples that meet the difference threshold as error samples, and obtain the set of error-prone samples;

[0044] The set of error-prone samples is input into the modified student network to obtain the predicted probability output.

[0045] The predicted category index for each sample is determined based on the predicted probability output;

[0046] Based on the predicted category index, the error-prone samples are divided into known category samples and unknown category samples;

[0047] Determine the distance from the known class sample to the known class center and the Euclidean distance from the unknown class sample to the unknown class center;

[0048] The loss function is the sum of the distances from the error-prone samples to their class centers.

[0049] The modified student network is optimized based on the loss function.

[0050] Secondly, embodiments of this application also provide an open-set cross-domain hyperspectral image classification system based on critical reflective learning, comprising:

[0051] The feature extraction module is used to extract features from the source domain data and the target domain data respectively, and obtain source domain embedding features and target domain embedding features.

[0052] The prediction output module is used to input the source domain embedding features and the target domain embedding features into the teacher network to obtain the teacher network output; and to input the source domain embedding features and the target domain embedding features into the student network to obtain the student network output.

[0053] An output correction module is configured to, in response to the fact that the dimension of the teacher network output is smaller than the dimension of the student network output, expand the dimension of the teacher network output so that the dimension of the teacher network output is equal to the dimension of the student network output, thereby obtaining a corrected teacher network; and control the student network to learn the output result of the corrected teacher network, thereby obtaining a corrected student network.

[0054] The category prediction module is used to input the target domain embedding features into the modified student network to obtain the category prediction results for the target domain data.

[0055] Beneficial Effects: Compared with existing technologies, the present application provides an open-set cross-domain hyperspectral image classification method based on critical reflective learning. First, features are extracted from source and target domain data to obtain source and target domain embedding features. These features are then input into a teacher network and a student network, respectively, to obtain teacher and student network outputs. The dimensions of the teacher and student network outputs are determined. If the teacher network output dimension is smaller than the student network output dimension, it is expanded to match the student network output dimension, resulting in a corrected teacher network. Simultaneously, the student network learns from the corrected teacher network output, resulting in a corrected student network. The target domain embedding features are then input into the corrected student network to obtain the category prediction results for the target domain data. This application introduces the teacher-student concept into the open-set cross-domain hyperspectral image classification task, overcoming the limitation of open-set classifiers relying on thresholds. The teacher network guides the student network to master the decision boundaries of known / unknown categories. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating the steps of an open-set cross-domain hyperspectral image classification method based on critical reflective learning, provided in an embodiment of this application.

[0058] Figure 2 A flowchart illustrating the steps of expanding the dimension of the teacher network output in the open-set cross-domain hyperspectral image classification method based on critical reflective learning provided in this application embodiment;

[0059] Figure 3A detailed flowchart of step 330 in the open set cross-domain hyperspectral image classification method based on critical reflective learning provided in the embodiments of this application;

[0060] Figure 4 The flowchart illustrates the steps for obtaining the corrected student network by controlling the output of the teacher network in the open-set cross-domain hyperspectral image classification method based on critical reflective learning provided in this application embodiment.

[0061] Figure 5 A flowchart illustrating the subdomain alignment steps in an open-set cross-domain hyperspectral image classification method based on critical reflective learning, provided in this application embodiment.

[0062] Figure 6 A flowchart illustrating the steps of contrastive learning in an open-set cross-domain hyperspectral image classification method based on critical reflective learning, provided in an embodiment of this application.

[0063] Figure 7 A flowchart illustrating the steps of a modified student network learning from error-prone samples in a modified teacher network in an open-set cross-domain hyperspectral image classification method based on critical reflective learning, as provided in this application embodiment.

[0064] Figure 8 A module connection diagram of an open-set cross-domain hyperspectral image classification system based on critical reflective learning provided in the embodiments of this application;

[0065] Figure 9 Design diagram of an open-set cross-domain hyperspectral image classification method based on critical reflective learning provided in embodiments of this application;

[0066] Figure labels: 10, Feature extraction module; 20, Prediction output module; 30, Output correction module; 40, Category prediction module. Detailed Implementation

[0067] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0068] In recent years, deep learning-based HSI (Model-Based Sensor) classification methods have achieved impressive performance in fields such as remote sensing interpretation, environmental monitoring, and land use planning. However, these methods typically assume that training and testing data follow the principle of independent and identically distributed (i.i.d.) distribution. In real-world scenarios, due to differences in environment, equipment, and other factors during HSI acquisition, there is often a significant data distribution discrepancy between the training (source domain) and testing (target domain) scenarios. In such cases, these methods often struggle to achieve satisfactory performance.

[0069] Domain adaptation can mitigate the distributional differences between different HSIs through cross-domain distribution adaptation, ensuring that models trained on source domain data maintain good generalization performance in the target domain. However, it should be noted that the effectiveness of existing domain adaptation methods largely relies on the assumption of closed sets where the source and target domains have the same classes, making it difficult to adapt to scenarios where the target domain contains open sets with unknown classes. In this case, due to the lack of an unknown class identification mechanism, such methods may incorrectly align known and unknown classes during cross-domain distribution adaptation, thus inducing negative transfer.

[0070] Therefore, open-set cross-domain hyperspectral image classification has emerged, aiming to extend cross-domain scenes from closed sets to open sets, enabling models to identify unknown classes in the target domain. Existing methods often rely on threshold-based solutions when identifying known classes. Specifically, researchers calculate distance metrics between samples, such as Euclidean or cosine distance, and compare them with a preset threshold to determine the sample category. The threshold is usually determined based on prototype calculations of known class samples in the source domain data or estimations of output probabilities. However, although known and unknown classes can be distinguished in the feature space by setting appropriate thresholds, such methods often face challenges in generalization performance limitations in complex scenes with blurred inter-class boundaries.

[0071] Therefore, overcoming the limitations of threshold-based schemes and developing an unknown class recognition mechanism capable of learning complex boundary relationships between unknown and known classes is a key problem to be solved. The difficulty lies in the fact that since unknown classes are unknown during training, the model lacks supervisory information about them, posing a significant challenge to unknown class recognition. To address this issue, this invention uses entropy as a measure of uncertainty. High entropy values ​​are generated when the decision boundary is uncertain about the classification result of the target instance. Furthermore, for unknown categories, the lack of prior training instances prevents the classifier from accurately predicting its category, also leading to increased entropy. Based on this, the probability information of unknown categories can be extracted from the predictions of the closed-set classifier. Furthermore, the closed-set classifier can be viewed as a teacher network, providing supervisory signals to the open-set classifier, which then acts as a student network, learning the complex boundary relationships between known and unknown classes from the output of the teacher network.

[0072] In view of this, embodiments of this application provide an open-set cross-domain hyperspectral image classification method based on critical reflective learning. First, features are extracted from source and target domain data to obtain source and target domain embedding features. These features are then input into a teacher network and a student network, respectively, to obtain teacher and student network outputs. The dimensions of the teacher and student network outputs are determined. If the teacher network output dimension is smaller than the student network output dimension, it is expanded to match the student network output dimension, resulting in a corrected teacher network. Simultaneously, the student network learns from the corrected teacher network output, resulting in a corrected student network. The target domain embedding features are then input into the corrected student network to obtain the category prediction results for the target domain data. This application introduces the teacher-student approach to open-set cross-domain hyperspectral image classification, overcoming the limitation of open-set classifiers relying on thresholds. Through teacher network guidance, the student network masters the decision boundaries of known / unknown categories.

[0073] In some embodiments, please refer to Figure 1 and Figure 9 , Figure 1 A flowchart illustrating the steps of the open-set cross-domain hyperspectral image classification method based on critical reflective learning provided in this application embodiment. Figure 9 The diagram illustrates the design of the open-set cross-domain hyperspectral image classification method based on critical reflective learning provided in this application embodiment. Specifically, the open-set cross-domain hyperspectral image classification method based on critical reflective learning provided in this application embodiment is implemented through steps 100 to 400:

[0074] Step 100: Extract features from the source domain data and the target domain data respectively to obtain the source domain embedding features and the target domain embedding features.

[0075] Specifically, the hyperspectral images of the source domain and the target domain are input into the feature extractor, and features are extracted from both spatial and spectral aspects of the hyperspectral images of the source and target domains to obtain discriminative domain-invariant features:

[0076]

[0077] Where G is the feature extractor; x s For data in the source domain; x t For the data in the target domain; Z(x) s Z(x) represents the source domain embedding feature, used to characterize the domain-invariant features of the source domain that are distinctive; t ) is the target domain embedding feature, used to characterize the domain-invariant features that make the target domain identifiable.

[0078] In some embodiments, before inputting data into the teacher network, a pre-built teacher network is trained using labeled source domain data to obtain...

[0079]

[0080] in, For the trained teacher network TE;n s L represents the amount of data in the source domain, used to characterize the number of samples in the source domain. CE Cross-entropy loss; G is the feature extractor; x s The data is from the source domain, i.e., samples from the source domain; y s The labels are the source domain samples; Softmax is a smooth maximization operation used to transform the neural network output into a probability distribution. For the student network, a source classification loss L is constructed on the source domain. cls1 Furthermore, for the target domain, students need to learn the classification boundary of the unknown class through online learning to obtain the classification loss L. cls2 .

[0081] Step 200: Input the source domain embedding features and the target domain embedding features into the teacher network and obtain the teacher network output; input the source domain embedding features and the target domain embedding features into the student network and obtain the student network output.

[0082] Step 300: In response to the fact that the dimension of the teacher network output is smaller than the dimension of the student network output, expand the dimension of the teacher network output so that the dimension of the teacher network output is equal to the dimension of the student network output, and obtain the modified teacher network; control the student network to learn the output result of the modified teacher network, and obtain the modified student network.

[0083] In some embodiments, please refer to Figure 2 , Figure 2 The flowchart illustrates the steps for expanding the dimension of the teacher network output in the open-set cross-domain hyperspectral image classification method based on critical reflective learning provided in this application embodiment. The method for expanding the dimension of the teacher network output is specifically implemented through steps 310 to 330:

[0084] Step 310: Obtain the entropy value of the probability distribution output by the teacher network after classifying the embedded features of the target domain based on the entropy-assisted model.

[0085] Understandably, since there are no available labels in the target domain, entropy is used to assist the model in distinguishing between known and unknown class instances in the target domain. This enhances the model's robustness when facing unknown classes and helps learn a more accurate decision boundary, especially between known and unknown classes. The entropy value is defined as H(TE(Z(x)). tThe teacher network TE extracts the target domain embedding feature Z(x) from the feature extractor G. t The entropy of the probability distribution output after classification is given by the following formula:

[0086]

[0087] Among them, H(TE(Z(x) t For the teacher network TE, the embedding feature Z(x) in the target domain is... t The entropy value of the probability distribution output after classification; i represents the known categories of the data, i = 1, 2, ..., K, where K is the number of known categories; p(y i |x t The target domain data x output by the Teacher Network (TE) t The probability of belonging to a known category i; y i The predicted label is for the known category i.

[0088] Step 320: Obtain the decision factor based on the entropy value.

[0089] Specifically, the formulas for representing the criterion factors include:

[0090]

[0091] Where ω is the decision factor, used to characterize the target domain data x. t The probability of belonging to an unknown category; H(TE(Z(x) t For the teacher network TE, the embedding feature Z(x) in the target domain is... t The entropy value of the probability distribution output after classification.

[0092] Understandably, the decision factor ω transforms the entropy value into a probability value between 0 and 1, representing instance x. t The probability of belonging to an unknown category. When the entropy value H(TE(Z(x)) t When the entropy is low (i.e., the classifier is relatively certain about the classification result), ω is close to 0; when the entropy is high (i.e., the classifier is uncertain about the classification result), ω is close to 1, indicating that the instance may be of an unknown category.

[0093] Step 330: Use the decision factor as a dimension to expand the output dimension of the teacher network prediction.

[0094] In some embodiments, please refer to Figure 3 , Figure 3 This is a flowchart illustrating step 330 of the open-set cross-domain hyperspectral image classification method based on critical reflective learning provided in this application embodiment. The method for expanding the output dimension of the teacher network is specifically implemented through steps 331 to 332:

[0095] Step 331: Based on the decision factor and the teacher network output, obtain the adjusted probability distribution of the known categories by element-wise multiplication.

[0096] Step 332: Expand the output dimension of the teacher network based on the adjusted probability distribution of the decision factor and known categories, and construct a new output distribution for the teacher network.

[0097] Understandably, since the teacher network's output dimension is the known number of categories K, the student network cannot directly learn the teacher network's output knowledge. Therefore, the teacher network's output dimension is first expanded to K+1. Specifically, the decision factor ω can be considered to some extent as the probability that a sample belongs to an unknown category, while 1-ω can represent the probability that a sample belongs to a known category. The 1-ω is then compared with the teacher network's original output {p(y...}... i |x t By combining {i = 1, 2, ..., K}, and multiplying element-wise, we obtain the adjusted probability distribution {p} for the known categories. known (y i |x t ),i=1,2,…,K}:

[0098] p known (y i |x t )=p(y i |x t )⊙(1-ω),i=1,2,…,K;

[0099] By adding ω as an additional dimension to represent the probability of the unknown category, the output dimension of the teacher network is expanded from K to K+1, thus constructing a new output distribution q(y). i |x t ):

[0100] q(y i |x t ) = p known (y i |x t :ω), i=1,2,…,K;

[0101] Where q(y) i |x t ) represents the new output distribution of the teacher network; p known (y i |x t )=p(y i |x t )⊙(1-ω),i=1,2,…,K is the adjusted probability distribution of the known categories, p(y i |x t The target domain data x output by the Teacher Network (TE) tThe probability of belonging to a known category i is used to characterize the teacher network output, ⊙ represents the element-wise product, and ω is the decision factor used to characterize the target domain data x. t The probability of belonging to an unknown category, 1-ω, is used to characterize the target domain data x. t The probability of belonging to a known category, where i is the known category of the data, i = 1, 2, ..., K, and K is the number of known categories.

[0102] In some embodiments, please refer to Figure 4 , Figure 4 The flowchart for obtaining the corrected student network output in the open-set cross-domain hyperspectral image classification method based on critical reflective learning provided in this application embodiment is specifically implemented through steps 340 to 350:

[0103] Step 340: Obtain the minimized output of the student network.

[0104] Step 350: Control the student network to learn based on minimizing the KL divergence between the output and the output of the corrected teacher network to obtain the corrected student network.

[0105] Understandably, student networks minimize their own output. The output q(y) of the modified teacher network i |x t The KL divergence between known and unknown classes is used for learning, which helps to learn the complex boundary relationships between known and unknown classes and improve the accuracy of identifying unknown classes.

[0106]

[0107] Among them, L KL To minimize the KL divergence between the output and the output of the corrected teacher network; Let q(y) be the probability that the i-th sample in the student network ST belongs to the j-th class, which is used to represent minimizing the output; i |x t ) represents the new output distribution of the teacher network, used to characterize the output of the modified teacher network; q ij (y i |x t ) represents the probability that the i-th sample output by the teacher network belongs to the j-th category; i is the sample, N is the batch size of the samples, j is the category, and K+1 is the total number of categories.

[0108] In some embodiments, please refer to Figure 5 , Figure 5The flowchart of the subdomain alignment steps in the open-set cross-domain hyperspectral image classification method based on critical reflective learning provided in this application embodiment shows that the method of obtaining known category data from the predicted results of the target domain output by the corrected student network and aligning it with the known categories in the source domain is specifically implemented through steps a to c:

[0109] Step a: Align the source and target domains as a whole through domain adversarial analysis.

[0110] Understandably, by using domain adversarial methods to align the source and target domains holistically, the feature extractor G attempts to learn a domain-invariant feature representation that makes it difficult for the domain discriminator D to distinguish features between the source and target domains, while the domain discriminator D attempts to maximize its ability to distinguish between the source and target domains.

[0111]

[0112]

[0113] Where G is the feature extractor; D is the neighborhood discriminator; L CE For cross-entropy loss; z s Embedded features of source domain data; y s Labels for source domain samples; L d To combat losses in the domain; For the data distribution p from the source domain s The expected value of the sampled value; For the data distribution p of the target domain, the sample is... t The expected value of the mid-sample; z s and z t The features extracted by the feature extractor G from the source and target domain samples.

[0114] Step b: Based on the data category, divide the source domain data and target domain data into multiple subdomains respectively. The number of subdomains in the source domain data, the number of subdomains in the target domain data, and the category of the samples are the same.

[0115] Step c: Use the modified student network to output the prediction results of the target domain data, filter out the known category samples and perform LMMD subdomain alignment with the known categories of the source domain data.

[0116] Understandably, the source domain D is determined based on the category of the sample. s and target domain D t The target domain is divided into K subdomains. The predicted results for the target domain samples are then used with the modified student network. Samples of known classes are selected and aligned with the known classes of the source domain using LMMD subdomain alignment.

[0117]

[0118] In the formula, L LMMD For subdomain alignment operations; X s X is the set of samples from the source domain; t Y is the target domain sample set; s A collection of source domain tags; Predict a set of labels for the target domain; k∈{1,2,...,K} is the category index, and K is the number of subdomains; This represents the i-th source domain sample; Represents the source domain; This represents the j-th target domain sample; Indicates the target domain; This represents the source domain sample features obtained after the feature extractor. and the target domain sample features obtained by the feature extractor Euclidean distance.

[0119] In some embodiments, please refer to Figure 6 , Figure 6 The flowchart of the contrastive learning steps in the open-set cross-domain hyperspectral image classification method based on critical reflective learning provided in the embodiments of this application shows that the method of determining the similarity between source domain embedded features and distinguishing unknown category data in the target domain and known category data in the source domain based on the similarity is specifically implemented through steps d to f:

[0120] Step d: Determine the similarity matrix between the domain-invariant features of the source domain based on the Gaussian kernel function;

[0121] Step e: Determine the contrast loss function based on the similarity matrix;

[0122] Step f: Based on the contrastive loss function, determine the unknown data in the target domain and the known data in the source domain.

[0123] Understandably, a Gaussian kernel function can be defined to compute the similarity matrix between source domain features z:

[0124]

[0125] Among them, L con To compare the loss functions; i is a sample; N is the batch size of the samples; P(i) is the loss function compared to sample x. i The set of sample indices p of the same category; A(i) is the set of indices p of samples other than x. i The set of all sample indices a except for τ; τ is the temperature parameter; K(x) i ,x p ) and K(x i ,x a ) represent samples x respectively i Its positive sample xp and negative sample x a The Gaussian kernel similarity between them; μ is the bandwidth parameter of the Gaussian kernel.

[0126] This contrastive loss function encourages the model to bring samples belonging to the same category closer together in the feature space, while pushing samples from different categories further apart. In this way, the student network can form clearer category boundaries in the feature space, thereby improving the model's ability to distinguish between known and unknown samples.

[0127] In some embodiments, please refer to Figure 7 , Figure 7 The flowchart illustrates the steps of a modified student network learning from error-prone samples in a modified teacher network within an open-set cross-domain hyperspectral image classification method based on critical reflective learning, as provided in this application embodiment. The modified student network identifies and learns from the modified teacher network. The method for optimizing the modified student network is determined through steps g to o.

[0128] Step g: Based on the average entropy value, divide the target domain data into known categories and unknown categories, and determine the center of the known category and the center of the unknown category.

[0129] Specifically, based on the average value of entropy. The samples of the target domain data are divided into known category samples X. k and unknown category sample X unk .

[0130] Step h: Obtain the Euclidean distance from each sample in the target domain data to the known class center and the unknown class center, and concatenate them to obtain the distance set from the sample to each class center. Then, convert the distance set into the confidence score of the corresponding class through the activation function.

[0131] Specifically, the distance from each sample to the known class center c is calculated. k and unknown category center c unk Euclidean distance d k and d unk Then, d k and d unk By concatenating these distances, we obtain a set d of predicted distances d from known and unknown class samples in a batch to their respective class centers. The calculated d is then converted into a confidence score of a sample belonging to class C using the Sigmoid function.

[0132] Step i: Obtain the difference between the confidence level and the predicted probability of the corrected teacher network, identify the samples that meet the difference threshold as error samples, and obtain the set of error-prone samples.

[0133] Specifically, these confidence levels are calculated along with the modified teacher network prediction probabilities. The differences between the samples are analyzed, and the top N samples with the largest differences are selected as error samples, resulting in the error-prone sample set X. fault .

[0134] Step j: Input the set of error-prone samples into the modified student network and obtain the predicted probability output.

[0135] Specifically, the formulas representing the predicted probability output include:

[0136] O = Softmax(ST(G(X) fault )));

[0137] Where O represents the predicted probability output; Softmax is the smoothing maximization operation used to transform the neural network output into a probability distribution; G is the feature extractor; X... fault This is a set of error-prone samples.

[0138] Step k: Determine the predicted class index for each sample based on the predicted probability output;

[0139] Step 1: Based on the predicted category index, classify the pre-error-prone samples into known category samples and unknown category samples;

[0140] Step m: Determine the distance from known class samples to the known class center and the Euclidean distance from unknown class samples to the unknown class center;

[0141] Step n: Obtain the sum of distances from error-prone samples to their class centers as the loss function.

[0142] Specifically, the formula for the loss function representing the distance from the error-prone sample to its class center includes:

[0143]

[0144] Among them, L dis The loss function is the distance from the error-prone sample to its class center; and Corrected student network output of known and unknown class samples; and These represent the category centers of the known and unknown classes, respectively.

[0145] Step o: Optimize the corrected student network based on the loss function.

[0146] Step 400: Embed the target domain into the student network with the feature input correction to obtain the category prediction result for the target domain.

[0147] In some embodiments, the model uses an optimization problem during training to find the best one-to-one match between the prediction and the ground truth label, thereby optimizing the association between the model output and the actual label. The total loss is expressed as:

[0148] L total =L c +β1L dis +β2L D ;

[0149]

[0150] L D =L d +L LMMD +L con ;

[0151] Among them, L total β1 represents the total loss; β2 and β1 are weighting coefficients used to balance the weights of classification tasks, reflective teaching, and domain adaptation tasks; α represents the loss weighting coefficient.

[0152] Understandably, the open-set cross-domain hyperspectral image classification method based on critical reflective learning provided in the embodiments of this application first extracts features from source domain data and target domain data to obtain source domain embedding features and target domain embedding features. These features are then input into a teacher network and a student network, respectively, to obtain teacher network output and student network output. The dimensions of the teacher network output and the student network output are determined. If the dimension of the teacher network output is smaller than that of the student network output, the dimension of the teacher network output is expanded to match the dimension of the student network output, resulting in a corrected teacher network. Simultaneously, the student network learns the output of the corrected teacher network, resulting in a corrected student network. The target domain embedding features are then input into the corrected student network to obtain the category prediction result for the target domain data. This application introduces the teacher-student concept into the open-set cross-domain hyperspectral image classification task, overcoming the limitation of open-set classifiers relying on thresholds. Through teacher network guidance, the student network masters the decision boundary of known / unknown categories.

[0153] Accordingly, please refer to Figure 8 and Figure 9 , Figure 8 The module connection diagram of the open-set cross-domain hyperspectral image classification system based on critical reflective learning provided in the embodiments of this application is as follows:

[0154] Feature extraction module 10 is used to extract features from source domain data and target domain data respectively, and obtain source domain embedding features and target domain embedding features;

[0155] The prediction output module 20 is used to input the source domain embedding features and the target domain embedding features into the teacher network to obtain the teacher network output; and to input the source domain embedding features and the target domain embedding features into the student network to obtain the student network output.

[0156] Output correction module 30 is used to expand the dimension of the teacher network output to make the dimension of the teacher network output equal to the dimension of the student network output in response to the teacher network output dimension being smaller than the student network output dimension, thereby obtaining the corrected teacher network; and to control the student network to learn the output result of the corrected teacher network, thereby obtaining the corrected student network.

[0157] The category prediction module 40 is used to embed the target domain into the student network with feature input correction to obtain the category prediction results for the target domain data.

[0158] In some embodiments, the output correction module 30 is specifically used for:

[0159] The entropy value of the probability distribution output by the teacher network after classifying the embedded features of the target domain is obtained based on the entropy-assisted model.

[0160] The decision factor is obtained based on the entropy value;

[0161] The decision factor is used as a dimension to expand the predicted output of the teacher network, thus expanding the output dimension of the teacher network.

[0162] In some embodiments, the output correction module 30 is specifically used for:

[0163] Based on the decision factor and the teacher network output, the adjusted probability distribution of the known categories is obtained by element-wise multiplication;

[0164] The output dimension of the teacher network is expanded based on the adjusted probability distribution of the decision factor and known categories, and a new output distribution of the teacher network is constructed:

[0165] q(y i |x t ) = p known (y i |x t :ω), i=1,2,…,K;

[0166] Where q(y) i |x t ) represents the new output distribution of the teacher network; p known (y i |x t )=p(y i |x t )⊙(1-ω),i=1,2,…,K is the adjusted probability distribution of the known categories, p(yi |x t The target domain data x output by the Teacher Network (TE) t The probability of belonging to a known category i is used to characterize the teacher network output, ⊙ represents the element-wise product, and ω is the decision factor used to characterize the target domain data x. t The probability of belonging to an unknown category, 1-ω, is used to characterize the target domain data x. t The probability of belonging to a known category, where i is the known category of the data, i = 1, 2, ..., K, and K is the number of known categories.

[0167] In some embodiments, the output correction module 30 is specifically used for:

[0168] Obtain the minimized output of the student network;

[0169] The student network is controlled to learn based on minimizing the KL divergence between the output and the output of the corrected teacher network to obtain the corrected student network.

[0170] In some embodiments, the open-set cross-domain hyperspectral image classification system based on critical reflective learning is specifically used for:

[0171] Obtain known category data from the prediction results of the target domain in the corrected student network output, and perform subdomain alignment with the known categories in the source domain;

[0172] Determine the similarity between embedded features in the source domain, and distinguish between unknown category data in the target domain and known category data in the source domain based on the similarity.

[0173] In some embodiments, the open-set cross-domain hyperspectral image classification system based on critical reflective learning is specifically used for:

[0174] Align the source and target domains as a whole through domain adversarial analysis;

[0175] Based on the data category, the source domain data and target domain data are divided into multiple subdomains respectively. The number of subdomains in the source domain data, the number of subdomains in the target domain data, and the category of the samples are the same.

[0176] The predicted results of the target domain data are output using the modified student network. The known category samples are then filtered out and aligned with the known categories of the source domain data using LMMD subdomain alignment.

[0177] In some embodiments, the open-set cross-domain hyperspectral image classification system based on critical reflective learning is specifically used for:

[0178] The similarity matrix between domain-invariant features of the source domain is determined based on the Gaussian kernel function.

[0179] Determine the contrast loss function based on the similarity matrix;

[0180] The contrastive loss function is used to distinguish between unknown data in the target domain and known data in the source domain.

[0181] In some embodiments, the open-set cross-domain hyperspectral image classification system based on critical reflective learning is specifically used for:

[0182] Based on the average entropy value, the target domain data is divided into known categories and unknown categories, and the centers of known categories and unknown categories are determined.

[0183] Obtain the Euclidean distance from each sample in the target domain data to the known class center and the unknown class center, and concatenate them to obtain the distance set from each sample to the class center. Then, convert the distance set into the confidence score of the corresponding class through the activation function.

[0184] Obtain the difference between the confidence level and the predicted probability of the corrected teacher network, identify the samples that meet the difference threshold as error samples, and obtain the set of error-prone samples;

[0185] Input the set of error-prone samples into the modified student network and obtain the predicted probability output;

[0186] The predicted category index for each sample is determined based on the predicted probability output;

[0187] Based on the predicted category index, the pre-error-prone samples are divided into known category samples and unknown category samples;

[0188] Determine the distance from known class samples to known class centers and the Euclidean distance from unknown class samples to unknown class centers;

[0189] The loss function is the sum of the distances from error-prone samples to their class centers.

[0190] The student network is optimized and corrected based on the loss function.

[0191] This application has provided a detailed description of an open-set cross-domain hyperspectral image classification method and system based on critical reflective learning, as provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An open-set cross-domain hyperspectral image classification method based on critical reflective learning, characterized in that, include: Feature extraction is performed on the source domain data and the target domain data respectively to obtain the source domain embedding features and the target domain embedding features; The source domain embedding features and the target domain embedding features are input into the teacher network to obtain the teacher network output; the source domain embedding features and the target domain embedding features are input into the student network to obtain the student network output. In response to the fact that the dimension of the teacher network output is smaller than the dimension of the student network output, the dimension of the teacher network output is expanded so that the dimension of the teacher network output is equal to the dimension of the student network output, thereby obtaining a modified teacher network; the student network is then controlled to learn the output results of the modified teacher network, thereby obtaining a modified student network. The steps to expand the dimensions of the teacher network output include: The entropy value of the probability distribution output by the teacher network after classifying the embedded features of the target domain is obtained based on the entropy-assisted model. The determination factor is obtained based on the entropy value; the characterization formula of the determination factor includes: Wherein, ω is the determination factor, used to characterize the target domain data x. t The probability of belonging to an unknown category; H(TE(Z(x) t The teacher network TE embeds the feature Z(x) into the target domain. t The entropy value of the probability distribution output after classification: Where i represents the known categories of the data, i = 1, 2, ..., K, and K is the number of known categories; p(y i |x t The target domain data x output by the teacher network TE is... t The probability of belonging to a known category i; y i The predicted label for the known category i; The decision factor is used as a dimension to expand the predicted output of the teacher network, thereby expanding the output dimension of the teacher network; the step of expanding the output dimension of the teacher network includes: Based on the determination factor and the teacher network output, the adjusted probability distribution of the known categories is obtained by element-wise multiplication; The output dimension of the teacher network is expanded based on the adjusted probability distribution of the decision factor and the known categories, thereby constructing a new output distribution for the teacher network: q(y i |x t )=p known (y i |x t :ω),i=1,2,…,K; Where q(y) i |x t ) represents the new output distribution of the teacher network; p known (y i |x t )=p(y i |x t )⊙(1-ω),i=1,2,…,K is the adjusted probability distribution of the known categories, p(y i |x t The target domain data x output by the teacher network TE is... t The probability of belonging to a known category i is used to characterize the teacher network output, ⊙ is the element-wise product, and ω is the decision factor used to characterize the target domain data x. t The probability of belonging to an unknown category, 1-ω, is used to characterize the target domain data x. t The probability of belonging to a known category, where i is the known category of the data, i = 1, 2, ..., K, and K is the number of known categories; The target domain embedding features are input into the modified student network to obtain the category prediction results for the target domain data.

2. The open-set cross-domain hyperspectral image classification method based on critical reflective learning according to claim 1, characterized in that, The step of controlling the student network to learn the output of the modified teacher network and obtaining the modified student network includes: Obtain the minimized output of the student network; The modified student network is obtained by controlling the student network to learn based on the KL divergence between the minimized output and the output of the modified teacher network.

3. The open-set cross-domain hyperspectral image classification method based on critical reflective learning according to claim 1, characterized in that, Before inputting the target domain embedding features into the modified student network, the open-set cross-domain hyperspectral image classification method based on critical reflective learning further includes: Obtain the known category data from the prediction results of the target domain output by the corrected student network, and perform subdomain alignment with the known categories in the source domain data; Determine the similarity between source domain embedded features, and distinguish between unknown category data in the target domain data and known category data in the source domain data based on the similarity.

4. The open-set cross-domain hyperspectral image classification method based on critical reflective learning according to claim 3, characterized in that, The step of obtaining known category data from the prediction results of the target domain output by the corrected student network and aligning it with the known categories in the source domain data includes: Align the source and target domains as a whole through domain adversarial analysis; Based on the data category, the source domain data and the target domain data are divided into multiple subdomains, and the number of subdomains in the source domain data, the number of subdomains in the target domain data, and the data category are the same. The modified student network outputs the prediction results of the target domain data, filters out known category samples, and performs LMMD subdomain alignment with the known categories of the source domain data.

5. The open-set cross-domain hyperspectral image classification method based on critical reflective learning according to claim 3, characterized in that, The step of determining the similarity between source domain embedded features and distinguishing unknown category data in the target domain and known category data in the source domain data based on the similarity includes: The similarity matrix between the domain-invariant features of the source domain data is determined based on the Gaussian kernel function. The contrast loss function is determined based on the similarity matrix; The contrastive loss function is used to determine the distinction between unknown data in the target domain and known data in the source domain.

6. The open-set cross-domain hyperspectral image classification method based on critical reflective learning according to claim 1, characterized in that, It also includes controlling the modified student network to identify error-prone samples from the modified teacher network, and learning from the error-prone samples to optimize the modified student network: Based on the average value of the entropy, the target domain data is divided into known categories and unknown categories, and the centers of the known categories and unknown categories are determined. Obtain the Euclidean distance from each sample in the target domain data to the known class center and the unknown class center, and concatenate them to obtain the distance set from the sample to each class center. Then, convert the distance set into the confidence score of the corresponding class through an activation function. Obtain the difference between the confidence level and the predicted probability of the corrected teacher network, determine the samples that meet the difference threshold as error samples, and obtain the set of error-prone samples; The set of error-prone samples is input into the modified student network to obtain the predicted probability output. The predicted category index for each sample is determined based on the predicted probability output; Based on the predicted category index, the error-prone samples are divided into known category samples and unknown category samples; Determine the distance from the known class sample to the known class center and the Euclidean distance from the unknown class sample to the unknown class center; The loss function is the sum of the distances from the error-prone samples to their class centers. The modified student network is optimized based on the loss function.

7. An open-set cross-domain hyperspectral image classification system based on critical reflective learning, comprising: The feature extraction module (10) is used to extract features from the source domain data and the target domain data respectively, and obtain source domain embedding features and target domain embedding features; The prediction output module (20) is used to input the source domain embedding features and the target domain embedding features into the teacher network to obtain the teacher network output; and to input the source domain embedding features and the target domain embedding features into the student network to obtain the student network output. Output correction module (30) is used to expand the dimension of the teacher network output in response to the fact that the dimension of the teacher network output is smaller than the dimension of the student network output, so that the dimension of the teacher network output is equal to the dimension of the student network output, and obtain a corrected teacher network; control the student network to learn the output result of the corrected teacher network, and obtain a corrected student network. The steps to expand the dimensions of the teacher network output include: The entropy value of the probability distribution output by the teacher network after classifying the embedded features of the target domain is obtained based on the entropy-assisted model. The determination factor is obtained based on the entropy value; the characterization formula of the determination factor includes: Wherein, ω is the determination factor, used to characterize the target domain data x. t The probability of belonging to an unknown category; H(TE(Z(x) t The teacher network TE embeds the feature Z(x) into the target domain. t The entropy value of the probability distribution output after classification: Where i represents the known categories of the data, i = 1, 2, ..., K, and K is the number of known categories; p(y i |x t The target domain data x output by the teacher network TE is... t The probability of belonging to a known category i; y i The predicted label for the known category i; The decision factor is used as a dimension to expand the predicted output of the teacher network, thereby expanding the output dimension of the teacher network; the step of expanding the output dimension of the teacher network includes: Based on the determination factor and the teacher network output, the adjusted probability distribution of the known categories is obtained by element-wise multiplication; The output dimension of the teacher network is expanded based on the adjusted probability distribution of the decision factor and the known categories, thereby constructing a new output distribution for the teacher network: q(y i |x t )=p known (y i |x t :ω),i=1,2,…,K; Where q(y) i |x t ) represents the new output distribution of the teacher network; p known (y i |x t )=p(y i |x t )⊙(1-ω),i=1,2,…,K is the adjusted probability distribution of the known categories, p(y i |x t The target domain data x output by the teacher network TE is... t The probability of belonging to a known category i is used to characterize the teacher network output, ⊙ is the element-wise product, and ω is the decision factor used to characterize the target domain data x. t The probability of belonging to an unknown category, 1-ω, is used to characterize the target domain data x. t The probability of belonging to a known category, where i is the known category of the data, i = 1, 2, ..., K, and K is the number of known categories; The category prediction module (40) is used to input the target domain embedding features into the modified student network to obtain the category prediction results for the target domain data.

Images