Hyperspectral image partial label learning method and system based on twin reconstruction network

Abstract

The invention relates to a hyperspectral image partial label learning method and system based on a twin reconstruction network, and belongs to the technical field of image recognition. The method comprises the following steps: S1, preprocessing a hyperspectral image; S2, preprocessing data; s3, inputting the training sample into a twin reconstruction network for representation learning; s4, label disambiguation: representing a classifier of the input network according to the sample features obtained in the S3, and performing label disambiguation according to the output of the classifier; s5, calculating overall loss and updating model parameters; s6, after training is completed, the test samples are classified, and a test result is obtained. The performance of the method in a hyperspectral image classification task is better than that of other partial label algorithms, and the method can effectively train a model under the condition that supervision information is limited.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and in particular relates to a hyperspectral image partial label learning method and system based on a twin reconstruction network. Background Art

[0002] Hyperspectral imaging is a cutting-edge technology that can capture rich spectral information about objects. It can obtain hyperspectral images with high spectral resolution and multiple bands, and then obtain almost continuous spectral curves of objects. It can select or extract specific bands as needed to highlight the characteristics of target objects. Therefore, hyperspectral images are widely used in environmental detection, military reconnaissance, and food safety analysis. In order to tap the practical application potential of hyperspectral image data, people have explored data processing technologies including hyperspectral image target detection, hyperspectral image unmixing, and hyperspectral image classification. As one of the most critical technologies among various hyperspectral image processing technologies, the purpose of hyperspectral image classification is to identify the specific object type of each pixel in the image. Benefiting from the development of deep learning technology, it plays an important role in most of the above applications.

[0003] The hyperspectral image classification method based on deep learning is essentially a data-driven learning method, so a large number of labeled samples are required to supervise the model training process to achieve satisfactory classification performance. Since hyperspectral image classification is a pixel-level classification task, hyperspectral image annotation requires detailed annotation of pixels in the image, which makes it difficult and expensive to collect a large amount of annotated hyperspectral data. The reason lies in the label ambiguity problem in hyperspectral images: in some areas of hyperspectral images, the inter-class similarity of pixels is high but the intra-class similarity is low. In addition, the annotators are not very familiar with the spectral characteristics of certain types of objects, which makes it difficult for annotators to identify the true category labels of such pixels. In this case, the annotators tend to assign labels that seem to be true labels to pixels. These labels constitute the candidate label set and this data is called partially labeled data. At the same time, for this type of data, researchers have proposed a partial label learning (PLL) framework to reduce the model's labeling requirements for data, thereby alleviating labeling pressure and reducing labeling costs.

[0004] Partial label learning is a weakly supervised learning framework that allows training samples to be associated with a candidate label set consisting of multiple selected labels, of which there is only one true label and the others are false positive labels. The core of PLL is how to identify the true label from the candidate label set, that is, label disambiguation, and then provide effective supervision information for the training process. In recent years, partial label methods based on deep learning have been widely used. These methods achieve label disambiguation by treating the true label as a hidden variable and continuously optimizing it during the model training process. However, most of the existing methods are designed for ordinary natural images, and rarely consider the characteristics of hyperspectral images, so they are difficult to be directly applied to hyperspectral images. Therefore, in the field of hyperspectral image classification, how to design a good partial label learning framework based on the characteristics of hyperspectral images is the key to reducing its annotation pressure and reducing annotation costs. Summary of the invention

[0005] Aiming at the data characteristics of hyperspectral images, the present invention first provides a hyperspectral image partial label learning method based on a twin reconstruction network, adopts a coherent framework combining a reconstruction network and a twin network to perform spatial-spectral feature extraction and high-quality feature representation; adopts a simplified cross-attention mechanism to establish the interaction between spatial feature representation and spectral feature representation; adopts a new maximum matching fusion mechanism, that is, using the semantic representation of the label that best matches the feature representation to enhance the reliability of the feature representation; based on the setting that the gap between the predicted probability of the potential true label and the remaining labels should be as large as possible, a new maximum difference loss is adopted to guide the label disambiguation and model training process. This method can better capture the spatial-spectral information in hyperspectral images, can provide high-quality and reliable feature representation for label disambiguation, and has advantages over other partial label learning methods in label disambiguation.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A hyperspectral image partial label learning method based on a twin reconstruction network includes the following steps: S1: hyperspectral image acquisition, obtaining a target scene hyperspectral image (HSI) I∈R containing c types of ground objects h×w×b , where h and w are the sizes of the images, and b is the spectral dimension; S2: Data preprocessing, preprocess the hyperspectral image to obtain the data form for spectral and spatial information extraction, and divide the training sample data and test sample data; S3: Input the training samples into the twin reconstruction network for representation learning; S4: Label disambiguation, input the classifier of the network based on the sample feature representation obtained in S3, and perform label disambiguation based on the classifier output; S5: Calculate the overall loss and update the model parameters; S6: After training, classify the test samples to obtain the test results.

[0008] Further, in step S2, the hyperspectral image is preprocessed, specifically including: S11, data dimension reduction and blocking, using the principal component analysis method to reduce the spectral dimension from b to l, and then using a window of size s×s with each pixel as the center to divide it into data blocks for spatial information extraction (spatial data), and the original spectrum of the central pixel is used for spectral information extraction (spectral data); S12, data partitioning, using the original spectrum of each pixel and the data block after dimension reduction as model input, and dividing them into training set and test set according to the ratio of 1:99; S13, candidate label confidence vector construction, firstly, the non-real label of the pixel is randomly converted into a candidate label according to a certain threshold R, and then the candidate label vector y is generated for the training sample according to the one-hot encoding method (the value of the corresponding candidate label position is 1, otherwise it is 0), and finally the non-zero elements are normalized to obtain the initial candidate label confidence vector of the pixel.

[0009] Further, in step S3, feature extraction is performed on the training sample; first, before feature extraction is performed on the spectral dimension, the one-dimensional original spectral vector I of the pixel is transformed into i ∈R b Mapping to the two-dimensional representation I′ i ∈R b×d , thereby embedding each spectrum into a more expressive d-dimensional feature representation; and then using the long short-term memory network as the spectral encoding layer to further process I′ i ={s1, s2, ..., s l}∈R b×d , so as to obtain long-distance spectral information, that is, when the time step is p, the hidden state can be updated with the help of input and p-1 step output. The specific formula is as follows:

[0010] Spe p =f ReLU (f LSTM (Spe p-1 ,s p ))

[0011] where s p is the embedded feature vector of the pth spectrum in the pixel, Spe p represents the characteristic representation of the pth spectrum after spectral encoding (Spe is the overall spectral characteristic representation), f LSTM (·) and f ReLU (·) denote the long and short memory networks and activation functions, respectively.

[0012] Furthermore, for each pixel as the center of the data block h i ∈R s×s×1 Perform spatial feature extraction.i Downsampling is performed in the spatial dimension to capture the spatial feature representation of the pixel, Spa. The specific formula is as follows:

[0013]

[0014] in is a convolutional network with a kernel size of k×k, f pool (·) is the pooling operation process.

[0015] Furthermore, after spectral feature extraction and spatial feature extraction, a semantic interaction is established between the output spectral feature representation Spe and the spatial feature representation Spa through the cross-attention mechanism, thereby capturing more valuable information in the hyperspectral image. The cross-attention mechanism first enhances the features of Spe and Spa respectively through two linear fully connected layers. The specific formula is as follows:

[0016] Spe′=Spe×W1

[0017] Spa′=Reshape(Spa)×W2

[0018] Where W1 and W2 are the mapping weights of the linear fully connected layer, and Reshap(·) is to reshape the 3D spatial feature representation into a 2D representation like the spectral representation. Then the similarity score matrix between Spe′ and Spa′ is calculated as follows:

[0019]

[0020] Where d is the number of feature representation dimensions, and Softmax(·) is a normalization operation. Finally, the dot products between Spe′ and Spa′ and the similarity score matrix S are calculated respectively, and then added to the corresponding spectral feature representations Spe and Spa respectively to obtain the feature representation after the space-spectrum interaction. The specific formula is as follows:

[0021] Spe=Spe+S×Spa

[0022] Spa=Spa+Reshape(S T ×Spe)

[0023] Furthermore, the feature representations Spe and Spa after spatial-spectral interaction are used as the input of the second spectral feature extraction and spatial extraction respectively, and their outputs are then subjected to spatial-spectral interaction through cross-attention. Finally, the feature representations after the second spatial-spectral interaction are subjected to the last spectral feature extraction and spatial extraction respectively to obtain the final spectral feature representation Spe and spatial feature representation Spa.

[0024] After three spectral feature extractions and spatial feature extractions, the output feature representations Spe and Spa are averaged and concatenated according to the feature dimension to obtain a comprehensive spatial spectrum feature representation. The specific formula is as follows:

[0025] V1 = mean(Spe, dim = 0)

[0026] V2=mean(Flatten(Spa),dim=0)

[0027] V = [V1, V2]

[0028] Flatten(·) is used to flatten the three-dimensional feature space into a two-dimensional feature space, mean(·) is used for feature averaging operation, and “[]” represents the concatenation operation on the channel dimension.

[0029] Furthermore, each label word is encoded through the pre-trained language model (Bert), and then a query table S∈R containing the semantic representation of each label in the label space is obtained. l×d , that is, the mth row S of S m Represents the d-dimensional semantic representation vector of the mth label. Then the feature recognition is enhanced through the maximum matching fusion mechanism. First, the semantic similarity between the spatial spectrum feature representation V and S is calculated, and then the label semantic representation with the greatest similarity is regarded as the potential true semantic representation of the pixel, and it is weighted and summed with V to obtain a high-quality feature representation that enables the classifier to more easily identify the potential true label. The specific formula is as follows:

[0030] M=(V×(FC(S)) T )

[0031] m=argmax(M)

[0032] V=w×V+(1-w)×S m

[0033] Where w is a weight coefficient between 0 and 1.

[0034] Furthermore, based on the encoding and decoding mode of the reconstructed network mode, the spectral decoder and the spatial decoder are designed respectively, and the spectral feature representation Spe and the spatial feature representation Spe output after three feature extractions are restored to the original input form, and then the consistency constraint between the original input and the decoded output is established, thereby enhancing the feature representation ability in the form of self-supervision to a certain extent. The specific formula is as follows:

[0035]

[0036] where f tanh (.) is the tanh activation function, l mseis the consistency constraint loss.

[0037] Furthermore, in order to further improve the robustness of spectral feature extraction and spatial feature extraction in feature representation, the spectral input I i and spatial input I′ i Add random Gaussian noise and then input it into the twin spatial spectrum encoder that shares weights with the above encoding process to obtain the spatial spectrum feature representation Then V and Input them into the classifier respectively to obtain the predicted probability vector p and Then calculate p and The KL divergence between the two means q is used as the alignment loss of the two. The specific formula is as follows:

[0038]

[0039] Among them l alig is the alignment loss.

[0040] Further, in step S4, the label message is processed according to the classifier output in step S3, that is, the label confidence vector is updated with the label prediction vector p The specific formula is as follows:

[0041]

[0042] where p i is the predicted probability of label i in the prediction vector p. If label i is in the candidate label set (y i =1), then according to p i Update the value of the corresponding label in the candidate label confidence vector otherwise is 0.

[0043] Furthermore, since the larger the predicted difference between the real label and the non-real label is during the model training process, the more the real label can be distinguished, the maximum difference loss is designed to further promote the label disambiguation process, that is, the maximum predicted probability in the prediction result is regarded as the most likely potential real label, and then the difference between it and the predicted probability of other labels is calculated and the power value of e is taken as one of the losses for model update. The specific formula is as follows:

[0044] m=argmax(p)

[0045]

[0046] Where argmax(.) is to find the maximum value in the predicted probability vector to predict the label m, l max is the maximum difference loss value.

[0047] Finally Consider it as weak supervision information to calculate the classification loss (cross entropy loss) of the prediction result p, and calculate The classification loss of the semantic similarity score M in step S3 ensures that the correct label semantic vector is selected. The specific formula is as follows:

[0048]

[0049] Further, in step S5, the alignment loss l alig , consistency loss l mse , maximum difference loss l max And the classification loss l cls Add them according to certain weights to get the final loss value and update the model parameters. The specific formula is as follows:

[0050] loss = l cls +w1l mse +w2l mse +w3l max

[0051] Among them, w1, w2, and w3 are weight values ​​between 0 and 1.

[0052] Finally, in step S6, the predicted samples are input into the trained model to obtain the final test results.

[0053] In addition, the present invention also provides a hyperspectral image partial label learning system based on a twin reconstruction network, in which the method described above is adopted.

[0054] The beneficial effects of the present invention are:

[0055] The twin heterogeneous reconstruction network proposed in the present invention can fully utilize and capture the spatial and spectral information in hyperspectral images, promote the representation ability and robustness of enhanced features, among which the cross-attention module can mine valuable information in space and spectrum. The maximum matching fusion mechanism proposed in the present invention can enhance the category recognition features of spatial-spectral information and promote the classification process. The maximum difference loss proposed in the present invention can promote the label disambiguation process. The experimental results on a hyperspectral image dataset show that the performance of the proposed method is better than the current advanced partial label learning method.

[0056] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0058] Figure 1 is a flow chart of the method of the present invention;

[0059] Figure 2 It is the overall framework diagram of the present invention;

[0060] Figure 3 Diagram of the cross attention mechanism;

[0061] Figure 4 To maximize the matching mechanism diagram;

[0062] Figure 5 It is the PaviaU dataset, where (a) is the true color image and (b) is the ground truth image;

[0063] Figure 6 Visualization results of different methods on the PaviaU dataset, including (a) PiCO, (b) PaPi, (c) ParSE, (d) PL-AGGD, (e) DPCLS, (f) SLAP, (g) the result of the present invention, and (h) the ground truth map; DETAILED DESCRIPTION

[0064] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0065] Figure 1 The present invention provides a method and system for learning partial labels of hyperspectral images based on twin reconstruction networks. Figure 1 As shown in Figure 2, in the data preprocessing stage after obtaining the hyperspectral image, the principal component analysis method is used to reduce the dimension, and the data is cut into pieces to obtain the spatial data, and the original spectrum of the pixel is used as the spectral input. The deep learning network for hyperspectral partial label learning is shown in Figure 2. Figure 2As shown, it can learn rich spatial-spectral features from hyperspectral images and identify true labels from candidate labels, thereby obtaining a hyperspectral image classification model with good performance. The network is mainly composed of an encoding-decoding structure, a twin encoding structure, and a label disambiguation process. First, in order to capture sufficient spatial-spectral features, the present invention utilizes an encoding-decoding structure in the form of a reconstruction network, which consists of a spatial encoding-decoding process and a spectral encoding-decoding process. In order to enhance the representation of the learned features, the present invention utilizes a cross-attention mechanism to realize the interaction between spectral features and spatial features, and designs a maximum matching fusion mechanism to enhance features using label semantic representation. Secondly, in order to further enhance the robustness of feature representation, the present invention designs a twin encoding structure that shares parameter weights with the encoding structure to process noisy spatial data and spectral data, and then inputs the output feature representations of the two encoders into the classifier to obtain label prediction results, and calculates the alignment loss based on the label prediction results. Finally, in order to make the prediction results of the classifier more consistent with the actual situation and achieve more reliable label disambiguation, the present invention designs a maximum gap loss to make the gap between the maximum value and the remaining values ​​in the prediction results as large as possible, thereby making it easier to find the true label.

[0066] Specifically, the technical solution of the present invention includes the following contents:

[0067] 1. Hyperspectral image acquisition: Acquire hyperspectral image (HSI) data I∈R containing c types of ground objects n×w×b , where h and w are the spatial sizes, b is the number of spectral bands, and c is the number of feature type labels;

[0068] 2. Data preprocessing: Preprocess the hyperspectral image. In this method, the data is firstly reduced in dimension and divided into blocks. The principal component analysis method is used to reduce the spectral dimension from b to l. A 13×13 block is used as a sliding window with each pixel as the center to obtain the spatial data for spatial information extraction. The center of the sliding window is used as its label and the original spectrum of the central pixel is used for the spectral data of spectral information extraction. Then the spatial data and spectral data are divided into training set and test set according to the ratio of 1:99. Then, the candidate label confidence vector is constructed for the training sample. The candidate label is obtained by flipping the non-real label of the pixel according to a certain threshold R=0.3. Then, the candidate label vector is generated for the training sample according to the one-hot encoding method (the value of the position corresponding to the candidate label is 1, otherwise it is 0). Finally, the non-zero elements are normalized to obtain the initial candidate label confidence vector of the pixel.

[0069] 3. Input the training samples into the twin reconstruction network for representation learning, such as Figure 2As shown. In the twin reconstruction network, a space spectrum encoder, a spatial decoder, a spectral decoder and a twin space spectrum encoder are constructed for extracting spectral information and spatial information. In the space spectrum encoder, the spectral data and spatial data are first input into the spectral encoding layer and the spatial encoding layer based on the long short-term memory network and the convolutional network respectively, and the spectral feature representation Spe and the spatial feature representation Spa are output. The formula of the encoding operation is as follows:

[0070] Spe p =f ReLU (f LSTM (Spe p-1 ,s p ))

[0071]

[0072] 4. In the process of spatial spectrum encoding, the feature interaction between spectral feature representation and spatial feature representation is established based on the cross-attention mechanism to obtain more valuable information, such as Figure 3 As shown, the interactive features are added to the spectral feature representation and the spatial feature representation, and used as the input of the next spectral encoding layer and the spatial encoding layer respectively. The formula of the cross attention mechanism is as follows:

[0073] Spe′=Spe×W1

[0074] Spa′=Reshape(Spa)×W2

[0075]

[0076] Spe=Spe+S×Spa

[0077] Spa=Spa+Reshape(S T ×Spe)

[0078] Where W1 and W2 are the mapping weights of the linear fully connected layer, Reshap(·) is to reshape the 3D spatial feature representation into a 2D representation like the spectral representation, and d is the dimension of the feature.

[0079] 5. Use the maximum matching mechanism in the last layer of spectral encoding, such as Figure 4 First, each label word is encoded through the pre-trained language model (Bert), and then the query table S∈R containing the semantic representation of each label in the label space is obtained. lxd, then the outputs of the last spectral coding layer and the spatial coding layer are flattened and averaged, and then the two are concatenated according to the feature dimension to obtain a comprehensive spatial spectrum feature representation V. The semantic representation of the label most similar to V is weighted and summed with it through the maximum matching mechanism to obtain the final semantic representation. The formula of the maximum matching mechanism is as follows:

[0080] M=(V×(FC(S)) T

[0081] m=argmax(M)

[0082] V=w×V+(1-w)×S m

[0083] 6. After spectral information extraction and spatial information extraction, the spectral feature representation Spe and spatial feature representation Spe output by the last spectral coding layer and spatial coding layer are restored to the original input form based on the reconstruction network, and then the consistency constraint between the original input and the decoded output is established. The formulas for the decoding process and feature consistency constraint are as follows:

[0084]

[0085] where f tanh (.) is the tanh activation function, l mse is the consistency constraint loss.

[0086] 7. In order to further enhance the robustness of feature representation, for the noisy spectral data and spatial data, a twin spatial spectrum encoder that shares weights with the spatial spectrum encoder is designed to obtain the spatial spectrum feature representation Then V and Input them into the classifier respectively to obtain the predicted probability vector p and Then calculate p and The KL divergence between the two means q is used as the alignment loss of the two. The specific formula is as follows:

[0087]

[0088] Among them l alig is the alignment loss.

[0089] 8. After classifying the predictions, use the predicted probabilities to disambiguate the labels, that is, use the label prediction vector p to update the label confidence vector The formula for label disambiguation is as follows:

[0090]

[0091] where p iis the predicted probability of label i in the prediction vector p. If label i is in the candidate label set (y i =1), then according to p i Update the value of the corresponding label in the candidate label confidence vector otherwise is 0.

[0092] 9. For the predicted label vector p, the maximum difference loss is designed to measure the gap between the maximum value in the predicted probability and other predicted values ​​to further enhance the label disambiguation process. First, find the maximum label prediction probability value p in p m , subtract it from the predicted values ​​of the remaining labels and raise it to the power, the formula is as follows:

[0093]

[0094] Finally, the label confidence vector Treat it as weak supervision information to calculate the classification loss (cross entropy loss) of the prediction result p, and The classification loss is calculated with the label semantic similarity score M in step 5 to ensure that the correct label semantics is selected. The specific formula is as follows:

[0095]

[0096] 10. Calculate the overall loss and update the model parameters. Add the above calculated losses according to certain weights to get the overall loss of the model, and update the model parameters by calculating the model update gradient. The formula for the overall loss is as follows:

[0097] loss = l cls +w1l mse +w2l mse +w3l max

[0098] Among them, w1, W2, and w3 are weight values ​​between 0 and 1. After the model training is completed, the test set is input into the trained network for label prediction and the predicted results are output.

[0099] like Figure 4 This is an open source hyperspectral classification dataset PaviaU's specific ground object type distribution map. Figure 5This is the experimental result of the maximum difference loss guided twin reconstruction network on the PavaiU dataset described in the present invention. It can be seen that the types of objects are well classified. The classification effect of the present invention can be further illustrated by comparative experiments. The method of the present invention and other existing partial label methods SLAP, PL-AGGD, PiCO, PaPi, ParSE, and DPCLS methods are compared on the Pavia University dataset to calculate the overall accuracy Overall Accuracy (OA), Kappa coefficient (KC), Average Accuracy, and classification accuracy of each category, where all indicators are the larger the better. Table 1 gives the values ​​of various indicators of the classification results of different methods:

[0100] Table 1 Index values ​​of classification results of different methods:

[0101]

[0102] It can be seen that the method of the present invention achieves the best accuracy on the data set with 1% of training samples. Figure 6 The visual classification results of the above methods are given, and it can be seen that the performance of the method described in the present invention is better than other partial label methods.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A hyperspectral image partial label learning method based on a twin reconstruction network, characterized by: The following steps are involved: S1: Hyperspectral data acquisition; S2: data preprocessing; S3: Input the training samples into the twin reconstruction network for representation learning; S4: Label disambiguation; S5: Calculate the overall loss and update the model; S6: After training is completed, classify the test samples to obtain prediction results.

2. The hyperspectral image partial label learning method based on the twin reconstruction network according to claim 1 is characterized by: In step S1, a target scene hyperspectral image (HSI) I∈R containing c types of ground objects is obtained. h×w×b , where h and w are the dimensions of the image, and b is the spectral dimension.

3. The hyperspectral image partial label learning method and system based on twin reconstruction network according to claim 2 is characterized by: In step S2, the acquired hyperspectral image is preprocessed and divided. A 13×13 window is used to divide each pixel of the hyperspectral image processed by principal component analysis into blocks to obtain corresponding spatial data. Then, the original spectrum of the corresponding central pixel is used as the spectral data and a candidate label vector is assigned to it. The two types of data are divided into training set and test set in a ratio of 1:99 for model training and testing.

4. The hyperspectral image partial label learning method based on twin reconstruction network according to claim 3 is characterized by: In step S3, the twin reconstruction network includes a space spectrum encoder for extracting spectral information and spatial information, a spatial decoder, a spectral decoder and a twin space spectrum encoder. In the space spectrum encoder, a spectral encoding layer and a spatial encoding layer for spectral data and spatial data are constructed based on a long short-term memory network and a convolutional network, respectively, and a spectral feature representation and a spatial feature representation are output. The formula of the encoding operation is as follows: Spe p =f ReLU (f LSTM (Spe p-1 ,s p )) where s p is the embedded feature vector of the pth spectrum in the spectral data, Spe p represents the characteristic representation of the pth spectrum after spectral encoding, h i is spatial data, Spe and Spa refer to spectral feature representation and spatial feature representation respectively, f LSTM is the long short-term memory network function, is the convolution function with kernel k×k, f pool is the pooling operation, f ReLU is the ReLU activation function.

5. The hyperspectral image partial label learning method based on twin reconstruction network according to claim 4 is characterized by: In the process of spatial-spectral encoding, a semantic interaction is established between the output spectral feature representation Spe and the spatial feature representation Spa through the cross-attention mechanism, thereby capturing more valuable information in the hyperspectral image. The formula of the cross-attention mechanism is as follows: Spe=Spe+S×Spa Spa=Spa+Reshape(S T ×Spe) Where Spe' and Spa' refer to the feature representations after fully connecting Spe and Spa respectively.

6. The hyperspectral image partial label learning method based on twin reconstruction network according to claim 4 is characterized by: In the last layer of the spatial spectral encoder, a maximum matching mechanism is constructed to encode each label word through the pre-trained language model (Bert), and then obtain the query table S∈R containing the semantic representation of each label in the label space. l×d , then the outputs of the last spectral coding layer and the spatial coding layer are flattened and averaged, and then the two are concatenated according to the feature dimension to obtain a comprehensive spatial spectrum feature representation V. Finally, the label semantic representation most similar to V is weighted and summed with it through the maximum matching mechanism to obtain an enhanced semantic representation. The operation formula of the maximum matching mechanism is as follows: M=(V×(FC(S)) T ) m=argmax(M), V=w×V+(1-w)×S m where argmax(·) is the operation that selects the label that best matches.

7. The hyperspectral image partial label learning method based on twin reconstruction network according to claim 4 is characterized by: Based on the encoding and decoding form of the reconstruction network, the spectral decoding and spatial decoding processes are constructed. The spectral feature representation Spe and the spatial feature representation Spe output by the last spectral encoding layer and the spatial encoding layer are restored to the original input form, and then the consistency constraint between the original input and the decoded output is established. The formulas for the decoding process and feature consistency constraints are as follows: where f tanh (·) is the tanh activation function, l mse is the consistency constraint loss.

8. The hyperspectral image partial label learning method based on twin reconstruction network according to claim 4 is characterized by: In order to further enhance the robustness of feature representation, for the noisy spectral data and spatial data, a twin spatial spectral encoder that shares weights with the spatial spectral encoder is designed to obtain the spatial spectral feature representation Then V and Input them into the classifier respectively to obtain the predicted probability vector p and Then calculate p and The KL divergence between the two means q is used as the alignment loss of the two. The specific formula is as follows: Among them l alig is the alignment loss.

9. The hyperspectral image partial label learning method based on twin reconstruction network according to claim 8, characterized in that: It is necessary to perform label disambiguation based on the predicted label probability vector p, that is, to update the label confidence vector with the label prediction vector p The formula for label disambiguation is as follows: Then, in order to further enhance the label disambiguation process, the maximum difference loss is designed to measure the difference between the maximum value in the predicted probability and other predicted values. The larger the difference, the smaller the loss value. First, find the maximum label prediction probability value p in p m , subtract it from the predicted values ​​of the remaining labels and raise it to the power, the formula is as follows: Further label confidence vector is regarded as weak supervision information to calculate the classification loss of the prediction result p, and The classification loss is calculated with the label semantic similarity score M to ensure that the correct label semantics is selected. The specific formula is as follows: Finally, each loss is added according to a certain weight to obtain the overall loss of the model, and the model update gradient is calculated to update the model parameters. The formula for the overall loss is as follows: loss=l cls +w1l mse +w2l mse +w3l max Among them, w1, w2, and w3 are weight values ​​between 0 and 1. After the model training is completed, the test set is input into the trained network for label prediction and the predicted results are output.

10. A hyperspectral image partial label learning system based on a twin reconstruction network, characterized by: The system is configured with a hyperspectral image partial label learning method based on a twin reconstruction network as described in any one of claims 1 to 9.

Images