Training method, device and electronic device for hyperspectral image classification model

By generating pseudo-hyperspectral samples and employing multi-scale convolutional embedding and multi-path Transformer coding techniques, the dependence on labeled samples in the training of hyperspectral image classification models is resolved, achieving efficient image classification.

CN118823516BActive Publication Date: 2024-11-19北京观微科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411292550.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-14
Publication Date
2024-11-19
Estimated Expiration
2044-09-14

AI Technical Summary

Technical Problem

In existing technologies, training hyperspectral image classification models relies on a large number of labeled samples, which makes it difficult to obtain samples and thus difficult to train effectively.

Method used

By acquiring unlabeled and labeled hyperspectral image samples, pseudo-hyperspectral samples are generated using an initial generator. Then, by combining dimensionality reduction processing and multi-scale convolutional embedding, multi-path Transformer encoding and other techniques, the generator and discriminator model parameters are updated, reducing the dependence on labeled samples.

Benefits of technology

By reducing reliance on labeled samples, an efficient hyperspectral image classification model is trained, which can accurately classify hyperspectral images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118823516B_ABST
    Figure CN118823516B_ABST
Patent Text Reader

Abstract

The present application provides a training method, device and electronic device for a hyperspectral image classification model, and relates to the field of image processing technology. The method includes: inputting random noise into an initial generator in an initial hyperspectral image classification model to obtain a pseudo hyperspectral image sample, performing dimensionality reduction processing on each hyperspectral image sample among unlabeled hyperspectral image samples, labeled hyperspectral image samples and pseudo hyperspectral image samples, extracting multiple image blocks of the hyperspectral image sample with each pixel in the hyperspectral image sample after dimensionality reduction as the center, and inputting the multiple image blocks into an initial discriminator in the initial hyperspectral image classification model to obtain a discrimination result of the hyperspectral image sample, and then updating the model parameters of the initial generator and the initial discriminator based on the discrimination results of the unlabeled hyperspectral image sample, the labeled hyperspectral image sample and the pseudo hyperspectral image sample, and obtaining a hyperspectral image classification model, wherein the hyperspectral image classification model includes the updated discriminator, and can train the hyperspectral image classification model while reducing the dependence on a large number of labeled hyperspectral image samples, so that the hyperspectral image can be classified based on the trained hyperspectral image classification model in the future.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a training method, device and electronic equipment for a hyperspectral image classification model. Background Art

[0002] Hyperspectral images are three-dimensional data composed of hundreds of continuous spectral bands, which contain rich spectral information and can extract detailed features of different ground objects. The purpose of hyperspectral image classification is to determine a category for each pixel.

[0003] At present, it is mainly based on supervised learning methods, and the hyperspectral image classification model is trained through a large number of labeled hyperspectral image samples, so that the hyperspectral images can be classified based on the trained hyperspectral image classification model.

[0004] However, the above-mentioned supervised learning method relies on a large number of labeled hyperspectral image samples, which are difficult to obtain and are not conducive to the training of hyperspectral image classification models. Therefore, how to train a hyperspectral image classification model while reducing the dependence on a large number of labeled hyperspectral image samples is a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0005] The present application provides a method, device and electronic device for training a hyperspectral image classification model, which can train the hyperspectral image classification model while reducing the dependence on a large number of labeled hyperspectral image samples, so that the hyperspectral images can be subsequently classified based on the trained hyperspectral image classification model.

[0006] The present application provides a training method for a hyperspectral image classification model, comprising:

[0007] Obtain unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples;

[0008] Inputting random noise into the initial generator in the initial hyperspectral image classification model to obtain pseudo hyperspectral image samples;

[0009] For each hyperspectral image sample among the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo hyperspectral image samples, a dimensionality reduction process is performed on the hyperspectral image samples, and a plurality of image blocks of the hyperspectral image samples are extracted with each pixel in the hyperspectral image samples after dimensionality reduction as the center, and the plurality of image blocks are input into the initial discriminator in the initial hyperspectral image classification model to obtain a discrimination result of the hyperspectral image samples;

[0010] Based on the discrimination results of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo hyperspectral image samples, the model parameters of the initial generator and the initial discriminator are updated to obtain a hyperspectral image classification model, which includes the updated discriminator.

[0011] According to a training method for a hyperspectral image classification model provided by the present application, the initial discriminator includes a multi-scale convolution embedding module, a multi-path Transformer encoding module, a local-global feature fusion module, and a classification recognition module. The multiple image blocks are input into the initial discriminator in the initial hyperspectral image classification model to obtain the discrimination result of the hyperspectral image sample, including:

[0012] Inputting the multiple image blocks into the multi-scale convolution embedding module to obtain multi-scale convolution features of the multiple image blocks;

[0013] Inputting the multi-scale convolutional features of each of the multiple image blocks into the multi-path Transformer encoding module to obtain a plurality of global encoding features and local encoding features of each of the multiple image blocks;

[0014] Inputting the multiple global coding features and the local coding features of the hyperspectral image sample into the local-global feature fusion module to obtain a target fusion feature;

[0015] The target fusion feature is input into the classification and recognition module to obtain the discrimination result of the hyperspectral image sample.

[0016] According to a training method for a hyperspectral image classification model provided by the present application, the multi-scale convolution embedding module includes a first depth convolution unit, a second depth convolution unit, and a third depth convolution unit connected in parallel, and the plurality of image blocks are input into the multi-scale convolution embedding module to obtain multi-scale convolution features of the plurality of image blocks, including:

[0017] Inputting the plurality of image blocks into the first depth convolution unit to obtain first convolution features of the plurality of image blocks;

[0018] Inputting a plurality of image blocks of the hyperspectral image sample into the second deep convolution unit to obtain second convolution features of each of the plurality of image blocks;

[0019] Inputting a plurality of image blocks of the hyperspectral image sample into the third deep convolution unit to obtain third convolution features of each of the plurality of image blocks;

[0020] Among them, the multi-scale convolution feature includes the first convolution feature, the second convolution feature and the third convolution feature.

[0021] According to a training method for a hyperspectral image classification model provided by the present application, the multi-path Transformer encoding module includes a first Transformer encoder connected in series with the first deep convolution unit, a second Transformer encoder connected in series with the second deep convolution unit, and a third Transformer encoder and a local feature convolution unit connected in series with the third deep convolution unit, respectively. The multi-scale convolution features of each of the multiple image blocks are input into the multi-path Transformer encoding module to obtain multiple global encoding features and local encoding features of each of the multiple image blocks, including:

[0022] Inputting the first convolutional features of each of the multiple image blocks into the first Transformer encoder to obtain the first global encoding features of each of the multiple image blocks;

[0023] Inputting the second convolutional features of each of the multiple image blocks into the second Transformer encoder to obtain the second global encoding features of each of the multiple image blocks;

[0024] Inputting the third convolutional features of each of the multiple image blocks into the third Transformer encoder to obtain the third global encoding features of each of the multiple image blocks, and inputting the third convolutional features of each of the multiple image blocks into the local feature convolution unit to obtain the local encoding features of each of the multiple image blocks;

[0025] The multiple global coding features include the first global coding feature, the second global coding feature and the third global coding feature.

[0026] According to a training method for a hyperspectral image classification model provided by the present application, the first Transformer encoder includes a first normalization layer, a multi-head self-attention layer, a second normalization layer and a feedforward network connected in series in sequence, and the first convolutional features of each of the multiple image blocks are input into the first Transformer encoder to obtain the first global coding features of each of the multiple image blocks, including:

[0027] Inputting the first convolutional features of each of the plurality of image blocks into the first normalization layer to obtain the first normalized features of each of the plurality of image blocks;

[0028] Inputting the first normalized features of each of the multiple image blocks into the multi-head self-attention layer to obtain the multi-head attention features of each of the multiple image blocks;

[0029] Inputting the first sum of the first convolutional features and the multi-head attention features of each of the multiple image blocks into a second normalization layer to obtain the second normalized features of each of the multiple image blocks;

[0030] Inputting the second normalized features of each of the multiple image blocks into the feed-forward network to obtain encoding features of each of the multiple image blocks;

[0031] The first global coding feature is the sum of the coding feature and the first sum value.

[0032] According to a training method for a hyperspectral image classification model provided by the present application, the local feature convolution unit includes a first convolution layer, a depth convolution layer, and a second convolution layer connected in series in sequence, and the third convolution features of each of the multiple image blocks are input into the local feature convolution unit to obtain the local coding features of each of the multiple image blocks, including:

[0033] Inputting the third convolutional features of each of the plurality of image blocks into the first convolutional layer to obtain fourth convolutional features of each of the plurality of image blocks;

[0034] Inputting the fourth convolutional features of each of the multiple image blocks into the deep convolutional layer to obtain the deep convolutional features of each of the multiple image blocks;

[0035] The deep convolution features of each of the multiple image blocks are input into the second convolution layer to obtain the local coding features of each of the multiple image blocks.

[0036] According to a training method for a hyperspectral image classification model provided by the present application, the initial generator includes a fully connected layer and a plurality of fourth Transformer encoders connected in series in sequence, and the random noise is input into the initial generator in the initial hyperspectral image classification model to obtain a pseudo hyperspectral image sample, including:

[0037] Inputting the random noise into the fully connected layer to obtain feature information of the random noise;

[0038] The characteristic information of the random noise is input into the plurality of fourth Transformer encoders to obtain the pseudo hyperspectral image samples.

[0039] The present application also provides a training device for a hyperspectral image classification model, comprising:

[0040] An acquisition unit, used for acquiring unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples;

[0041] A first processing unit is used to input random noise into an initial generator in an initial hyperspectral image classification model to obtain a pseudo hyperspectral image sample;

[0042] a second processing unit, for performing dimensionality reduction processing on each of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo hyperspectral image samples, extracting a plurality of image blocks of the hyperspectral image samples with each pixel in the hyperspectral image samples after dimensionality reduction as the center, and inputting the plurality of image blocks into an initial discriminator in the initial hyperspectral image classification model to obtain a discrimination result of the hyperspectral image samples;

[0043] An updating unit is used to update the model parameters of the initial generator and the initial discriminator based on the discrimination results of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo-hyperspectral image samples, so as to obtain a hyperspectral image classification model, wherein the hyperspectral image classification model includes the updated discriminator.

[0044] The present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the training method for the hyperspectral image classification model as described in any one of the above items is implemented.

[0045] The present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the training method for the hyperspectral image classification model as described in any one of the above items is implemented.

[0046] The present application also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the training method of the hyperspectral image classification model as described in any one of the above items is implemented.

[0047] The training method, device and electronic device of the hyperspectral image classification model provided by the present application can first obtain unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples when training the hyperspectral image classification model, input random noise into the initial generator in the initial hyperspectral image classification model to obtain pseudo hyperspectral image samples, perform dimensionality reduction processing on the hyperspectral image samples for each hyperspectral image sample among the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo hyperspectral image samples, extract multiple image blocks of the hyperspectral image samples with each pixel in the hyperspectral image samples after dimensionality reduction as the center, and input the multiple image blocks into the initial discriminator in the initial hyperspectral image classification model to obtain To the discrimination result of the hyperspectral image samples, and then based on the discrimination results of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo-hyperspectral image samples, the model parameters of the initial generator and the initial discriminator are updated to obtain the hyperspectral image classification model. The hyperspectral image classification model includes the updated discriminator. In this way, the pseudo-hyperspectral image samples are obtained through the initial generator, and the unlabeled hyperspectral image samples and the corresponding labeled hyperspectral image samples are combined to jointly train the hyperspectral image classification model. The hyperspectral image classification model can be trained while reducing the dependence on a large number of labeled hyperspectral image samples, so that the hyperspectral image can be classified based on the trained hyperspectral image classification model in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 A schematic diagram of a process flow of a method for training a hyperspectral image classification model provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of the structure of an initial hyperspectral image classification model provided in an embodiment of the present application;

[0051] Figure 3 A schematic diagram of the structure of an initial generator provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of the structure of an initial discriminator provided in an embodiment of the present application;

[0053] Figure 5A schematic flow chart of a method for inputting multiple image blocks of a hyperspectral image sample into an initial discriminator in an initial hyperspectral image classification model to obtain a discrimination result of the hyperspectral image sample provided in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of the structure of another initial discriminator provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of the structure of a multi-path Transformer encoding module provided in an embodiment of the present application;

[0056] Figure 8 A schematic diagram of the structure of a local feature convolution unit provided in an embodiment of the present application;

[0057] Fig. 9 A schematic diagram of the structure of a local-global feature fusion module provided in an embodiment of the present application;

[0058] Fig.10 A schematic diagram of the structure of a training device for a hyperspectral image classification model provided in an embodiment of the present application;

[0059] Fig.11 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0061] In the embodiments of the present application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In the text description of the present application, the character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0062] The technical solution provided by the embodiment of the present application can be adapted to a variety of hyperspectral image classification scenarios. For example, crop classification and yield estimation, mineral resource exploration, urban land planning, ocean monitoring and post-disaster assessment. At present, it is mainly based on supervised learning methods, and a hyperspectral image classification model is trained through a large number of labeled hyperspectral image samples, so that the hyperspectral image can be classified based on the trained hyperspectral image classification model.

[0063] However, the above-mentioned supervised learning method relies on a large number of labeled hyperspectral image samples, which are difficult to obtain and are not conducive to the training of hyperspectral image classification models.

[0064] In order to train a hyperspectral image classification model while reducing the reliance on a large number of labeled hyperspectral image samples, so that the hyperspectral images can be classified based on the trained hyperspectral image classification model, the present application embodiment provides a method for training a hyperspectral image classification model. The following specific embodiments will be used to describe the training method for the hyperspectral image classification model provided by the present application in detail. It can be understood that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0065] Figure 1 A flow chart of a method for training a hyperspectral image classification model provided in an embodiment of the present application, which method can be executed by software and / or hardware devices, for example, see Figure 1 As shown, the training method of the hyperspectral image classification model may include:

[0066] S101: Obtain unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples.

[0067] For example, unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples can be obtained in a variety of ways. For example, unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples sent by other electronic devices can be received; unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples can be obtained by searching from local storage; or unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples can be obtained from a third-party database, etc. The specific settings can be made according to actual needs and are not further limited in the embodiments of the present application.

[0068] S102, inputting random noise into an initial generator in an initial hyperspectral image classification model to obtain a pseudo hyperspectral image sample.

[0069] For example, see Figure 2 As shown, Figure 2A structural schematic diagram of an initial hyperspectral image classification model provided in an embodiment of the present application, the initial hyperspectral image classification model may include an initial generator and an initial discriminator, wherein the initial generator is used to process input random noise to obtain pseudo hyperspectral image samples; the initial discriminator is used to process multiple image blocks corresponding to each hyperspectral image sample among unlabeled hyperspectral image samples, labeled hyperspectral image samples and pseudo hyperspectral image samples, to obtain discrimination results corresponding to each of the unlabeled hyperspectral image samples, labeled hyperspectral image samples and pseudo hyperspectral image samples.

[0070] It can be understood that in the embodiment of the present application, a large number of pseudo-hyperspectral image samples can be obtained through the initial generator, which can not only reduce the dependence on labeled hyperspectral image samples to a certain extent, but also reconstruct the hyperspectral image information from a global perspective, and can better represent the complete distribution of real hyperspectral samples, thereby generating more realistic pseudo-hyperspectral image samples.

[0071] It should be noted that in the embodiment of the present application, there is no order between S101 and S102. S101 can be executed first and then S102; S102 can also be executed first and then S101. Of course, S101 and S102 can also be executed at the same time. Here, the embodiment of the present application is just explained by taking the example of executing S101 first and then S102, but it does not mean that the embodiment of the present application is limited to this.

[0072] After obtaining the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo-hyperspectral image samples respectively, S103 can be executed:

[0073] S103, for each hyperspectral image sample in the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo-hyperspectral image samples, perform dimensionality reduction processing on the hyperspectral image samples, extract multiple image blocks of the hyperspectral image samples with each pixel in the hyperspectral image samples after dimensionality reduction as the center, and input the multiple image blocks into the initial discriminator in the initial hyperspectral image classification model to obtain the discrimination result of the hyperspectral image samples.

[0074] For example, in the embodiment of the present application, the principal component analysis (PCA) algorithm can be used to perform dimensionality reduction processing on the hyperspectral image, and other dimensionality reduction algorithms can also be used to perform dimensionality reduction processing on the hyperspectral image, which can be specifically set according to actual needs.

[0075] After obtaining the hyperspectral image samples after dimensionality reduction, multiple image blocks corresponding to the hyperspectral image can be extracted with each pixel in the hyperspectral image samples after dimensionality reduction as the center, thereby obtaining multiple image blocks.

[0076] For example, in an embodiment of the present application, the real label of each image block may be the label of the central pixel, and the position information of the image block may be the position information of the central pixel, which may be specifically set according to actual needs.

[0077] Based on the above description, after obtaining multiple image blocks corresponding to each hyperspectral image sample in the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo-hyperspectral image samples, the multiple image blocks corresponding to each hyperspectral image sample can be used as the input of the initial discriminator, and the discriminator outputs the discrimination result corresponding to each hyperspectral image sample, and then executes the following S104:

[0078] S104, based on the discrimination results of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo-hyperspectral image samples, the model parameters of the initial generator and the initial discriminator are updated to obtain a hyperspectral image classification model, wherein the hyperspectral image classification model includes the updated discriminator.

[0079] Among them, for unlabeled hyperspectral image samples and pseudo-hyperspectral image samples, the discrimination result can be 0 or 1, 0 represents a false sample, and 1 represents a true sample; for labeled hyperspectral image samples, the discrimination result is a multi-category probability distribution. In addition, in the embodiment of the present application, the label information of the unlabeled hyperspectral image sample can be 1, and the label information of the pseudo-hyperspectral image sample can be 0.

[0080] For example, in the embodiment of the present application, when updating the model parameters of the initial generator and the initial discriminator based on the discrimination results of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo-hyperspectral image samples, the loss function corresponding to the unlabeled hyperspectral image samples can be constructed based on the label information and discrimination results of the unlabeled hyperspectral image samples, the loss function corresponding to the labeled hyperspectral image samples can be constructed based on the label information and discrimination results of the labeled hyperspectral image samples, and the loss function corresponding to the pseudo-hyperspectral image samples can be constructed based on the label information and discrimination results of the pseudo-hyperspectral image samples; and then these loss functions are combined to jointly update the model parameters of the initial generator and the initial discriminator until the updated hyperspectral image classification model converges, or meets the training cutoff condition, such as the number of training times reaches the preset number of times, etc., to obtain the final hyperspectral image classification model. Among them, the value of the preset number of times can be set according to actual needs.

[0081] It can be seen that in the embodiment of the present application, when training the hyperspectral image classification model, unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples can be first obtained, and random noise can be input into the initial generator in the initial hyperspectral image classification model to obtain pseudo hyperspectral image samples. For each hyperspectral image sample among the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo hyperspectral image samples, the hyperspectral image samples are subjected to dimensionality reduction processing, and multiple image blocks of the hyperspectral image samples are extracted with each pixel in the hyperspectral image samples after dimensionality reduction as the center, and the multiple image blocks are input into the initial discriminator in the initial hyperspectral image classification model to obtain hyperspectral image samples. The discrimination results of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo-hyperspectral image samples are used to update the model parameters of the initial generator and the initial discriminator to obtain the hyperspectral image classification model. The hyperspectral image classification model includes the updated discriminator. In this way, the pseudo-hyperspectral image samples are obtained through the initial generator, and the unlabeled hyperspectral image samples and the corresponding labeled hyperspectral image samples are combined to jointly train the hyperspectral image classification model. The hyperspectral image classification model can be trained while reducing the dependence on a large number of labeled hyperspectral image samples, so that the hyperspectral image can be classified based on the trained hyperspectral image classification model in the future.

[0082] based on Figure 1 In the embodiment shown, after the hyperspectral image classification model is trained, classification prediction can be performed on the hyperspectral image to be processed based on the hyperspectral image classification model. Different from the initial hyperspectral image classification model, the trained hyperspectral image classification model only includes the updated discriminator, so as to perform classification prediction on the hyperspectral image through the updated discriminator.

[0083] Based on the above Figure 1 In the embodiment shown, for example, in the embodiment of the present application, the initial generator includes a fully connected layer and a plurality of fourth Transformer encoders connected in series. Taking three fourth Transformer encoders as an example, see Figure 3 As shown, Figure 3 A schematic diagram of the structure of an initial generator provided in an embodiment of the present application.

[0084] Combination Figure 3The initial generator shown in the figure inputs random noise into the initial generator in the initial hyperspectral image classification model. When obtaining a pseudo hyperspectral image sample, the random noise can be first input into the fully connected layer to obtain the characteristic information of the random noise; and then the characteristic information of the random noise is input into multiple fourth Transformer encoders connected in series in sequence to obtain the pseudo hyperspectral image sample. In this way, the initial generator based on Transformer can generate a large number of pseudo hyperspectral image samples, which can not only reduce the dependence on labeled hyperspectral image samples to a certain extent, but also reconstruct the hyperspectral image information from a global perspective, and can better represent the complete distribution of real hyperspectral image samples, thereby generating more realistic pseudo hyperspectral image samples.

[0085] Based on any of the above embodiments, for example, in the embodiment of the present application, the initial discriminator includes a multi-scale convolution embedding module, a multi-path Transformer encoding module, a local-global feature fusion module, and a classification recognition module, which can be seen in Figure 4 As shown, Figure 4 A schematic diagram of the structure of an initial discriminator provided in an embodiment of the present application.

[0086] Combination Figure 4 The initial discriminator shown in FIG. 1 inputs multiple image blocks of the hyperspectral image samples into the initial discriminator in the initial hyperspectral image classification model to obtain the discrimination result of the hyperspectral image samples. The specific process can be seen in FIG. Figure 5 As shown, Figure 5 A schematic flow chart of a method for inputting multiple image blocks of a hyperspectral image sample into an initial discriminator in an initial hyperspectral image classification model to obtain a discrimination result of the hyperspectral image sample is provided in an embodiment of the present application. The method may include:

[0087] S501, inputting multiple image blocks of the hyperspectral image sample into a multi-scale convolution embedding module to obtain multi-scale convolution features of each of the multiple image blocks.

[0088] For example, in the embodiments of the present application, see Figure 6 As shown, Figure 6A structural schematic diagram of another initial discriminator provided for an embodiment of the present application, wherein a multi-scale convolution embedding module in the initial discriminator may include a first deep convolution unit, a second deep convolution unit, and a third deep convolution unit connected in parallel, and when a plurality of image blocks are input into the multi-scale convolution embedding module to obtain multi-scale convolution features of each of the plurality of image blocks, the plurality of image blocks of the hyperspectral image samples may be first input into the first deep convolution unit to obtain first convolution features of each of the plurality of image blocks; the plurality of image blocks of the hyperspectral image samples may be input into the second deep convolution unit to obtain second convolution features of each of the plurality of image blocks; and the plurality of image blocks of the hyperspectral image samples may be input into the third deep convolution unit to obtain third convolution features of each of the plurality of image blocks.

[0089] Among them, the multi-scale convolution feature includes a first convolution feature, a second convolution feature and a third convolution feature.

[0090] For example, in an embodiment of the present application, the first depth convolution unit can be a 7×7 depth convolution, the second depth convolution unit can be a 5×5 depth convolution, and the third depth convolution unit can be a 3×3 depth convolution. The specific settings can be made according to actual needs. Here, the embodiment of the present application only takes these three different scales of depth convolution as examples for illustration, but it does not mean that the embodiment of the present application is limited to this.

[0091] Combination Figure 6 The multi-scale convolution embedding module shown can perform convolution processing on multiple image blocks of the hyperspectral image sample through three depth convolution units of different scales in the multi-scale convolution embedding module to obtain multi-scale convolution features, and after obtaining the multi-scale convolution features, execute the following S502:

[0092] S502: Input the multi-scale convolutional features of the multiple image blocks into a multi-path Transformer encoding module to obtain multiple global encoding features and local encoding features of the multiple image blocks.

[0093] By way of example, in an embodiment of the present application, a multi-path Transformer encoding module includes a first Transformer encoder connected in series with a first depth convolution unit, a second Transformer encoder connected in series with a second depth convolution unit, and a third Transformer encoder and a local feature convolution unit connected in series with a third depth convolution unit, respectively. When the multi-scale convolution features of multiple image blocks are input into the multi-path Transformer encoding module, the first convolution features of the multiple image blocks can be first input into the first Transformer encoder to obtain the first global encoding features of the multiple image blocks; the second convolution features of the multiple image blocks can be input into the second Transformer encoder to obtain the second global encoding features of the multiple image blocks; the third convolution features of the multiple image blocks can be input into the third Transformer encoder to obtain the third global encoding features of the multiple image blocks, and the third convolution features of the multiple image blocks can be input into the local feature convolution unit to obtain the local encoding features of the multiple image blocks.

[0094] The multiple global coding features include a first global coding feature, a second global coding feature and a third global coding feature.

[0095] For example, in the embodiment of the present application, the first Transformer encoder includes a first normalization layer, a multi-head self-attention layer, a second normalization layer and a feedforward network connected in series, as shown in Figure 7 As shown, Figure 7 A structural schematic diagram of a multi-path Transformer encoding module provided in an embodiment of the present application, wherein the first convolution features of each of the multiple image blocks are input into the first Transformer encoder to obtain the first global encoding features of each of the multiple image blocks. When the first convolution features of each of the multiple image blocks are input into the first normalization layer to obtain the first normalized features of each of the multiple image blocks; the first normalized features of each of the multiple image blocks are input into the multi-head self-attention layer to obtain the multi-head attention features of each of the multiple image blocks; the first sum of the first convolution features of each of the multiple image blocks and the multi-head attention features is input into the second normalization layer to obtain the second normalized features of each of the multiple image blocks; the second normalized features of each of the multiple image blocks are input into the feedforward network to obtain the encoding features of each of the multiple image blocks; wherein the first global encoding feature is the sum of the encoding feature and the first sum.

[0096] It should be noted that, in the embodiment of the present application, the structures of the second Transformer encoder and the third Transformer encoder are similar to those described above. Figure 7The structure of the first Transformer encoder shown is similar. Its specific implementation can refer to the relevant description of the first Transformer encoder, and the embodiments of the present application will not be repeated here.

[0097] For example, in the embodiment of the present application, the local feature convolution unit includes a first convolution layer, a depth convolution layer, and a second convolution layer connected in series in sequence, as shown in Figure 8 As shown, Figure 8 A structural schematic diagram of a local feature convolution unit provided in an embodiment of the present application, when the third convolution features of multiple image blocks are respectively input into the local feature convolution unit to obtain the local coding features of the multiple image blocks, the third convolution features of the multiple image blocks can be first input into the first convolution layer to obtain the fourth convolution features of the multiple image blocks; the fourth convolution features of the multiple image blocks are respectively input into the deep convolution layer to obtain the deep convolution features of the multiple image blocks; and the deep convolution features of the multiple image blocks are respectively input into the second convolution layer to obtain the local coding features of the multiple image blocks. In this way, based on the local feature convolution unit, the local coding features of the multiple image blocks can be obtained.

[0098] For example, in an embodiment of the present application, the first convolution layer may be a 1×1 convolution, the depth convolution layer may be a 3×3 convolution, and the second convolution layer may be a 1×1 convolution, which may be specifically set according to actual needs.

[0099] Based on the multi-path Transformer encoding module, the global encoding features and local encoding features of the multiple images of the hyperspectral image sample can be obtained respectively, that is, after the multiple global encoding features and local encoding features of the hyperspectral image sample are obtained, the following S503 can be further performed:

[0100] S503: Input multiple global coding features and local coding features of the hyperspectral image sample into a local-global feature fusion module to obtain a target fusion feature.

[0101] For example, in the embodiment of the present application, the structure of the local-global feature fusion module can be seen in Fig. 9 As shown, Fig. 9 A structural schematic diagram of a local-global feature fusion module provided in an embodiment of the present application, the local-global feature fusion module may include a splicing unit and a convolution layer. For example, the convolution layer may be a 1×1 convolution.

[0102] In this way, based on the local-global feature fusion module, multiple global coding features and local coding features are fused, which can capture global and local information at the same time, and complete the interaction between global and local features. After obtaining the target fusion feature, the following S504 can be executed:

[0103] S504: Input the target fusion features into the classification and recognition module to obtain the discrimination result of the hyperspectral image sample.

[0104] See also Figure 6 As shown in the figure, the classification and recognition module includes a linear layer, Softmax and Sigmoid, wherein the linear layer is used to classify and recognize the target fusion features, and the target fusion features of the unlabeled hyperspectral image samples and the pseudo-hyperspectral image samples are input into the Sigmoid to determine the discrimination results of the unlabeled hyperspectral image samples and the pseudo-hyperspectral image samples, that is, to identify them as true samples or false samples; the labeled hyperspectral image samples are input into the Softmax to determine the discrimination results of the labeled hyperspectral image samples, that is, the probability distribution of multiple categories.

[0105] It can be seen that in the embodiment of the present application, when the discrimination result of the hyperspectral image sample is determined by the initial discriminator, the multiple image blocks of the hyperspectral image sample can be first input into the multi-scale convolution embedding module to obtain the multi-scale convolution features of the multiple image blocks; and the multi-scale convolution features of the multiple image blocks are input into the multi-path Transformer encoding module to obtain the multiple global coding features and local coding features of the multiple image blocks; the multiple global coding features and local coding features of the hyperspectral image sample are input into the local-global feature fusion module to obtain the target fusion feature; and the target fusion feature is input into the classification and recognition module to obtain the discrimination result of the hyperspectral image sample. In this way, by passing through the multi-scale convolution and multi-path Transformer encoding modules in sequence, the global and local information can be captured at the same time, so that when the discrimination result of the hyperspectral image sample is determined based on the fused target fusion feature, the accuracy of the discrimination result can be effectively improved.

[0106] The following is a description of a training device for a hyperspectral image classification model provided in the present application. The training device for a hyperspectral image classification model described below and the hyperspectral image classification method described above can be referred to in correspondence with each other.

[0107] Fig.10 A structural diagram of a training device for a hyperspectral image classification model provided in an embodiment of the present application, for example, see Fig.10 As shown, the training device 100 of the hyperspectral image classification model may include:

[0108] An acquisition unit 1001 is used to acquire unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples;

[0109] The first processing unit 1002 is used to input random noise into an initial generator in an initial hyperspectral image classification model to obtain a pseudo hyperspectral image sample;

[0110] The second processing unit 1003 is used for performing dimensionality reduction processing on each hyperspectral image sample in the unlabeled hyperspectral image sample, the labeled hyperspectral image sample and the pseudo hyperspectral image sample, extracting multiple image blocks of the hyperspectral image sample with each pixel in the hyperspectral image sample after dimensionality reduction as the center, and inputting the multiple image blocks into the initial discriminator in the initial hyperspectral image classification model to obtain the discrimination result of the hyperspectral image sample;

[0111] The updating unit 1004 is used to update the model parameters of the initial generator and the initial discriminator based on the discrimination results of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo-hyperspectral image samples, so as to obtain a hyperspectral image classification model, wherein the hyperspectral image classification model includes the updated discriminator.

[0112] For example, in an embodiment of the present application, the initial discriminator includes a multi-scale convolution embedding module, a multi-path Transformer encoding module, a local-global feature fusion module, and a classification recognition module. The second processing unit 1003 is used to input the multiple image blocks into the initial discriminator in the initial hyperspectral image classification model to obtain the discrimination result of the hyperspectral image sample, including:

[0113] Inputting the multiple image blocks into the multi-scale convolution embedding module to obtain multi-scale convolution features of the multiple image blocks;

[0114] Inputting the multi-scale convolutional features of each of the multiple image blocks into the multi-path Transformer encoding module to obtain a plurality of global encoding features and local encoding features of each of the multiple image blocks;

[0115] Inputting the multiple global coding features and the local coding features of the hyperspectral image sample into the local-global feature fusion module to obtain a target fusion feature;

[0116] The target fusion feature is input into the classification and recognition module to obtain the discrimination result of the hyperspectral image sample.

[0117] For example, in an embodiment of the present application, the multi-scale convolution embedding module includes a first depth convolution unit, a second depth convolution unit, and a third depth convolution unit connected in parallel, and the second processing unit 1003 is used to input the multiple image blocks into the multi-scale convolution embedding module to obtain multi-scale convolution features of each of the multiple image blocks, including:

[0118] Inputting the plurality of image blocks into the first depth convolution unit to obtain first convolution features of the plurality of image blocks;

[0119] Inputting a plurality of image blocks of the hyperspectral image sample into the second deep convolution unit to obtain second convolution features of each of the plurality of image blocks;

[0120] Inputting a plurality of image blocks of the hyperspectral image sample into the third deep convolution unit to obtain third convolution features of each of the plurality of image blocks;

[0121] Among them, the multi-scale convolution feature includes the first convolution feature, the second convolution feature and the third convolution feature.

[0122] For example, in an embodiment of the present application, the multi-path Transformer encoding module includes a first Transformer encoder connected in series with the first depth convolution unit, a second Transformer encoder connected in series with the second depth convolution unit, and a third Transformer encoder and a local feature convolution unit connected in series with the third depth convolution unit, respectively. The second processing unit 1003 is used to input the multi-scale convolution features of each of the multiple image blocks into the multi-path Transformer encoding module to obtain multiple global encoding features and local encoding features of each of the multiple image blocks, including:

[0123] Inputting the first convolutional features of each of the multiple image blocks into the first Transformer encoder to obtain the first global encoding features of each of the multiple image blocks;

[0124] Inputting the second convolutional features of each of the multiple image blocks into the second Transformer encoder to obtain the second global encoding features of each of the multiple image blocks;

[0125] Inputting the third convolutional features of each of the multiple image blocks into the third Transformer encoder to obtain the third global encoding features of each of the multiple image blocks, and inputting the third convolutional features of each of the multiple image blocks into the local feature convolution unit to obtain the local encoding features of each of the multiple image blocks;

[0126] The multiple global coding features include the first global coding feature, the second global coding feature and the third global coding feature.

[0127] For example, in an embodiment of the present application, the first Transformer encoder includes a first normalization layer, a multi-head self-attention layer, a second normalization layer and a feedforward network connected in series, and the second processing unit 1003 is used to input the first convolutional features of each of the multiple image blocks into the first Transformer encoder to obtain the first global encoding features of each of the multiple image blocks, including:

[0128] Inputting the first convolutional features of each of the plurality of image blocks into the first normalization layer to obtain the first normalized features of each of the plurality of image blocks;

[0129] Inputting the first normalized features of each of the multiple image blocks into the multi-head self-attention layer to obtain the multi-head attention features of each of the multiple image blocks;

[0130] Inputting the first sum of the first convolutional features and the multi-head attention features of each of the multiple image blocks into a second normalization layer to obtain the second normalized features of each of the multiple image blocks;

[0131] Inputting the second normalized features of each of the multiple image blocks into the feed-forward network to obtain encoding features of each of the multiple image blocks;

[0132] The first global coding feature is the sum of the coding feature and the first sum value.

[0133] For example, in an embodiment of the present application, the local feature convolution unit includes a first convolution layer, a depth convolution layer, and a second convolution layer connected in series in sequence, and the second processing unit 1003 is used to input the third convolution features of each of the multiple image blocks into the local feature convolution unit to obtain the local coding features of each of the multiple image blocks, including:

[0134] Inputting the third convolutional features of each of the plurality of image blocks into the first convolutional layer to obtain fourth convolutional features of each of the plurality of image blocks;

[0135] Inputting the fourth convolutional features of each of the multiple image blocks into the deep convolutional layer to obtain the deep convolutional features of each of the multiple image blocks;

[0136] The deep convolution features of each of the multiple image blocks are input into the second convolution layer to obtain the local coding features of each of the multiple image blocks.

[0137] For example, in an embodiment of the present application, the initial generator includes a fully connected layer and a plurality of fourth Transformer encoders connected in series in sequence, and the first processing unit 1002 is used to input random noise into the initial generator in the initial hyperspectral image classification model to obtain a pseudo hyperspectral image sample, including:

[0138] Inputting the random noise into the fully connected layer to obtain feature information of the random noise;

[0139] The characteristic information of the random noise is input into the plurality of fourth Transformer encoders to obtain the pseudo hyperspectral image samples.

[0140] The training device 100 for the hyperspectral image classification model provided in the embodiment of the present application can execute the technical solution of the hyperspectral image classification method in any of the above-mentioned embodiments, and its implementation principle and beneficial effects are similar to the implementation principle and beneficial effects of the hyperspectral image classification method. Please refer to the implementation principle and beneficial effects of the hyperspectral image classification method, which will not be repeated here.

[0141] Fig.11 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present application, such as Fig.11 As shown, the electronic device may include: a processor (processor) 1110, a communication interface (Communications Interface) 1120, a memory (memory) 1130 and a communication bus 1140, wherein the processor 1110, the communication interface 1120, and the memory 1130 communicate with each other through the communication bus 1140. The processor 1110 can call the logic instructions in the memory 1130 to execute the training method of the hyperspectral image classification model, which includes: obtaining unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples; inputting random noise into the initial generator in the initial hyperspectral image classification model to obtain pseudo hyperspectral image samples; for each hyperspectral image sample among the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo hyperspectral image samples, performing dimensionality reduction processing on the hyperspectral image samples, extracting multiple image blocks of the hyperspectral image samples with each pixel in the hyperspectral image samples after dimensionality reduction as the center, and inputting the multiple image blocks into the initial discriminator in the initial hyperspectral image classification model to obtain the discrimination result of the hyperspectral image samples; based on the discrimination results of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo hyperspectral image samples, updating the model parameters of the initial generator and the initial discriminator to obtain the hyperspectral image classification model, and the hyperspectral image classification model includes the updated discriminator.

[0142] In addition, the logic instructions in the above-mentioned memory 1130 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0143] On the other hand, the present application also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the above-mentioned training method of the hyperspectral image classification model, the method comprising: obtaining unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples; inputting random noise into an initial generator in the initial hyperspectral image classification model to obtain pseudo hyperspectral image samples; for each hyperspectral sample among the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo hyperspectral image samples, spectral image samples, performing dimensionality reduction processing on the hyperspectral image samples, taking each pixel in the hyperspectral image samples after dimensionality reduction as the center, extracting multiple image blocks of the hyperspectral image samples, and inputting the multiple image blocks into the initial discriminator in the initial hyperspectral image classification model to obtain the discrimination result of the hyperspectral image samples; based on the discrimination results of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo-hyperspectral image samples, the model parameters of the initial generator and the initial discriminator are updated to obtain the hyperspectral image classification model, and the hyperspectral image classification model includes the updated discriminator.

[0144] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the training method of the above-mentioned hyperspectral image classification model, the method comprising: obtaining unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples; inputting random noise into an initial generator in an initial hyperspectral image classification model to obtain pseudo hyperspectral image samples; for each hyperspectral image sample among the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo hyperspectral image samples, performing dimensionality reduction processing on the hyperspectral image samples, extracting multiple image blocks of the hyperspectral image samples with each pixel in the hyperspectral image samples after dimensionality reduction as the center, and inputting the multiple image blocks into an initial discriminator in the initial hyperspectral image classification model to obtain a discrimination result of the hyperspectral image samples; based on the discrimination results of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo hyperspectral image samples, updating the model parameters of the initial generator and the initial discriminator to obtain a hyperspectral image classification model, the hyperspectral image classification model including the updated discriminator.

[0145] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A training method for a hyperspectral image classification model, characterized in that: include: Obtain unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples; Inputting random noise into the initial generator in the initial hyperspectral image classification model to obtain pseudo hyperspectral image samples; For each hyperspectral image sample among the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo hyperspectral image samples, a dimensionality reduction process is performed on the hyperspectral image samples, and a plurality of image blocks of the hyperspectral image samples are extracted with each pixel in the hyperspectral image samples after dimensionality reduction as the center, and the plurality of image blocks are input into the initial discriminator in the initial hyperspectral image classification model to obtain a discrimination result of the hyperspectral image samples; Based on the discrimination results of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo-hyperspectral image samples, the model parameters of the initial generator and the initial discriminator are updated to obtain a hyperspectral image classification model, wherein the hyperspectral image classification model includes the updated discriminator; The initial discriminator includes a multi-scale convolution embedding module, a multi-path Transformer encoding module, a local-global feature fusion module and a classification recognition module. The multiple image blocks are input into the initial discriminator in the initial hyperspectral image classification model to obtain the discrimination result of the hyperspectral image sample, including: Inputting the multiple image blocks into the multi-scale convolution embedding module to obtain multi-scale convolution features of the multiple image blocks; Inputting the multi-scale convolutional features of each of the multiple image blocks into the multi-path Transformer encoding module to obtain a plurality of global encoding features and local encoding features of each of the multiple image blocks; Inputting the multiple global coding features and the local coding features of the hyperspectral image sample into the local-global feature fusion module to obtain a target fusion feature; Inputting the target fusion feature into the classification and recognition module to obtain the discrimination result of the hyperspectral image sample; The multi-scale convolution embedding module includes a first depth convolution unit, a second depth convolution unit, and a third depth convolution unit connected in parallel, and the inputting of the multiple image blocks into the multi-scale convolution embedding module to obtain multi-scale convolution features of the multiple image blocks respectively includes: Inputting the plurality of image blocks into the first depth convolution unit to obtain first convolution features of the plurality of image blocks; Inputting a plurality of image blocks of the hyperspectral image sample into the second deep convolution unit to obtain second convolution features of each of the plurality of image blocks; Inputting a plurality of image blocks of the hyperspectral image sample into the third deep convolution unit to obtain third convolution features of each of the plurality of image blocks; Wherein, the multi-scale convolution feature includes the first convolution feature, the second convolution feature and the third convolution feature; The multi-path Transformer encoding module includes a first Transformer encoder connected in series with the first depth convolution unit, a second Transformer encoder connected in series with the second depth convolution unit, and a third Transformer encoder and a local feature convolution unit connected in series with the third depth convolution unit, respectively. The multi-scale convolution features of each of the multiple image blocks are input into the multi-path Transformer encoding module to obtain multiple global encoding features and local encoding features of each of the multiple image blocks, including: Inputting the first convolutional features of each of the multiple image blocks into the first Transformer encoder to obtain the first global encoding features of each of the multiple image blocks; Inputting the second convolutional features of each of the multiple image blocks into the second Transformer encoder to obtain the second global encoding features of each of the multiple image blocks; Inputting the third convolutional features of each of the multiple image blocks into the third Transformer encoder to obtain the third global encoding features of each of the multiple image blocks, and inputting the third convolutional features of each of the multiple image blocks into the local feature convolution unit to obtain the local encoding features of each of the multiple image blocks; The multiple global coding features include the first global coding feature, the second global coding feature and the third global coding feature.

2. The method according to claim 1, characterized in that: The first Transformer encoder includes a first normalization layer, a multi-head self-attention layer, a second normalization layer, and a feedforward network connected in series in sequence, and the first convolutional features of each of the multiple image blocks are input into the first Transformer encoder to obtain the first global encoding features of each of the multiple image blocks, including: Inputting the first convolutional features of each of the plurality of image blocks into the first normalization layer to obtain the first normalized features of each of the plurality of image blocks; Inputting the first normalized features of each of the multiple image blocks into the multi-head self-attention layer to obtain the multi-head attention features of each of the multiple image blocks; Inputting the first sum of the first convolutional features and the multi-head attention features of each of the multiple image blocks into a second normalization layer to obtain the second normalized features of each of the multiple image blocks; Inputting the second normalized features of each of the multiple image blocks into the feed-forward network to obtain encoding features of each of the multiple image blocks; The first global coding feature is the sum of the coding feature and the first sum value.

3. The method according to claim 1, characterized in that The local feature convolution unit includes a first convolution layer, a depth convolution layer, and a second convolution layer connected in series in sequence, and the third convolution features of each of the plurality of image blocks are input into the local feature convolution unit to obtain the local coding features of each of the plurality of image blocks, including: Inputting the third convolutional features of each of the plurality of image blocks into the first convolutional layer to obtain fourth convolutional features of each of the plurality of image blocks; Inputting the fourth convolutional features of each of the multiple image blocks into the deep convolutional layer to obtain the deep convolutional features of each of the multiple image blocks; The deep convolution features of each of the multiple image blocks are input into the second convolution layer to obtain the local coding features of each of the multiple image blocks.

4. The method according to any one of claims 1 to 3, characterized in that: The initial generator includes a fully connected layer and a plurality of fourth Transformer encoders connected in series in sequence, and the random noise is input into the initial generator in the initial hyperspectral image classification model to obtain a pseudo hyperspectral image sample, including: Inputting the random noise into the fully connected layer to obtain feature information of the random noise; The characteristic information of the random noise is input into the plurality of fourth Transformer encoders to obtain the pseudo hyperspectral image samples.

5. A training device for a hyperspectral image classification model, characterized in that: include: An acquisition unit, used for acquiring unlabeled hyperspectral image samples and corresponding labeled hyperspectral image samples; A first processing unit is used to input random noise into an initial generator in an initial hyperspectral image classification model to obtain a pseudo hyperspectral image sample; a second processing unit, for performing dimensionality reduction processing on each of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo hyperspectral image samples, extracting a plurality of image blocks of the hyperspectral image samples with each pixel in the hyperspectral image samples after dimensionality reduction as the center, and inputting the plurality of image blocks into an initial discriminator in the initial hyperspectral image classification model to obtain a discrimination result of the hyperspectral image samples; An updating unit, configured to update the model parameters of the initial generator and the initial discriminator based on the discrimination results of the unlabeled hyperspectral image samples, the labeled hyperspectral image samples and the pseudo-hyperspectral image samples, to obtain a hyperspectral image classification model, wherein the hyperspectral image classification model includes the updated discriminator; The initial discriminator includes a multi-scale convolution embedding module, a multi-path Transformer encoding module, a local-global feature fusion module and a classification recognition module. The second processing unit is used to input the multiple image blocks into the initial discriminator in the initial hyperspectral image classification model to obtain the discrimination result of the hyperspectral image sample, including: Inputting the multiple image blocks into the multi-scale convolution embedding module to obtain multi-scale convolution features of the multiple image blocks; Inputting the multi-scale convolutional features of each of the multiple image blocks into the multi-path Transformer encoding module to obtain a plurality of global encoding features and local encoding features of each of the multiple image blocks; Inputting the multiple global coding features and the local coding features of the hyperspectral image sample into the local-global feature fusion module to obtain a target fusion feature; Inputting the target fusion feature into the classification and recognition module to obtain the discrimination result of the hyperspectral image sample; The multi-scale convolution embedding module includes a first depth convolution unit, a second depth convolution unit and a third depth convolution unit connected in parallel, and the second processing unit is used to input the multiple image blocks into the multi-scale convolution embedding module to obtain multi-scale convolution features of the multiple image blocks, including: Inputting the plurality of image blocks into the first depth convolution unit to obtain first convolution features of the plurality of image blocks; Inputting a plurality of image blocks of the hyperspectral image sample into the second deep convolution unit to obtain second convolution features of each of the plurality of image blocks; Inputting a plurality of image blocks of the hyperspectral image sample into the third deep convolution unit to obtain third convolution features of each of the plurality of image blocks; Wherein, the multi-scale convolution feature includes the first convolution feature, the second convolution feature and the third convolution feature; The multi-path Transformer encoding module includes a first Transformer encoder connected in series with the first depth convolution unit, a second Transformer encoder connected in series with the second depth convolution unit, and a third Transformer encoder and a local feature convolution unit connected in series with the third depth convolution unit, respectively. The second processing unit is used to input the multi-scale convolution features of each of the multiple image blocks into the multi-path Transformer encoding module to obtain multiple global encoding features and local encoding features of each of the multiple image blocks, including: Inputting the first convolutional features of each of the multiple image blocks into the first Transformer encoder to obtain the first global encoding features of each of the multiple image blocks; Inputting the second convolutional features of each of the multiple image blocks into the second Transformer encoder to obtain the second global encoding features of each of the multiple image blocks; Inputting the third convolutional features of each of the multiple image blocks into the third Transformer encoder to obtain the third global encoding features of each of the multiple image blocks, and inputting the third convolutional features of each of the multiple image blocks into the local feature convolution unit to obtain the local encoding features of each of the multiple image blocks; The multiple global coding features include the first global coding feature, the second global coding feature and the third global coding feature.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the training method of the hyperspectral image classification model as described in any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the training method of the hyperspectral image classification model according to any one of claims 1 to 4 is implemented.