A hyperspectral image classification method and system based on secondary training

Through a quadratic training method, the model parameters are adjusted using the difference between higher-order and quadratic classification vectors, and combined with the linear classification model, the problem of data redundancy and nonlinear relationship in hyperspectral images is solved, and the classification accuracy is improved. Reduced the need for labeled samples.

CN118230023BActive Publication Date: 2024-12-03NANJING UNIV OF INFORMATION SCI & TECH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202410182358.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-02-19
Publication Date
2024-12-03
Estimated Expiration
2044-02-19

AI Technical Summary

Technical Problem

There are problems in hyperspectral image classification with redundant data, complex nonlinear relationships and insufficient samples, which leads to the inability of accurate classification of traditional methods, and deep learning models have a large demand for labeled data, resulting in low classification accuracy.

Method used

Using a quadratic training method, by extracting the difference between the high-order classification vector and the quadratic classification vector of surrounding pixel blocks as losses, adjusting the model parameters and connecting them to input the linear classification model, reducing the demand for labeled samples and improving classification Accuracy.

Benefits of technology

It improves the reuse rate of pixel block information, reduces the demand for labeled samples by deep learning models, and improves the accuracy of hyperspectral image classification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118230023B_ABST
    Figure CN118230023B_ABST
Patent Text Reader

Abstract

The present invention discloses a hyperspectral image classification method and system based on secondary training, which relates to the technical field of hyperspectral image classification. A high-order classification vector of a pixel is obtained by supervised learning through a Transformer classification model, a secondary training model with the same structure as the classification model is constructed, some pixels and the central pixel in the pixel block are input into the model, and a secondary classification vector is obtained as output. The difference between the high-order classification vector and the secondary classification vector is used as a loss to optimize the parameters of the secondary training model, and the high-order classification vector and the secondary classification vector are connected and input into a linear classification model to obtain the classification probability of the pixel. The linear classification model is fine-tuned using a small amount of labeled data, and the pixels to be classified are respectively used with the trained Transformer classification model, the secondary training model, and the linear classification model to obtain the classification probability of the pixel, and the category with the maximum probability is selected as the label, so as to realize hyperspectral image classification.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image classification, and in particular to a hyperspectral image classification method and system based on secondary training. Background Art

[0002] Hyperspectral remote sensing is one of the most commonly used remote sensing technologies, and plays a vital role in military exploration, surface landform monitoring, disaster prevention and other fields. Hyperspectral images can use the spectral curve characteristic information unique to various objects to identify many similar object categories that cannot be identified by spectral images, making hyperspectral image classification a research hotspot today. Hyperspectral image classification has the following difficulties: 1. Hyperspectral images have high spectral dimensions and serious data redundancy, which increases the computational complexity of classification tasks; 2. The data distribution is complex, and there is a nonlinear relationship and complex pattern between the spectral information of each pixel in the hyperspectral image and the corresponding object, making it impossible for traditional methods to accurately classify; 3. Insufficient samples, experts need to conduct field surveys, and sample labeling consumes manpower and material resources.

[0003] In recent years, with the development of deep learning, deep learning models have been gradually applied to hyperspectral image classification and have shown good classification results. Some scholars have applied convolutional neural networks and generative adversarial networks to hyperspectral image classification. However, the training process of these models often relies on a large amount of labeled data. If a small amount of labeled data is used, the accuracy of hyperspectral image classification will be low. Summary of the invention

[0004] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a hyperspectral image classification method and system based on secondary training, which can solve the problem of low classification accuracy caused by the inability of the current supervised model to fully utilize a small amount of labeled data.

[0005] In a first aspect, the purpose of the present invention can be achieved by the following technical solution: a hyperspectral image classification method based on secondary training, the method comprising the following steps:

[0006] Extracting surrounding pixel blocks from the image to be processed, inputting the surrounding pixel blocks into a pre-established Transformer classification model, and outputting a high-order classification vector, wherein the pixels of the image to be processed are labeled;

[0007] Select some pixels and the central pixel from the surrounding pixel block, input the some pixels and the central pixel into the pre-established secondary training model, and output a secondary classification vector;

[0008] Calculate the difference between the high-order classification vector and the secondary classification vector as the first loss, use the first loss to adjust the parameters of the secondary training model, and obtain the trained secondary training model;

[0009] The high-order classification vector is connected with the secondary classification vector, and is input into a pre-established linear classification model, and the classification probability of the pixel is outputted, and the second loss is calculated according to the classification probability of the pixel and the pixel label of the image to be processed, and the parameters of the linear classification model are adjusted according to the second loss to obtain the trained linear classification model;

[0010] The pixels to be classified are sequentially input into the Transformer classification model, the trained quadratic training model and the trained linear classification model to obtain the classification probability of the pixels to be classified. The classification probability of the pixels to be classified takes the category corresponding to the maximum probability as the pixel label, thereby realizing hyperspectral image classification.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: before inputting the surrounding pixel block into a pre-established Transformer classification model, adding a position code and a classification vector, after the Transformer encoding is completed, projecting the classification vector into a classification probability through a linear layer, and using the cross entropy between the classification probability and the pixel label of the image to be processed as a loss to optimize the Transformer classification model, and finally outputting a high-order classification vector.

[0012] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the pre-established Transformer classification model includes: a pixel feature mapping module and a plurality of encoding modules, the pixel feature mapping module is composed of a linear layer and a normalization layer, and the encoding module is composed of a normalization layer, an attention mechanism layer, a dropout layer, and a linear layer;

[0013] The pre-established secondary training model structure is the same as the Transformer classification model structure.

[0014] In combination with the first aspect, in some implementations of the first aspect, the method further includes: a method of calculating the difference between the high-order classification vector and the secondary classification vector as the first loss:

[0015] The first loss is the sum of the L1 norm and the spectral angle between the high-order classification vector and the secondary classification vector.

[0016] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the method of connecting the high-order classification vector with the secondary classification vector:

[0017] The high-order classification is connected with the secondary classification vector to obtain a classification vector with a higher feature dimension, which is marked as the final classification vector, and then the final classification vector is input into the pre-established linear classification model.

[0018] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the pre-established linear classification model:

[0019] For each sample The classification vector Input linear layer, mapping is obtained The classification probability ,in and Respectively represent the dimension of the connected vector and the total number of categories of the hyperspectral image. The process is expressed as:

[0020]

[0021] in, express The parameter matrix of express The bias parameter maps the classification vector to the classification probability , input the result into the softmax function, the calculation formula is as follows:

[0022]

[0023] in, Represents the final classification probability of the sample;

[0024] The linear classification model includes a linear layer and a softmax layer. The input dimension of the linear layer is the dimension of the input features, and the output dimension is the number of object categories in the hyperspectral image.

[0025] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the calculation formula of taking the sum of the L1 norm and the spectral angle between the high-order classification vector and the secondary classification vector as the first loss is as follows:

[0026] Compute high-order classification vectors With the secondary classification vector The spectral angular distance between:

[0027]

[0028] in, represents the spectral angle calculation function;

[0029] Compute high-order classification vectors With the secondary classification vector The L1 norm distance between:

[0030]

[0031] in, Indicates calculating the L1 norm of two vectors;

[0032] High-order classification vector With the secondary classification vector The difference between It can be expressed as the sum of the spectral angle distance and the L1 norm:

[0033] .

[0034] Preferably, the calculation formula for connecting the high-order classification with the secondary classification vector is as follows:

[0035] The high-order classification vector With the secondary classification vector Concatenate to get the final classification vector ;

[0036] The final classification vector Input Linear Classification Model , which is expressed as follows:

[0037]

[0038] in, Represents the classification probability of the pixel.

[0039] In a second aspect, in order to achieve the above-mentioned object, the present invention discloses a hyperspectral image classification system based on secondary training, comprising:

[0040] A primary input module is used to extract surrounding pixel blocks from the image to be processed, input the surrounding pixel blocks into a pre-established Transformer classification model, and output a high-order classification vector, wherein the pixels of the image to be processed are labeled;

[0041] A secondary input module is used to select some pixels and a central pixel from the surrounding pixel block, input the some pixels and the central pixel into a pre-established secondary training model, and output a secondary classification vector;

[0042] A primary training module is used to calculate the difference between the high-order classification vector and the secondary classification vector as a first loss, and use the first loss to adjust the parameters of the secondary training model to obtain a trained secondary training model;

[0043] A secondary training module, used for connecting the high-order classification vector with the secondary classification vector, and inputting them into a pre-established linear classification model, outputting the classification probability of the pixel, calculating the second loss according to the classification probability of the pixel and the pixel label of the image to be processed, and adjusting the parameters of the linear classification model according to the second loss to obtain a trained linear classification model;

[0044] The image classification module is used to input the pixels to be classified into the Transformer classification model, the trained quadratic training model and the trained linear classification model in sequence to obtain the classification probability of the pixels to be classified. The classification probability of the pixels to be classified takes the category corresponding to the maximum probability as the pixel label, thereby realizing hyperspectral image classification.

[0045] In combination with the second aspect, in some implementations of the second aspect, the system further includes: before the primary input module inputs the surrounding pixel block into the pre-established Transformer classification model, the position code and the classification vector are added; after the Transformer encoding is completed, the classification vector is projected into a classification probability through a linear layer, and the cross entropy between the classification probability and the pixel label of the image to be processed is used as a loss to optimize the Transformer classification model, and finally the high-order classification vector is output;

[0046] Or the pre-established Transformer classification model includes: a pixel feature mapping module and multiple encoding modules, the pixel feature mapping module consists of a linear layer and a normalization layer, and the encoding module consists of a normalization layer, an attention mechanism layer, a dropout layer, and a linear layer;

[0047] The secondary training model structure pre-established by the secondary input module is the same as the Transformer classification model structure;

[0048] The primary training module calculates the difference between the high-order classification vector and the secondary classification vector as the first loss method:

[0049] The L1 norm and the sum of the spectral angles between the high-order classification vector and the secondary classification vector are used as the first loss;

[0050] Or the calculation formula of the first loss is the sum of the L1 norm and the spectral angle between the high-order classification vector and the secondary classification vector as follows:

[0051] Compute high-order classification vectors With the secondary classification vector The spectral angular distance between:

[0052]

[0053] in, represents the spectral angle calculation function;

[0054] Compute high-order classification vectors With the secondary classification vector The L1 norm distance between:

[0055]

[0056] in, Indicates calculating the L1 norm of two vectors;

[0057] High-order classification vector With the secondary classification vector The difference between It can be expressed as the sum of the spectral angle distance and the L1 norm:

[0058]

[0059] Or the method in which the secondary training module connects the high-order classification vector with the secondary classification vector:

[0060] Connect the high-order classification vector with the secondary classification vector to obtain a classification vector with a higher feature dimension, mark it as the final classification vector, and then input the final classification vector into the pre-established linear classification model;

[0061] Or the pre-established linear classification model: The classification vector Input linear layer, mapping is obtained The classification probability ,in and Respectively represent the dimension of the connected vector and the total number of categories of the hyperspectral image. The process is expressed as:

[0062]

[0063] in, express The parameter matrix of express The bias parameter maps the classification vector to the classification probability , input the result into the softmax function, the calculation formula is as follows:

[0064]

[0065] in, Represents the final classification probability of the sample;

[0066] The linear classification model includes a linear layer and a softmax layer. The input dimension of the linear layer is the dimension of the input features, and the output dimension is the number of object categories in the hyperspectral image.

[0067] Alternatively, the calculation formula for connecting the high-order classification with the secondary classification vector in the secondary training module is as follows:

[0068] The high-order classification vector With the secondary classification vector Concatenate to get the final classification vector ;

[0069] The final classification vector Input Linear Classification Model , which is expressed as follows:

[0070]

[0071] in, Represents the classification probability of the pixel.

[0072] Beneficial effects of the present invention:

[0073] The present invention improves the reuse rate of pixel block information through secondary training, selects some pixels to learn complete high-order classification vectors, so as to reduce the demand of deep learning models for labeled samples, thereby improving the classification accuracy of deep learning models. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0075] Figure 1 is a flow chart of the method of the present invention;

[0076] Figure 2 A schematic diagram of a process of hyperspectral image classification based on secondary training in one embodiment;

[0077] Figure 3 A pseudo-color image of an Indian Pines hyperspectral image in one embodiment;

[0078] Figure 4 is the real feature type of the Indian Pines hyperspectral image in one embodiment;

[0079] Figure 5 The classification results of SVM for this comparative example are shown in the following figure.

[0080] Figure 6 This is the classification result of the comparative example 2D-CNN of this application;

[0081] Figure 7 This is the classification result of 3D-CNN in the comparative example of this application;

[0082] Figure 8 This is the classification result of SSFTT, a comparative example of this application;

[0083] Fig. 9 This is the classification result of the comparative example ViT of this application;

[0084] Fig.10 The classification results of the proposed method for this application;

[0085] Fig.11 Schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0086] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0087] Embodiment 1: Below, the relevant terms involved in the embodiment of the present application are introduced:

[0088] Hyperspectral image: Spectral images with spectral resolutions in the range of 10-2λ are called hyperspectral images. After the development of remote sensing technology in the second half of the 20th century, major changes have taken place in theory, technology and application. Among them, the emergence and rapid development of hyperspectral imaging technology is undoubtedly a very prominent aspect of this change. Through hyperspectral sensors, i.e. imaging spectrometers, mounted on different space platforms, the target area is imaged simultaneously in the ultraviolet, visible light, near-infrared and mid-infrared regions of the electromagnetic spectrum with dozens to hundreds of continuous and subdivided spectral bands. While obtaining surface image information, its spectral information is also obtained, which is the first time that the spectrum and image are truly combined. Compared with multispectral remote sensing images, hyperspectral images have not only greatly improved in terms of information richness, but also provided the possibility of more reasonable and effective analysis and processing of such spectral data in terms of processing technology.

[0089] Deep learning model: Deep learning is a general term for a class of pattern analysis methods. In terms of specific research content, it mainly involves three types of methods:

[0090] (1) A neural network system based on convolution operations, namely a convolutional neural network (CNN).

[0091] (2) Autoencoding neural networks based on multi-layer neurons, including autoencoders and sparse coding, which has received widespread attention in recent years.

[0092] (3) Pre-training is performed in the form of a multi-layer autoencoder neural network, and then the deep belief network (DBN) is used to further optimize the neural network weights by combining the identification information.

[0093] Transformer model: The Transformer model is built based on the Self-Attention mechanism. The Self-Attention mechanism is a method that can calculate the relationship between different positions in a sequence. In the Transformer, each input is divided into multiple subsequences after passing through the Embedding layer. Each subsequence passes through multiple layers of Self-Attention and fully connected layers, and finally obtains the output through a linear transformation.

[0094] Specifically, the Transformer model consists of two parts: the encoder and the decoder. The encoder is mainly responsible for converting the input sequence into a fixed-length vector representation, and the decoder decodes this vector into an output sequence.

[0095] Pixel block: A pixel block is an area composed of multiple pixels, which is used to represent a part of the content in the image. A pixel block is an area composed of multiple pixels. Each pixel contains part of the image information, such as color, brightness, etc. By combining multiple pixels together, a larger image can be formed;

[0096] Linear classification model: A linear classification model refers to a classification model whose decision boundary is a linear boundary.

[0097] Secondary classification model: has the same network structure as the Transformer classification model.

[0098] like Figure 1 As shown, a hyperspectral image classification method based on secondary training comprises the following steps:

[0099] Extracting surrounding pixel blocks from the image to be processed, inputting the surrounding pixel blocks into a pre-established Transformer classification model, and outputting a high-order classification vector, wherein the pixels of the image to be processed are labeled;

[0100] The acquisition of images to be processed includes: remote sensing satellites use hyperspectral imaging technology to obtain hyperspectral images of the earth's surface; aircraft or unmanned aerial vehicles are equipped with imaging spectrometers to obtain hyperspectral images; and hyperspectral images of objects are obtained through hyperspectral imagers.

[0101] In this embodiment, the image to be processed is acquired by an airborne imaging spectrometer, a UAV flight plan is formulated according to the range of the hyperspectral image to be acquired, and the UAV is used to perform a flight mission to obtain a hyperspectral image. Compared with remote sensing satellites, airborne equipment can provide higher spatial resolution and clearer image details, providing a better image basis for the subsequent classification and monitoring of surface targets. It also has the characteristics of rapid response and high efficiency, and can obtain the required hyperspectral image data in time in an emergency. Among them, before the surrounding pixel blocks are input into the pre-established Transformer classification model, the position encoding and classification vector are added. After the Transformer encoding is completed, the classification vector is projected into a classification probability through a linear layer, and the cross entropy between the classification probability and the pixel label of the image to be processed is used as a loss to optimize the Transformer classification model, and finally the high-order classification vector is output.

[0102] Further, such as Figure 2 As shown in the figure, the pre-established Transformer classification model includes: The Transformer classification model is mainly composed of a pixel feature mapping module and multiple encoding modules. The pixel feature mapping consists of a linear layer and a normalization layer, which maps the input pixel block to a feature matrix. The encoding module consists of a normalization layer, an attention mechanism layer, a dropout layer, and a linear layer. The encoding module calculates the attention score of the feature matrix and assigns different weights to the feature matrix according to the attention.

[0103] Select some pixels and the central pixel from the surrounding pixel block, input the some pixels and the central pixel into the pre-established secondary training model, and output a secondary classification vector;

[0104] It should be noted that the structure of the pre-established secondary training model is the same as that of the Transformer classification model. The difference lies in the model input. The input of the secondary model is a random selection of pixels and the center pixel in the pixel block. This operation discards the pixels in the pixel block that are potentially unhelpful for model classification. The center pixel is essentially a pixel to be classified, which has the essential feature representation of the pixel to be classified and is very critical to model classification, so it is chosen to be retained.

[0105] Calculate the difference between the high-order classification vector and the secondary classification vector as the first loss, use the first loss to adjust the parameters of the secondary training model, and obtain the trained secondary training model;

[0106] The method of calculating the difference between the high-order classification vector and the secondary classification vector as the first loss is:

[0107] The difference can be calculated using Euclidean distance, Manhattan distance, or Chebyshev distance.

[0108] In this embodiment, the L1 norm and the sum of the spectral angles between the high-order classification vector and the secondary classification vector are used as the first loss. The advantage of using this method is that both the L1 norm and the sum of the spectral angles have good robustness to outliers or noise. The L1 norm can effectively resist the interference of outliers, while the sum of the spectral angles has a certain tolerance to noise. Therefore, combining them as a loss function can improve the model's adaptability to outliers and noise. In addition, the combination of the L1 norm and the sum of the spectral angles can prompt the model to seek optimization in multiple aspects, thereby making the model more diverse. This helps to avoid falling into a local optimal solution and improves the possibility of global optimization.

[0109] It should be noted that the calculation formula of the first loss using the sum of the L1 norm and the spectral angle between the high-order classification vector and the secondary classification vector is as follows:

[0110] Compute high-order classification vectors With the secondary classification vector The spectral angular distance between:

[0111]

[0112] in, represents the spectral angle calculation function;

[0113] Compute high-order classification vectors With the secondary classification vector The L1 norm distance between:

[0114]

[0115] in, Indicates calculating the L1 norm of two vectors;

[0116] High-order classification vector With the secondary classification vector The difference between It can be expressed as the sum of the spectral angle distance and the L1 norm:

[0117] .

[0118] The high-order classification vector is connected with the secondary classification vector, and is input into a pre-established linear classification model, and the classification probability of the pixel is outputted, and the second loss is calculated according to the classification probability of the pixel and the pixel label of the image to be processed, and the parameters of the linear classification model are adjusted according to the second loss to obtain the trained linear classification model;

[0119] Among them, the method of connecting the high-order classification vector with the secondary classification vector is:

[0120] The high-order classification is connected with the secondary classification vector to obtain a classification vector with a higher feature dimension, which is marked as the final classification vector, and then the final classification vector is input into the pre-established linear classification model.

[0121] The calculation formula for connecting the high-order classification with the secondary classification vector is as follows:

[0122] The high-order classification vector With the secondary classification vector Concatenate to get the final classification vector ;

[0123] The final classification vector Input Linear Classification Model , which is expressed as follows:

[0124]

[0125] in, Represents the classification probability of the pixel.

[0126] And the pre-established linear classification model:

[0127] For each sample The classification vector Input linear layer, mapping is obtained The classification probability ,in and Respectively represent the dimension of the connected vector and the total number of categories of the hyperspectral image. The process is expressed as:

[0128]

[0129] in, express The parameter matrix of express The bias parameter maps the classification vector to the classification probability , input the result into the softmax function, the calculation formula is as follows:

[0130]

[0131] in, Represents the final classification probability of the sample;

[0132] The linear classification model includes a linear layer and a softmax layer. The input dimension of the linear layer is the dimension of the input features, and the output dimension is the number of object categories in the hyperspectral image.

[0133] The pixels to be classified are sequentially input into the Transformer classification model, the trained quadratic training model and the trained linear classification model to obtain the classification probability of the pixels to be classified. The classification probability of the pixels to be classified takes the category corresponding to the maximum probability as the pixel label, thereby realizing image classification.

[0134] The hyperspectral remote sensing image dataset used in the present embodiment is the Indian Pines hyperspectral dataset, which contains 200 effective spectral bands. Figure 3 As shown, the pseudo-color image of the Indian Pines dataset is displayed, with a spatial size of 145 145 pixels, the ground sampling distance is 20 meters. Figure 4 As shown, the real ground object types are displayed, including 16 ground object categories. Among them, there are 10249 pixel samples that can be used for experiments. The classification result comparison experiment uses support vector machine (SVM), two-dimensional convolutional network (2D-CNN), three-dimensional convolutional network (3D-CNN), spatial spectrum Transformer network (SSFTT), visual Transformer network (ViT) and the sampling method (Ours) of this application to classify a small amount of labeled sample data of the Indian Pines dataset in the embodiment.

[0135] The experimental setting is as follows: 5% of the samples are randomly selected from 10249 labeled pixels as the training set, and the remaining 95% of the pixels are used as the test set. Under this condition, 5 repeated experiments were conducted, and the classification accuracy is shown in Table 1. ± indicates the fluctuation range of the experimental results.

[0136] Table 1 Comparison of image classification accuracy of Indian Pines dataset

[0137]

[0138] As can be seen from Table 1, this application achieved the best classification results. This application mainly reuses labeled data through secondary training, effectively learns high-order classification vectors through partial pixels, and provides protection for deep learning classification models under small sample classification.

[0139] To visualize the classification results, Figure 5-9 The classification results of SVM, 2D-CNN, 3D-CNN, SSFTT and ViT are shown respectively. Fig.10 The classification result diagram of this application is shown. It can be seen intuitively that with the help of the secondary training model of this application, the classification model can accurately identify the ground object category to which the sample belongs.

[0140] The above proves the feasibility of this application in hyperspectral image classification. This application greatly reduces the demand for labeled data for deep learning models in the field of remote sensing images, so that the deep learning model can also perform well under the condition of a small amount of labeled data, effectively improving the accuracy of hyperspectral image classification.

[0141] Embodiment 2: Fig.11 As shown, the present invention discloses a hyperspectral image classification system based on secondary training, comprising:

[0142] A primary input module is used to extract surrounding pixel blocks from the image to be processed, input the surrounding pixel blocks into a pre-established Transformer classification model, and output a high-order classification vector, wherein the pixels of the image to be processed are labeled;

[0143] A secondary input module is used to select some pixels and a central pixel from the surrounding pixel block, input the some pixels and the central pixel into a pre-established secondary training model, and output a secondary classification vector;

[0144] A primary training module is used to calculate the difference between the high-order classification vector and the secondary classification vector as a first loss, and use the first loss to adjust the parameters of the secondary training model to obtain a trained secondary training model;

[0145] A secondary training module, used for connecting the high-order classification vector with the secondary classification vector, and inputting them into a pre-established linear classification model, outputting the classification probability of the pixel, calculating the second loss according to the classification probability of the pixel and the pixel label of the image to be processed, and adjusting the parameters of the linear classification model according to the second loss to obtain a trained linear classification model;

[0146] The image classification module is used to input the pixels to be classified into the Transformer classification model, the trained quadratic training model and the trained linear classification model in sequence to obtain the classification probability of the pixels to be classified. The classification probability of the pixels to be classified takes the category corresponding to the maximum probability as the pixel label, thereby realizing image classification.

[0147] In combination with the second aspect, in some implementations of the second aspect, the system further includes: before the primary input module inputs the surrounding pixel block into the pre-established Transformer classification model, the position code and the classification vector are added; after the Transformer encoding is completed, the classification vector is projected into a classification probability through a linear layer, and the cross entropy between the classification probability and the pixel label of the image to be processed is used as a loss to optimize the Transformer classification model, and finally the high-order classification vector is output;

[0148] Or the pre-established Transformer classification model includes: a pixel feature mapping module and multiple encoding modules, the pixel feature mapping module consists of a linear layer and a normalization layer, and the encoding module consists of a normalization layer, an attention mechanism layer, a dropout layer, and a linear layer;

[0149] The secondary training model structure pre-established by the secondary input module is the same as the Transformer classification model structure;

[0150] The primary training module calculates the difference between the high-order classification vector and the secondary classification vector as the first loss method:

[0151] The L1 norm and the sum of the spectral angles between the high-order classification vector and the secondary classification vector are used as the first loss;

[0152] Or the calculation formula of the first loss is the sum of the L1 norm and the spectral angle between the high-order classification vector and the secondary classification vector as follows:

[0153] Compute high-order classification vectors With the secondary classification vector The spectral angular distance between:

[0154]

[0155] in, represents the spectral angle calculation function;

[0156] Compute high-order classification vectors With the secondary classification vector The L1 norm distance between:

[0157]

[0158] in, Indicates calculating the L1 norm of two vectors;

[0159] High-order classification vector With the secondary classification vector The difference between It can be expressed as the sum of the spectral angle distance and the L1 norm:

[0160]

[0161] Or the method in which the secondary training module connects the high-order classification vector with the secondary classification vector:

[0162] Connect the high-order classification vector with the secondary classification vector to obtain a classification vector with a higher feature dimension, mark it as the final classification vector, and then input the final classification vector into the pre-established linear classification model;

[0163] Or the pre-established linear classification model: The classification vector Input linear layer, mapping is obtained The classification probability ,in and Respectively represent the dimension of the connected vector and the total number of categories of the hyperspectral image. The process is expressed as:

[0164]

[0165] in, express The parameter matrix of express The bias parameter maps the classification vector to the classification probability , input the result into the softmax function, the calculation formula is as follows:

[0166]

[0167] in, Represents the final classification probability of the sample;

[0168] The linear classification model includes a linear layer and a softmax layer. The input dimension of the linear layer is the dimension of the input features, and the output dimension is the number of object categories in the hyperspectral image.

[0169] Alternatively, the calculation formula for connecting the high-order classification with the secondary classification vector in the secondary training module is as follows:

[0170] The high-order classification vector With the secondary classification vector Concatenate to get the final classification vector ;

[0171] The final classification vector Input Linear Classification Model , which is expressed as follows:

[0172]

[0173] in, Represents the classification probability of the pixel.

[0174] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0175] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by the processor to execute the above method. The storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0176] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0177] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure may have various changes and improvements, and these changes and improvements fall within the scope of the present disclosure to be protected.

Claims

1. A hyperspectral image classification method based on secondary training, characterized in that: The method comprises the following steps: Extracting surrounding pixel blocks for each pixel in the hyperspectral image to be processed, inputting the surrounding pixel blocks into a pre-established Transformer classification model, and outputting a high-order classification vector, wherein the pixels of the hyperspectral image to be processed are labeled; Select some pixels and the central pixel from the surrounding pixel block, input the some pixels and the central pixel into the pre-established secondary training model, and output a secondary classification vector; Calculate the difference between the high-order classification vector and the secondary classification vector as the first loss, use the first loss to adjust the parameters of the secondary training model, and obtain the trained secondary training model; The high-order classification vector is connected with the secondary classification vector, and is input into a pre-established linear classification model, and the classification probability of the pixel is outputted, and the second loss is calculated according to the classification probability of the pixel and the pixel label of the image to be processed, and the parameters of the linear classification model are adjusted according to the second loss to obtain the trained linear classification model; The surrounding pixel blocks of the pixel to be classified are input into the Transformer classification model, the trained quadratic training model and the trained linear classification model in turn to obtain the classification probability of the pixel to be classified. Among them, the surrounding pixel block of the pixel to be classified is input into the Transformer classification model to obtain the high-order classification vector of the pixel to be classified; some pixels and the central pixel are selected from the surrounding pixel block, and some pixels and the central pixel are input into the trained secondary training model to obtain the secondary classification vector of the pixel to be classified; the high-order classification vector of the pixel to be classified is connected with the secondary classification vector of the pixel to be classified, and input into the trained linear classification model to obtain the classification probability of the pixel to be classified; The classification probability of the pixels to be classified is taken as the pixel label corresponding to the category with the maximum probability, thereby realizing the classification of hyperspectral images.

2. The hyperspectral image classification method based on secondary training according to claim 1 is characterized in that: Before the surrounding pixel block is input into the pre-established Transformer classification model, the position code and the classification vector are added. After the Transformer encoding is completed, the classification vector is projected into the classification probability through the linear layer, and the cross entropy between the classification probability and the pixel label is used as the loss to optimize the Transformer classification model, and finally the high-order classification vector is output.

3. The hyperspectral image classification method based on secondary training according to claim 1 is characterized in that: The pre-established Transformer classification model includes: a pixel feature mapping module and multiple encoding modules, the pixel feature mapping module is composed of a linear layer and a normalization layer, and the encoding module is composed of a normalization layer, an attention mechanism layer, a dropout layer and a linear layer; The pre-established secondary training model structure is the same as the Transformer classification model structure.

4. The hyperspectral image classification method based on secondary training according to claim 1 is characterized in that: The method of calculating the difference between the high-order classification vector and the secondary classification vector as the first loss is: The first loss is the sum of the L1 norm and the spectral angle between the high-order classification vector and the secondary classification vector.

5. The hyperspectral image classification method based on secondary training according to claim 1 is characterized in that: The method of connecting the high-order classification vector with the secondary classification vector: The high-order classification vector is connected to the secondary classification vector to obtain a classification vector with a higher feature dimension, which is marked as the final classification vector, and then the final classification vector is input into the pre-established linear classification model.

6. The hyperspectral image classification method based on secondary training according to claim 5 is characterized in that: The pre-built linear classification model: For each sample The classification vector Input linear layer, mapping is obtained The classification probability ,in and Respectively represent the dimension of the connected vector and the total number of categories of the hyperspectral image. The process is expressed as: in, express The parameter matrix of express The bias parameter maps the classification vector to the classification probability , input the result into the softmax function, the calculation formula is as follows: in, Represents the final classification probability of the sample; The linear classification model includes a linear layer and a softmax layer. The input dimension of the linear layer is the dimension of the input features, and the output dimension is the number of object categories in the hyperspectral image.

7. The hyperspectral image classification method based on secondary training according to claim 4 is characterized in that: The calculation formula of the first loss is the sum of the L1 norm and the spectral angle between the high-order classification vector and the secondary classification vector as follows: Compute high-order classification vectors With the secondary classification vector The spectral angular distance between: in, represents the spectral angle calculation function; Compute high-order classification vectors With the secondary classification vector The L1 norm distance between: in, Indicates calculating the L1 norm of two vectors; High-order classification vector With the secondary classification vector The difference between It can be expressed as the sum of the spectral angular distance and the L1 norm: 。 8. The hyperspectral image classification method based on secondary training according to claim 5, characterized in that: The calculation formula for connecting the high-order classification with the secondary classification vector is as follows: The high-order classification vector With the secondary classification vector Concatenate to get the final classification vector ; The final classification vector Input Linear Classification Model , which is expressed as follows: in, Represents the classification probability of the pixel.

9. A hyperspectral image classification system based on secondary training, characterized in that: include: A primary input module is used to extract surrounding pixel blocks from the image to be processed, input the surrounding pixel blocks into a pre-established Transformer classification model, and output a high-order classification vector, wherein the pixels of the image to be processed are labeled; A secondary input module is used to select some pixels and a central pixel from the surrounding pixel block, input the some pixels and the central pixel into a pre-established secondary training model, and output a secondary classification vector; A primary training module is used to calculate the difference between the high-order classification vector and the secondary classification vector as a first loss, and use the first loss to adjust the parameters of the secondary training model to obtain a trained secondary training model; A secondary training module, used for connecting the high-order classification vector with the secondary classification vector, and inputting them into a pre-established linear classification model, outputting the classification probability of the pixel, calculating the second loss according to the classification probability of the pixel and the pixel label of the image to be processed, and adjusting the parameters of the linear classification model according to the second loss to obtain a trained linear classification model; The image classification module is used to input the pixels to be classified into the Transformer classification model, the trained quadratic training model and the trained linear classification model in sequence to obtain the classification probability of the pixels to be classified; Among them, the surrounding pixel block of the pixel to be classified is input into the Transformer classification model to obtain the high-order classification vector of the pixel to be classified; some pixels and the central pixel are selected from the surrounding pixel block, and some pixels and the central pixel are input into the trained secondary training model to obtain the secondary classification vector of the pixel to be classified; the high-order classification vector of the pixel to be classified is connected with the secondary classification vector of the pixel to be classified, and input into the trained linear classification model to obtain the classification probability of the pixel to be classified; The classification probability of the pixels to be classified is taken as the pixel label corresponding to the category with the maximum probability, thereby realizing the classification of hyperspectral images.

10. The hyperspectral image classification system based on secondary training according to claim 9, characterized in that: The primary input module adds position coding and classification vectors before inputting surrounding pixel blocks into a pre-established Transformer classification model. After the Transformer encoding is completed, the classification vectors are projected into classification probabilities through a linear layer, and the cross entropy between the classification probabilities and the pixel labels of the image to be processed is used as a loss to optimize the Transformer classification model, and finally outputs a high-order classification vector. Or the pre-established Transformer classification model includes: a pixel feature mapping module and multiple encoding modules, the pixel feature mapping module consists of a linear layer and a normalization layer, and the encoding module consists of a normalization layer, an attention mechanism layer, a dropout layer, and a linear layer; The secondary training model structure pre-established by the secondary input module is the same as the Transformer classification model structure; The primary training module calculates the difference between the high-order classification vector and the secondary classification vector as the first loss method: The L1 norm and the sum of the spectral angles between the high-order classification vector and the secondary classification vector are used as the first loss; Or the calculation formula of the first loss is the sum of the L1 norm and the spectral angle between the high-order classification vector and the secondary classification vector as follows: Compute high-order classification vectors With the secondary classification vector The spectral angular distance between: in, represents the spectral angle calculation function; Compute high-order classification vectors With the secondary classification vector The L1 norm distance between: in, Indicates calculating the L1 norm of two vectors; High-order classification vector With the secondary classification vector The difference between It can be expressed as the sum of the spectral angle distance and the L1 norm: Or the method in which the secondary training module connects the high-order classification vector with the secondary classification vector: Connect the high-order classification vector with the secondary classification vector to obtain a classification vector with a higher feature dimension, mark it as the final classification vector, and then input the final classification vector into the pre-established linear classification model; Or the pre-established linear classification model: The classification vector Input linear layer, mapping is obtained The classification probability ,in and Respectively represent the dimension of the connected vector and the total number of categories of the hyperspectral image. The process is expressed as: in, express The parameter matrix of express The bias parameter maps the classification vector to the classification probability , input the result into the softmax function, the calculation formula is as follows: in, Represents the final classification probability of the sample; The linear classification model includes a linear layer and a softmax layer. The input dimension of the linear layer is the dimension of the input features, and the output dimension is the number of ground object categories in the hyperspectral image. Alternatively, the calculation formula for connecting the high-order classification with the secondary classification vector in the secondary training module is as follows: The high-order classification vector With the secondary classification vector Concatenate to get the final classification vector ; The final classification vector Input Linear Classification Model , which is expressed as follows: in, Represents the classification probability of the pixel.

Citation Information

Patent Citations

  • Hyperspectral image classification method based on space-spectrum mixed self-attention mechanism

    CN116977723A