A hyperspectral remote sensing image quality grade evaluation method

Abstract

The application relates to a hyperspectral remote sensing image quality grade evaluation method, which comprises the following steps: dividing an image into fixed-size blocks, then constructing a quality evaluation model, obtaining a feature map with spectral weight through a spatial-spectral attention module, and inputting the feature map into a frequency spectrum integration embedding module to generate an embedding feature map of local adjacent spectral bands; after linear projection, the embedding feature map is input into an encoder, and a skip connection mechanism is set to fuse the outputs of different encoders; the output of the last encoder is input into a linear layer and an activation function layer to obtain a quality evaluation result of the image; the quality evaluation model is trained, and the trained model is used for image quality grade evaluation. The application can perceive the good and bad degrees of a hyperspectral remote sensing image, and can dynamically detect and adjust the image output by an image processing system in a hyperspectral imager according to the quality grade of the image, thereby providing a more effective basis for parameter optimization of a real-time hyperspectral imager system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a hyperspectral remote sensing image quality grade evaluation method. BACKGROUND

[0002] In recent years, with the rapid development of space technology and remote sensing technology, China's space industry has entered a period of rapid development. Various multifunctional satellites and probes are sent into space to observe the earth from multiple angles. Remote sensing imaging technology can perceive and identify the observed scene by using its different reflection characteristics without physical contact with the object. Satellite and other spacecraft devices collect images at different times, different spectral bands and different sites by high-altitude bird's eye view and provide remote sensing information for researchers by combining image processing and image enhancement technology. The current imaging spectrometer can observe the continuous spectrum from visible light to short-wave infrared, so we can obtain hyperspectral images (HSIs) with hundreds of narrow and nearly continuous spectral bands through the imaging spectrometer. However, due to factors such as cloud cover, equipment wear and tear, sensor noise, the actual obtained hyperspectral remote sensing image can have very little useful information for research, so the quality evaluation research work for hyperspectral remote sensing image is particularly important. The quality evaluation work of hyperspectral remote sensing image provides a quality assurance basis for real-time monitoring, maintenance and subsequent hyperspectral remote sensing image processing of the imaging instrument of the remote sensing spacecraft.

[0003] In recent years, domestic and foreign scholars have done a lot of research on single-band remote sensing image quality evaluation, but the research on hyperspectral remote sensing image is less involved. Therefore, based on the previous research on single-band remote sensing image, combined with the characteristics of hyperspectral, it has very important research and application value to carry out research on the quality evaluation of hyperspectral remote sensing image. So far, the quality evaluation of hyperspectral remote sensing image mostly adopts traditional feature extraction methods, that is, manually extracting features, and the typical methods include structural similarity SSIM, peak signal-to-noise ratio, variance method, etc. However, these indicators are not directly related to image interpretation ability, and cannot well mine spectral information and fuse image spatial and spectral features, so only shallow features can be extracted, which affects the model quality to some extent. Secondly, the existing quality evaluation methods are mostly full-reference and semi-reference methods. Due to the fact that perfect original images are often not available in real life, especially in the field of remote sensing, remote sensing images almost have no reference images, so full-reference and semi-reference methods have fewer application scenarios in practice. SUMMARY

[0004] The purpose of this invention is to propose a method for evaluating the quality level of hyperspectral remote sensing images. Using a deep learning model and an efficient and convenient evaluation algorithm, it can perceive the quality level of hyperspectral remote sensing images and dynamically monitor, detect, and adjust the output image of the image processing system in the hyperspectral imager based on the image quality level results. This provides a more effective basis for optimizing the parameters of the real-time hyperspectral imager system. It overcomes the limitation of traditional methods that can only extract shallow features during quality evaluation. The deep learning algorithm can automatically process large amounts of data, automatically learn and extract useful features from complex data, reducing the need for manual intervention and subjective judgment, and more accurately evaluating the quality of hyperspectral remote sensing images. Furthermore, it solves the problem of the lack of reference images in the current remote sensing field. This invention uses a referenceless method for quality evaluation, completely reducing the dependence on reference images.

[0005] The technical solution adopted in this invention is: a method for evaluating the quality level of hyperspectral remote sensing images, comprising the following steps:

[0006] S1: Construct a hyperspectral remote sensing image dataset and divide the hyperspectral remote sensing image dataset into a training set and a test set;

[0007] S2: Perform a segmentation operation on the hyperspectral remote sensing images in the hyperspectral remote sensing image dataset to segment the hyperspectral remote sensing images into sub-image blocks of fixed size;

[0008] S3: Construct a quality assessment model, which includes a Vision Transformer model, a spatial-spectral attention module, a spectral integration embedding module, and a cross-layer fusion residual module. The Vision Transformer model is the basic skeleton of the quality assessment model, including T encoders, where T ≥ 4. Each encoder includes a multi-head self-attention mechanism and a feedforward neural network for extracting image features. The spatial-spectral attention module and the spectral integration embedding module are sequentially set at the front end of the encoder of the Vision Transformer model, and the cross-layer fusion residual module is set between the Vision Transformer encoders.

[0009] S4: Input all sub-image blocks into the spatial-spectral attention module to extract feature maps with spectral weights; the spatial-spectral attention module performs max pooling and average pooling operations on the input image, learns the channel weights through two fully connected layers, and avoids gradient vanishing through the GELU activation function. Finally, the output of the GELU function is multiplied by the original input to obtain the feature map with spectral weights as the output of the spatial-spectral attention module.

[0010] S5: Input the feature map with spectral weights into the spectrum integration embedding module to obtain a locally adjacent spectral band grouped embedding feature map; the spectrum integration embedding module will group the number of channels... C The spectrally weighted feature map is flattened and then... n Grouping and integrating adjacent channels to obtain The feature map after grouping, that is, the feature map embedded by grouping local adjacent spectral bands;

[0011] S6: Group adjacent spectral bands into the feature map and perform linear projection to obtain the linear projection feature map;

[0012] S7: Input the linear projection feature map into the T encoders of the Vision Transformer model to perform multi-head self-attention and multilayer perceptron operations;

[0013] S8: Perform cross-layer fusion residual operation on the encoder through the cross-layer fusion residual module; the cross-layer fusion residual module performs cross-layer fusion residual operation on the encoder. and The output of the encoder as well as Perform splicing and blending operations to obtain the blended output. And a skip connection mechanism is used to fuse the output. Enter the number t +2 encoders;

[0014] S9: Input the feature map output by the last encoder into the linear layer and activation function layer to obtain the quality assessment result of the hyperspectral remote sensing image;

[0015] S10: Input the training set and the corresponding subjective quality level into the quality assessment model for training, and use the test set to test the model performance of the quality assessment model;

[0016] S11: After training is complete, input the hyperspectral remote sensing image to be evaluated into the quality assessment model to obtain the quality level.

[0017] Furthermore, in step S2, the hyperspectral remote sensing image is cropped using a sliding window method, dividing the hyperspectral remote sensing image into sub-image blocks of fixed size, and there is no overlapping area between different sub-image blocks.

[0018] Furthermore, the specific steps of step S4 are as follows:

[0019] S401: For input sub-image blocks Its vector dimension is ( C, H , W ),Right now ,in RThe set of real numbers represents the set of all real numbers. C This indicates the number of channels in the image. H Indicates the height of the image, W Represents the width of the image, input sub-image blocks. In pixels Features of the place Its vector dimension is 1×1, and the number of channels is C ,Right now , for features The specific expressions for performing average pooling and max pooling operations are as follows:

[0020] ;

[0021] ;

[0022] in, , yes In position The expression of time, This represents the features after global average pooling. This represents the features after the max pooling operation. This indicates taking the maximum region;

[0023] S402: The features obtained after global average pooling operation in step S401 are... Features after max pooling operation The input is fed into two fully connected layers, and the two results from the fully connected layers are added together and then a ReLU activation function is added to obtain the corresponding 1-D spectral attention map. :

[0024] ;

[0025] Among them, 1-D spectral attention map The dimension is 1×1, and the number of channels is C ,Right now linear transformation The dimension is ,Right now linear transformation The dimension is ,Right now , r These are customizable hyperparameters; the first fully connected layer serves as... The parameterized dimensionality reduction layer, the second fully connected layer is based on... Parameterized dimension-adding layers, linear transformation and linear transformation Shared weights; Represents the ReLU activation function;

[0026] S403: Input sub-image block Corresponding 1-D spectral attention map Multiplication yields the output of the spatial spectral attention module, which is the feature map with spectral weights corresponding to the input sub-image y. Its vector dimension is ( C, H , W ),Right now ,in R The set of real numbers represents the set of all real numbers. Represents the multiplication element.

[0027] Furthermore, the specific steps of step S5 are as follows:

[0028] Step S501: Transfer the feature map with spectral weights The 2D planar image of each band is unfolded into the corresponding 1D vector. express: , Indicates the first The 1-D vector of the channel, = 1,2…, C , among which, the 1-D vector of the channel The dimension is ,Right now 1-D vector The dimension is ,Right now ;

[0029] S502: Convert the 1-D vector middle n The vectors of adjacent bands are integrated as input to obtain the corresponding local adjacent spectral band grouping embedding feature map. :

[0030] ;

[0031] Among them, the feature map is embedded in the grouping of local adjacent spectral bands. The dimension is ,Right now , This indicates a grouping and integration operation. This indicates the floor function.

[0032] Further, the specific steps of step S6 are as follows: grouping and embedding the locally adjacent spectral bands obtained in step S502 into the feature map. Perform a linear transformation and use a fully connected layer to group and embed locally adjacent spectral bands into the feature map. Dimensions Dimensional reduction D Dimension, to obtain linear projection feature map Linear projection feature map The dimension is ,Right now , .

[0033] Furthermore, the specific steps of step S7 are as follows:

[0034] S701: After performing layer normalization on the linear projection feature map X obtained in step S6, the query vector is obtained through linear transformation. key vector K Sum value vector V :

[0035] ;

[0036] ;

[0037] ;

[0038] in, This is the first learnable weight matrix. This is the second learnable weight matrix. This is the third learnable weight matrix;

[0039] Then, a multi-head self-attention operation is performed to obtain the first... Query vector of heads , No. The key vector of each head and the The value vector of each head :

[0040] ;

[0041] ;

[0042] ;

[0043] ;

[0044] ;

[0045] in, The number of heads in a multi-head self-attention mechanism. This is the fourth learnable weight matrix. This is the fifth learnable weight matrix. This is the sixth learnable weight matrix. This is the seventh learnable weight matrix. This represents the first in the multi-head self-attention mechanism. Size;

[0046] The attention operation is as follows: , It is a key vector K transpose, It is a scaling factor. It is a query vector and key vector K The dimension; This indicates a bullish self-attention strategy. This indicates a join operation; MSA represents the output of a multi-head self-attention operation.

[0047] S702: Combine the output MSA and linear projection feature map from the multi-head self-attention operation in step S701. Adding them together yields the feature map. Feature map The dimension is also ( ); and feature map After layer normalization, the input is fed into the multilayer perceptron layer to obtain the output of the multilayer perceptron operation. ;in, It is a layer normalization operation. It is a multilayer perceptron operation; finally, the feature map is... Output of multilayer perceptron operation Add them together to get the first one. The output of the encoder .

[0048] Furthermore, in step S8, the first... The and the first The output of the encoder and , No. The output of the encoder and the The output of the encoder All dimensions are ,Right now , , d For the dimension of feature embedding, d = C × H × W ; for the first The output of the encoder and the The output of the encoder Perform cross-layer fusion connection operations to obtain the first Fusion output of layer-to-layer fusion residual module ,in, Represents the learnable network parameters, with dimension 1. ,Right now ; and will the Fusion output of layer-to-layer fusion residual module As the first One of the inputs of the encoder, the first Another input to the encoder is the first... The output of the encoder .

[0049] Furthermore, in step S9, the output of the last encoder... Perform MLP Head operation to obtain the final quality level result of the hyperspectral remote sensing image. ;in, It is the GELU activation function. It is a fully connected layer operation.

[0050] The beneficial effects of this invention are as follows:

[0051] (1) This invention uses deep learning algorithms and adopts the Vision Transformer model, i.e. the ViT model, as the basic network model. Compared with the traditional manual feature extraction method, this method effectively extracts the spatial and local spectral representation and deep features of hyperspectral images, and performs rapid and efficient evaluation of the quality classification of hyperspectral remote sensing images.

[0052] (2) This invention can directly capture the global spectral order information of hyperspectral images from the perspective of sequence through the spectral attention module, the spectral integration embedding module and the cross-layer fusion residual module, enhance the information transmission capability between bands, and perform feature fusion by combining spatial features and spectral features, thereby further improving the accuracy of image quality assessment.

[0053] (3) The present invention adopts a referenceless image quality assessment method, which solves the problem of lack of reference images in the field of remote sensing. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of the structure of the quality assessment model in an embodiment of the present invention;

[0057] Figure 3 This is a flowchart of the spatial spectrum attention module in an embodiment of the present invention;

[0058] Figure 4 This is a flowchart of the spectrum integration embedding module in an embodiment of the present invention;

[0059] Figure 5 This is a flowchart of the cross-layer fusion residual module in an embodiment of the present invention. Detailed Implementation

[0060] To better understand the above-described objects, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention; however, the invention may be practiced in other ways different from those described herein, and therefore, the invention is not limited to the specific embodiments disclosed below.

[0061] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning understood by one of ordinary skill in the art described herein. The terms “first,” “second,” and similar terms used in this patent application specification and claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an” or “a” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “connected” or “linked” and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” “right,” etc., are used only to indicate relative positional relationships, which change accordingly when the absolute position of the described object changes.

[0062] like Figure 1 As shown, a method for evaluating the quality level of hyperspectral remote sensing images includes the following steps:

[0063] S1: Construct a hyperspectral remote sensing image dataset and divide the hyperspectral remote sensing image dataset into a training set and a test set.

[0064] S2: Perform a segmentation operation on the hyperspectral remote sensing images in the hyperspectral remote sensing image dataset to divide the hyperspectral remote sensing images into sub-image blocks of fixed size.

[0065] This invention employs a sliding window method to crop hyperspectral remote sensing images, dividing them into fixed-size sub-image blocks with no overlapping areas between different sub-image blocks. In this embodiment, the sub-image block size is set to 8*8.

[0066] S3: Construct a quality assessment model, such as Figure 2 As shown, the quality assessment model includes a Vision Transformer model, a spatial-spectral attention module, a spectral integration embedding module, and a cross-layer fusion residual module. The Vision Transformer model is the basic skeleton of the quality assessment model, including T encoders, where T ≥ 4. Each encoder includes a multi-head self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism can enhance the model's understanding of the entire image, and the feedforward neural network processes the output at each position independently to further extract image features. The spatial-spectral attention module and the spectral integration embedding module are sequentially set at the front end of the encoder of the Vision Transformer model, and the cross-layer fusion residual module is set between the Vision Transformer encoders.

[0067] S4: Input all sub-image patches into the spatial spectral attention module to extract feature maps with spectral weights; such as... Figure 3 As shown, the spatial-spectral attention module performs max pooling and average pooling operations on the input image to obtain global statistical information for each channel. The pooling results are then fed into two fully connected layers, where channel weights are learned. The results are summed and then activated using the GELU function to prevent gradient vanishing. Finally, the GELU output is multiplied by the original input to obtain a feature map with spectral weights, which serves as the output of the spatial-spectral attention module. The spatial-spectral attention module can assign different weights to the numerous spectral channels of a hyperspectral image, enabling the network to better capture the features of the input data and improve the model's generalization ability. The specific steps are as follows:

[0068] S401: For input sub-image blocks Its vector dimension is ( C, H , W ),Right now ,in R The set of real numbers represents the set of all real numbers. C This indicates the number of channels in the image. H Indicates the height of the image, W Represents the width of the image, input sub-image blocks. In pixels Features of the place Its vector dimension is 1×1, and the number of channels is C ,Right now , for features The specific expressions for performing average pooling and max pooling operations are as follows:

[0069] ;

[0070] ;

[0071] in, , yes In position The expression of time, This represents the features after global average pooling. This represents the features after the max pooling operation. This indicates taking the largest region.

[0072] S402: The features obtained after global average pooling operation in step S401 are... Features after max pooling operation The input is fed into two fully connected layers, and the two results from the fully connected layers are added together and then a ReLU activation function is added to obtain the corresponding 1-D spectral attention map. :

[0073] ;

[0074] Among them, 1-D spectral attention map The dimension is 1×1, and the number of channels is C ,Right now linear transformation The dimension is ,Right now linear transformation The dimension is ,Right now , r These are customizable hyperparameters; the first fully connected layer serves as... The parameterized dimensionality reduction layer, the second fully connected layer is based on... Parameterized dimension-adding layers, linear transformation and linear transformation Shared weights; This represents the ReLU activation function.

[0075] S403: Input sub-image block Corresponding 1-D spectral attention map Multiplication yields the output of the spatial spectral attention module, which is the feature map with spectral weights corresponding to the input sub-image y. Its vector dimension is ( C, H , W ),Right now ,in R The set of real numbers represents the set of all real numbers. Represents the multiplication element.

[0076] S5: Input the feature map with spectral weights into the spectral integration embedding module to obtain the locally adjacent spectral band grouped embedding feature map; such as Figure 4 As shown, the spectrum integration embedding module integrates channels with a number of channels. C The spectrally weighted feature map is flattened and then... n Grouping and integrating adjacent channels to obtain The specific steps for creating the grouped feature map, i.e., the embedded feature map of locally adjacent spectral bands, are as follows:

[0077] Step S501: Transfer the feature map with spectral weights The 2D planar image of each band is unfolded into the corresponding 1D vector. express: , Indicates the first The 1-D vector of the channel, = 1,2…, C , among which, the 1-D vector of the channel The dimension is ,Right now 1-D vector The dimension is ,Right now .

[0078] S502: Convert the 1-D vector middle n The vectors of adjacent bands are integrated as input to obtain the corresponding local adjacent spectral band grouping embedding feature map. :

[0079] ;

[0080] Among them, the feature map is embedded in the grouping of local adjacent spectral bands. The dimension is ,Right now , This indicates a grouping and integration operation. This indicates a rounding down operation; in this embodiment of the invention, n The value of is 3.

[0081] S6: Group adjacent spectral bands into the feature map and perform linear projection to obtain a linearly projected feature map. The specific steps are as follows: Group adjacent spectral bands obtained in step S502 into the feature map. Perform a linear transformation and use a fully connected layer to group and embed locally adjacent spectral bands into the feature map. Dimensions Dimensional reduction D Dimension, to obtain linear projection feature map Linear projection feature map The dimension is ,Right now , .

[0082] The spectrum integration embedding module avoids the shortcomings of the Vision Transformer model, which inputs image patches into the encoder in a discrete order. It enhances the connection and information interaction between adjacent bands, learns detailed local spectral representations, and enhances the fusion between spatial and spectral information.

[0083] S7: Input the linear projection feature map into the T encoders of the Transformer model, and perform multi-head self-attention and multilayer perceptron operations. The specific steps are as follows:

[0084] S701: After performing layer normalization on the linear projection feature map X obtained in step S6, the query vector is obtained through linear transformation. key vector K Sum value vector V :

[0085] ;

[0086] ;

[0087] ;

[0088] in, This is the first learnable weight matrix. This is the second learnable weight matrix. This is the third learnable weight matrix.

[0089] Then, a multi-head self-attention operation is performed to obtain the first... Query vector of heads , No. The key vector of each head and the The value vector of each head :

[0090] ;

[0091] ;

[0092] ;

[0093] ;

[0094] ;

[0095] in, The number of heads in a multi-head self-attention mechanism. This is the fourth learnable weight matrix. This is the fifth learnable weight matrix. This is the sixth learnable weight matrix. This is the seventh learnable weight matrix. This represents the first in the multi-head self-attention mechanism. Size;

[0096] The attention operation is as follows: , It is a key vector K transpose, It is a scaling factor. It is a query vector and key vector K The dimension; This indicates a bullish self-attention strategy. This indicates a join operation, and MSA represents the output of a multi-head self-attention operation.

[0097] S702: Combine the output MSA and linear projection feature map from the multi-head self-attention operation in step S701. Adding them together yields the feature map. Feature map The dimension is also ; feature map After layer normalization, the input is fed into the multilayer perceptron layer to obtain the output of the multilayer perceptron operation. ;in, It is a layer normalization operation. It is a multilayer perceptron operation; finally, the feature map is... Output of multilayer perceptron operation Add them together to get the first one. The output of the encoder .

[0098] S8: Perform cross-layer fusion residual operation on the encoder through the cross-layer fusion residual module; such as Figure 5 As shown, the cross-layer fusion residual module takes the first... and The output of the encoder as well as Concatenate these two matrices into a single matrix to obtain the concatenated matrix. ; for the newly generated splicing matrix Perform the fusion operation: The resulting fusion output This refers to the fusion result of the outputs from two different encoders; a skip connection (SC) mechanism is used to fuse the fused outputs. Enter the number t In the +2 encoders. The cross-layer fusion residual module can enhance the information transfer between encoders. In other words, this module can convey memory-like components from shallow to deep layers, enhance the connection between many spectral channels of hyperspectral remote sensing images, and reduce information loss as the number of encoders gradually increases due to the addition of residual structures.

[0099] In the embodiments of the present invention, the first... The and the first The output of the encoder and , No. The output of the encoder and the The output of the encoder All dimensions are ,Right now , , d For the dimension of feature embedding, d = C × H × W ; for the first The output of the encoder and the The output of the encoder Perform cross-layer fusion connection operations to obtain the first Fusion output of layer-to-layer fusion residual module ,in, Represents the learnable network parameters, with dimension 1. ,Right now ; and will the Fusion output of layer-to-layer fusion residual module As the first One of the inputs of the encoder, the first Another input to the encoder is the first... The output of the encoder .

[0100] S9: Input the feature map output by the last encoder into the linear layer and activation function layer to obtain the quality assessment result of the hyperspectral remote sensing image. This embodiment of the invention uses the output of the last encoder... Perform MLP Head operation to obtain the final quality level result of the hyperspectral remote sensing image: ;in, It is the GELU activation function. It is a fully connected layer operation.

[0101] S10: Input the training set and the corresponding subjective quality level into the quality assessment model for training, and use the test set to test the model performance of the quality assessment model;

[0102] S11: After training is complete, input the hyperspectral remote sensing image to be evaluated into the quality assessment model to obtain the quality level.

[0103] In this embodiment of the invention, Y takes values ​​of 1, 2, 3, and 4, representing image quality levels of level 1, level 2, level 3, and level 4, respectively, wherein:

[0104] Level 1 indicates that the image texture is clear: the typical surface features of ground objects are highly recognizable, and the roof texture and other outline details of large man-made buildings can be clearly identified. This type of image can be directly used for image processing work such as hyperspectral remote sensing image fusion, and can also be directly used in the field of remote sensing to distinguish ground object categories and other information.

[0105] Level 2 indicates that the usable area of ​​the image is higher than 50%, and the surface features in these areas are easily identifiable. Less than 50% of the image area has some lost texture information, and contour lines and classification information cannot be accurately determined in these areas. This type of image can be used for image processing tasks such as hyperspectral remote sensing image denoising.

[0106] Level 3 indicates that the usable area of ​​the image is between 10% and 50%, and the texture information in these areas is relatively clear, which meets the requirements of remote sensing applications. Most image areas have a large amount of lost texture information, making it difficult to obtain accurate classification information and identify the contours of surface features in these areas. Such images can be used for image processing tasks such as hyperspectral remote sensing image denoising.

[0107] Level 4 indicates that the usable area of ​​the image is less than 10%, texture information is significantly lost in the entire image, and surface features in the image are usually not recognizable by the human eye. Such images have no value in practical applications and should be discarded directly.

[0108] Table 1 compares the performance of the model described in this embodiment with existing models. In Table 1, ssfeFormer represents the quality assessment model described in this embodiment. As can be seen from Table 1, the model described in this embodiment outperforms existing network models in overall accuracy (OA), average accuracy (AA), and Kappa coefficient. Table 2 shows the ablation experiment results. In Table 2, 1 represents the spatial-spectral attention module, 2 represents the spectral integration embedding module, and 3 represents the cross-layer fusion residual module. The ablation experiment results demonstrate that the three main modules proposed in this invention all contribute to the overall accuracy of the network model.

[0109] Table 1. Comparison of model performance between the model described in this embodiment and existing models.

[0110]

[0111] Table 2 Ablation Experiment Results

[0112]

[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for evaluating the quality level of hyperspectral remote sensing images, characterized in that, Includes the following steps: S1: Construct a hyperspectral remote sensing image dataset and divide the hyperspectral remote sensing image dataset into a training set and a test set; S2: Perform a segmentation operation on the hyperspectral remote sensing images in the hyperspectral remote sensing image dataset to segment the hyperspectral remote sensing images into sub-image blocks of fixed size; S3: Construct a quality assessment model, which includes a Vision Transformer model, a spatial-spectral attention module, a spectral integration embedding module, and a cross-layer fusion residual module; the Vision Transformer model is the basic skeleton of the quality assessment model, which includes T encoders, T ≥ 4, and each encoder includes a multi-head self-attention mechanism and a feedforward neural network for extracting image features; The spatial spectrum attention module and the spectrum integration embedding module are sequentially set at the front end of the encoder of the Vision Transformer model, and the cross-layer fusion residual module is set between the Vision Transformer encoders. S4: Input all sub-image blocks into the spatial-spectral attention module to extract feature maps with spectral weights; the spatial-spectral attention module performs max pooling and average pooling operations on the input image, learns the channel weights through two fully connected layers, and avoids gradient vanishing through the GELU activation function. Finally, the output of the GELU function is multiplied by the original input to obtain the feature map with spectral weights as the output of the spatial-spectral attention module. S5: Input the feature map with spectral weights into the spectral integration embedding module to obtain a locally adjacent spectral band grouped embedding feature map; the spectral integration embedding module flattens the feature map with spectral weights of C channels and integrates them into groups of n adjacent channels to obtain... The feature map after grouping, that is, the feature map embedded by grouping local adjacent spectral bands; S6: Group adjacent spectral bands into the feature map and perform linear projection to obtain the linear projection feature map; S7: Input the linear projection feature map into the T encoders of the Vision Transformer model to perform multi-head self-attention and multilayer perceptron operations; S8: Perform cross-layer fusion residual operation on the encoder through the cross-layer fusion residual module; the cross-layer fusion residual module performs cross-layer fusion residual operation on the encoder. and The output of the encoder as well as Perform splicing and blending operations to obtain the blended output. And a skip connection mechanism is used to fuse the output. Input into the (t+2)th encoder; S9: Input the feature map output by the last encoder into the linear layer and activation function layer to obtain the quality assessment result of the hyperspectral remote sensing image; S10: Input the training set and the corresponding subjective quality level into the quality assessment model for training, and use the test set to test the model performance of the quality assessment model; S11: After training is complete, input the hyperspectral remote sensing image to be evaluated into the quality assessment model to obtain the quality level.

2. The method for evaluating the quality level of hyperspectral remote sensing images according to claim 1, characterized in that, In step S2, the hyperspectral remote sensing image is cropped using the sliding window method, dividing the hyperspectral remote sensing image into sub-image blocks of fixed size, and there is no overlapping area between different sub-image blocks.

3. The method for evaluating the quality level of hyperspectral remote sensing images according to claim 2, characterized in that, The specific steps of step S4 are as follows: S401: For input sub-image blocks Its vector dimension is (C, H, W), that is Where R represents the set of real numbers, C represents the number of channels in the image, H represents the height of the image, and W represents the width of the image. The input is a sub-image patch. In pixels Features of the place Its vector dimension is 1×1, and the number of channels is C, that is... , for features The specific expressions for performing average pooling and max pooling operations are as follows: ； ； in, , yes In position The expression of time, This represents the features after global average pooling. This represents the features after the max pooling operation. This indicates taking the maximum region; S402: The features obtained after global average pooling operation in step S401 are... Features after max pooling operation The input is fed into two fully connected layers, and the two results from the fully connected layers are added together and then a ReLU activation function is added to obtain the corresponding 1-D spectral attention map. : ； Among them, 1-D spectral attention map The dimension is 1×1, and the number of channels is C, that is... linear transformation The dimension is ,Right now linear transformation The dimension is ,Right now r is a hyperparameter that can be set by the user; the first fully connected layer serves as the basis for... The parameterized dimensionality reduction layer, the second fully connected layer is based on... Parameterized dimension-adding layers, linear transformation and linear transformation Shared weights; Represents the ReLU activation function; S403: Input sub-image block Corresponding 1-D spectral attention map Multiplication yields the output of the spatial spectral attention module, which is the feature map with spectral weights corresponding to the input sub-image y. Its vector dimension is (C, H, W), that is Where R represents the set of real numbers, and represents the set of all real numbers. Represents the multiplication element.

4. The method for evaluating the quality level of hyperspectral remote sensing images according to claim 3, characterized in that, The specific steps of step S5 are as follows: Step S501: Transfer the feature map with spectral weights The 2D planar image of each band is unfolded into the corresponding 1D vector. express: , Indicates the first The 1-D vector of the channel, =1,2,…,C, where the first... 1-D vector of the channel The dimension is ,Right now 1-D vector The dimension is ,Right now ; S502: Convert the 1-D vector The vectors of n adjacent spectral bands are integrated as input to obtain the corresponding local adjacent spectral band grouping embedding feature map. : ； Among them, the feature map is embedded in the grouping of local adjacent spectral bands. The dimension is ,Right now , This indicates a grouping and integration operation. This indicates the floor function.

5. The method for evaluating the quality level of hyperspectral remote sensing images according to claim 4, characterized in that, The specific steps of step S6 are as follows: grouping and embedding the locally adjacent spectral bands obtained in step S502 into the feature map. Perform a linear transformation and use a fully connected layer to group and embed locally adjacent spectral bands into the feature map. Dimensions Dimensional reduction D Dimension, to obtain linear projection feature map Linear projection feature map The dimension is ,Right now , .

6. The method for evaluating the quality level of hyperspectral remote sensing images according to claim 5, characterized in that, The specific steps of step S7 are as follows: S701: After performing layer normalization on the linear projection feature map X obtained in step S6, the query vector is obtained through linear transformation. Key vector K and value vector V: ； ； ； in, This is the first learnable weight matrix. This is the second learnable weight matrix. This is the third learnable weight matrix; Then, a multi-head self-attention operation is performed to obtain the first... Query vector of heads , No. The key vector of each head and the The value vector of each head : ； ； ； ； ； in, The number of heads in a multi-head self-attention mechanism. This is the fourth learnable weight matrix. This is the fifth learnable weight matrix. This is the sixth learnable weight matrix. This is the seventh learnable weight matrix. This represents the first in the multi-head self-attention mechanism. Size; The attention operation is as follows: , It is the transpose of the key vector K. It is a scaling factor. It is a query vector And the dimension of the key vector K; This indicates a bullish self-attention strategy. This indicates a join operation; MSA represents the output of a multi-head self-attention operation. S702: Combine the output MSA and linear projection feature map from the multi-head self-attention operation in step S701. Adding them together yields the feature map. Feature map The dimension is also ( ); and feature map After layer normalization, the input is fed into the multilayer perceptron layer to obtain the output of the multilayer perceptron operation. ;in, It is a layer normalization operation. It is a multilayer perceptron operation; finally, the feature map is... Output of multilayer perceptron operation Add them together to get the first one. The output of the encoder .

7. The method for evaluating the quality level of hyperspectral remote sensing images according to claim 6, characterized in that, In step S8, the first... The and the first The output of the encoder and , No. The output of the encoder and the The output of the encoder All dimensions are ,Right now , d is the dimension of the feature embedding, d = C × H × W; for the th The output of the encoder and the The output of the encoder Perform cross-layer fusion connection operations to obtain the first Fusion output of layer-to-layer fusion residual module : ,in, Represents the learnable network parameters, with dimension 1. ,Right now ; and will the Fusion output of layer-to-layer fusion residual module As the first One of the inputs of the encoder, the first Another input to the encoder is the first... The output of the encoder .

8. The method for evaluating the quality level of hyperspectral remote sensing images according to claim 7, characterized in that, In step S9, the output of the last encoder is... Perform MLP Head operation to obtain the final quality level result of the hyperspectral remote sensing image. ;in, It is the GELU activation function. It is a fully connected layer operation.

Images