Airborne hyperspectral image quality evaluation method and system based on twin network architecture

Abstract

The invention discloses an airborne hyperspectral image quality evaluation method and system based on a twin network architecture. The method comprises the following steps: S1, acquiring an airborne hyperspectral image; s2, preprocessing the hyperspectral image; s3, selecting a sample group set from the preprocessed hyperspectral image, selecting an average value of the sample group set as a standard group, extracting a spectral index feature vector and a radiation index feature vector of each sample group, and converting the spectral index feature vector and the radiation index feature vector into a fan-shaped scattered image; s4, forming image pairs by the fan-shaped scattered images of the standard group and the fan-shaped scattered images of each sample group in sequence, and inputting the image pairs into a twin convolutional network model to obtain a similarity evaluation result of each group of image pairs; wherein the twin convolutional network model adopts a twin network and a convolutional neural network; and S5, obtaining a quality evaluation result of the hyperspectral image based on the similarity evaluation result of each group of image pairs. The invention provides a simple and convenient no-reference image quality evaluation method based on a twin convolutional network.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image quality evaluation, and in particular to an airborne hyperspectral image quality evaluation method and system based on a twin network architecture. Background Art

[0002] In image quality evaluation, mainstream methods are mostly divided into two categories. The first category mostly builds models based on feature indicators, sets thresholds to evaluate images, and uses fuzzy matrices and weighted synthesis algorithms to determine the evaluation thresholds, and then evaluates the image quality. In this type of method, weight is an important parameter, but the method of determining weight is mostly based on data sets of human judgment. Therefore, there are subjective errors that cannot be eliminated, and it is difficult to maintain consistency in judgment. Invention patent 201110121214.0 “A method for evaluating remote sensing image quality without reference” discloses a method for evaluating the quality and application capability of hyperspectral data, comprising the following steps: (1) determining the constraint relationship between the spatial resolution, spectral resolution, and signal-to-noise ratio of hyperspectral data; (2) using a simulation model based on radiation transmission and imaging process to generate simulated data of different payload performance parameters under the constraint conditions of step (1) under standard atmospheric conditions; (3) using hyperspectral data processing methods based on different principles to perform application analysis; (4) obtaining application processing results under different spatial resolution, spectral resolution, and signal-to-noise ratio payload performance parameters; (5) determining indicators for evaluating the quality and application capability of hyperspectral data; (6) determining the relationship between the evaluation indicators and the spatial resolution, spectral resolution, and signal-to-noise ratio payload performance parameters through statistical analysis and fitting methods, and establishing a data quality and application capability evaluation equation; (7) outputting the data quality and application capability evaluation results.

[0003] The second type of method uses joint spatial-spectral features and 3D convolutional networks to build a model. This type of method uses different levels of labeled datasets for model training to provide image quality grades. The labeled datasets are graded based on manual judgment of the degree of distortion. This method is highly subjective and targeted, and is not suitable for a wide range of hyperspectral image quality evaluations. For example, invention patent 202311848931.2 discloses a "hyperspectral remote sensing image quality grade assessment method," which includes the following steps: dividing the image into blocks of fixed size, then constructing a quality assessment model, obtaining a feature map with spectral weights through a spatial-spectral attention module, and inputting it into a spectral integration embedding module to generate an embedded feature map of local adjacent spectral bands; then, after linear projection, inputting it into an encoder, and setting a jump connection mechanism to fuse the outputs of different encoders; inputting the output of the last encoder into a linear layer and an activation function layer to obtain the image quality assessment result.

[0004] In summary, the above two methods: the weights and thresholds set by the first method rely on manual settings, and the label data sets of the second method are also manually determined, which often have problems such as strong subjectivity and poor practicality. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides an airborne hyperspectral image quality evaluation method and system based on a twin network architecture. The model is constructed using a twin network structure and a convolutional neural network. The image features of samples from the same land type test area are compared with the image features of standard samples in the hyperspectral image, and the image quality is evaluated based on the comparison results.

[0006] The present invention provides an airborne hyperspectral image quality assessment method based on a twin network architecture, the method comprising:

[0007] S1. Acquire airborne hyperspectral images;

[0008] S2. Preprocessing the hyperspectral image;

[0009] S3, selecting a sample set from the pre-processed hyperspectral image, selecting the average value of the sample set as a standard set, extracting the spectral index feature vector and the radiation index feature vector of each sample set, and converting them into a fan-shaped scattered line image;

[0010] S4. Sequentially forming image pairs from the fan-shaped scattered line images of the standard group and the fan-shaped scattered line images of each sample group, and inputting the pairs into a twin convolutional network model to obtain a similarity evaluation result for each image pair; wherein the twin convolutional network model adopts a twin network and a convolutional neural network;

[0011] S5. Based on the similarity evaluation results of each group of image pairs, obtain the quality evaluation results of the hyperspectral image.

[0012] Preferably, in step S1, the spectral resolution of the acquired hyperspectral image is not less than 10 nm, the spectral range is 350 to 2500 nm, and the spatial resolution is better than 5 m.

[0013] Preferably, in step S2, the hyperspectral image is preprocessed by the following method:

[0014] S21, radiometric calibration: input the image to be radiometrically calibrated and the image description file into ENVI software, perform radiometric calibration, and obtain the surface reflectivity after radiometric correction;

[0015] S22, atmospheric correction: After radiometric calibration, atmospheric correction is performed to obtain the atmospherically corrected surface reflectance;

[0016] S23. Orthorectification: Input the image to be orthorectified, the reference image and the DEM data into the ENVI software, output the ground control points and perform orthorectification on them to generate a planar orthoimage.

[0017] Preferably, step S3 includes:

[0018] S31, creating a sample group and a standard group: selecting a single land type from the pre-processed hyperspectral image, cutting out a test area, and dividing the test area into N sample A n (n=1,2,…,N), select the first sample A1 and the nth sample A n Composition of sample group A n A n , select the average value of all sample groups as the standard group A; where i, n, and N are all positive integers;

[0019] S32. Sample group feature information extraction: For each sample group, four spectral index values, namely, spectral angular distance, spectral information divergence, spectral similarity, and Pearson coefficient, are calculated as spectral index feature vectors. Four radiation index values, namely, information entropy, average gradient, contrast, and land enhancement index, are calculated as radiation index feature vectors.

[0020] S33, normalization of indicator feature vectors: normalizing the eight indicator feature vectors of each sample group;

[0021] S34. Visualization of the multi-dimensional characteristic vectors of the sample group: The normalized 8 indicator characteristic vectors of each sample group are interactively arranged and combined according to the two major categories of spectral indicators and radiation indicators, and converted into characteristic scatter lines. A fan-shaped scatter line diagram of the fan-shaped distribution is generated through the polar coordinate system.

[0022] Preferably, step S31 includes:

[0023] Set the image size of the test area to a×a, the sample size to b×b, and the sliding step size to l;

[0024] Set the fixed sample A1 to be a b×b pixel range starting from the upper left corner, slide the sample A1 to the right by l pixels step by step, and obtain horizontal samples A2, A3, A4...A i Similarly, slide sample A1 down by 1 pixel step by step to obtain vertical samples A i+1 、A i+2 …A n ; Wherein, a, b, l, i, n are all positive integers;

[0025] Combine each sample with sample A1 to form sample groups A1A1, A1A2…A1A n ;

[0026] The average value of all sample groups in the test area is taken as standard group A.

[0027] Preferably, in step S34,

[0028] For each normalized eigenvalue f k ∈(0,1)(k=1,2,…,8), define its polar coordinate mapping as:

[0029] r k =f k

[0030]

[0031] Among them, r k Indicates the characteristic scattered line length, θ k represents the angular position of the scattered line between 0 and 180°, and p represents the number of all eigenvectors;

[0032] Through this mapping, the eight indicator feature vectors are projected into eight characteristic scattered lines, which are distributed in a fan-shaped manner to complete the feature vector visualization.

[0033] Preferably, the twin convolutional network model includes a feature extraction model and a similarity evaluation model; the feature extraction model includes two sets of identical convolutional networks and additional convolutional layers with shared weights, which are respectively used to input the fan-shaped scattered line images of the standard group and the sample group, and output corresponding features; the similarity evaluation model includes a non-parametric square layer, a convolutional layer, a fully connected layer and a SoftMax function connected in sequence, which is used to input the two features output by the feature extraction model and output the similarity evaluation result.

[0034] Preferably, step S4 includes:

[0035] The fan-shaped scatter plots of the standard group and each sample group are sequentially formed into image pairs, which are respectively input into the feature extraction model to extract high-dimensional features f1 and f2 of the image pairs;

[0036] Input high-dimensional features f1 and f2 into the similarity evaluation model and output tensor f s , f s =(f1-f2) 2 , after the nonparametric square layer, f s Input the convolution layer to obtain a series of feature maps, which are then input into the fully connected layer for extension processing to obtain a one-dimensional vector. This one-dimensional vector is input into the SoftMax function to obtain a two-dimensional probability distribution vector.

[0037] Preferably, the feature extraction model and the similarity assessment model both adopt cross entropy loss;

[0038] The feature extraction model uses the cross entropy loss L id Expressed as:

[0039]

[0040] Among them, * represents the convolution operation, f represents the feature expression output of the convolutional network, and θ I are the parameters of the additional convolutional layer, is the probability of the predicted category, K is the total number of categories, t is the true target category, and p t is the one-hot encoding vector of the true target category. For the true target category, there is p t =1, other p i =0(i≠t);

[0041] The similarity evaluation model uses the cross entropy loss L ver Expressed as:

[0042]

[0043] Among them, f1 and f2 represent the two features of the similarity evaluation model, θ s are the parameters of the convolutional layer connected to the square layer, s is a two-dimensional vector indicating whether the two images are from the same category, represents the predicted probability, q i Represents the similarity label of an image pair; for an input image pair, if the two are from the same category, then q1 = 1, q2 = 0; otherwise, q1 = 0, q2 = 1.

[0044] Based on the same inventive concept, the present invention also provides an airborne hyperspectral image quality evaluation system based on a twin network architecture, the system comprising:

[0045] Acquisition module, used to acquire airborne hyperspectral images;

[0046] A preprocessing module, used for preprocessing the hyperspectral image;

[0047] a feature image extraction module, configured to select a sample set from the pre-processed hyperspectral image, select the average value of the sample set as a standard set, extract the spectral index feature vector and the radiation index feature vector of each sample set, and convert them into a fan-shaped scattered line image;

[0048] a similarity assessment module, configured to sequentially form image pairs from the standard group of fan-shaped scattered line images and the fan-shaped scattered line images from each sample group, input the pairs into a twin convolutional network model, and obtain a similarity assessment result for each image pair; wherein the twin convolutional network model adopts a twin network and a convolutional neural network;

[0049] The image quality evaluation module is used to obtain a quality evaluation result of the hyperspectral image based on the similarity evaluation result of each group of image pairs.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] The present invention adopts a feature imaging method, selects specific spectral indicators and radiation indicators, and uses completely objective similarity as a specific evaluation parameter to evaluate the quality of airborne hyperspectral images.

[0052] This invention uses a twin network architecture and convolutional neural network to build a model. It converts the spectral and radiation characteristics of images of the same land type into fan-shaped scattered line images. A pair of fan-shaped scattered line images are input into the twin convolutional network model for comparison to determine the image quality level. The higher the similarity between the two, the higher the image quality, indicating high spatial-spectral correlation, good resolution and uniformity, and low spectral heterogeneity. This invention is a reference-free image quality assessment method that uses the characteristic images of the sample set as evaluation indicators to quantify image quality, derive the similarity between the two, and then evaluate image quality. It can be widely used in image classification, agricultural remote sensing, urban planning and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of the flow of an airborne hyperspectral image quality assessment method based on a twin network architecture provided by the present invention;

[0054] Figure 2 The overall flow chart of the airborne hyperspectral image quality evaluation method based on the twin network architecture provided by the present invention;

[0055] Figure 3 A schematic diagram of the verification process of an airborne hyperspectral image quality assessment method based on a twin network architecture provided by the present invention;

[0056] Figure 4 A schematic diagram of the framework of the twin convolutional network model provided by the present invention;

[0057] Figure 5 This is an evaluation result diagram of the airborne hyperspectral image quality evaluation method based on the twin network architecture provided by the present invention. DETAILED DESCRIPTION

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0059] The present invention will be described in further detail below with reference to the accompanying drawings:

[0060] like Figure 1-2 As shown, an embodiment of the present invention provides an airborne hyperspectral image quality assessment method based on a twin network architecture, comprising:

[0061] S1. Acquire airborne hyperspectral images;

[0062] In the embodiment of the present invention, the spectral resolution of the acquired hyperspectral image is not less than 10 nm, the spectral range is 350 to 2500 nm, and the spatial resolution is better than 5 m.

[0063] S2, preprocessing the hyperspectral image;

[0064] In the embodiment of the present invention, in step S2, the hyperspectral image is preprocessed by the following method:

[0065] S21, radiometric calibration: input the image to be radiometrically calibrated and the image description file into ENVI software, perform radiometric calibration, and obtain the surface reflectivity after radiometric correction;

[0066] Using ENVI software, input the image to be radiometrically calibrated and the image description file. In the Toolbox, select Radiometric Correction > Radiometric Calibration. Perform radiometric calibration to obtain the corrected surface reflectance.

[0067] S22, atmospheric correction: After radiometric calibration, atmospheric correction is performed to obtain the atmospherically corrected surface reflectance;

[0068] In the Toolbox, select Radiometric Correction > Atmospheric Correction Module > FLAASH Atmospheric Correction to perform atmospheric correction and obtain the atmospherically corrected surface reflectance.

[0069] S23. Orthorectification: Input the image to be orthorectified, the reference image and the DEM data into the ENVI software, output the ground control points and perform orthorectification on them to generate a planar orthoimage.

[0070] Use ENVI software to input the image to be orthorectified, the reference image and DEM data, output GCPs and perform ortho projection correction to eliminate the influence of terrain or deformation caused by camera orientation, and generate a planar orthophoto.

[0071] S3, selecting a sample set from the pre-processed hyperspectral image, selecting the average value of the sample set as the standard set, extracting the spectral index feature vector and the radiation index feature vector of each sample set, and converting them into a fan-shaped scattered line image;

[0072] In the embodiment of the present invention, step S3 includes:

[0073] S31. Create sample groups and standard groups: Select a single land type from the pre-processed hyperspectral image, crop the test area, and divide the test area into N sample A n (n=1,2,…,N), select the first sample A1 and the nth sample A n Composition of sample group A n A n , select the average value of all sample groups as the standard group A; where i, n, and N are all positive integers;

[0074] In the embodiment of the present invention, step S31 includes:

[0075] Set the image size of the test area to a×a, the sample size to b×b, and the sliding step size to l;

[0076] Set the fixed sample A1 to be a b×b pixel range starting from the upper left corner, slide the sample A1 to the right by l pixels step by step, and obtain horizontal samples A2, A3, A4...A i Similarly, slide sample A1 down by 1 pixel step by step to obtain vertical samples A i+1 、A i+2 …A n ; Wherein, a, b, l, i, n are all positive integers;

[0077] Combine each sample with sample A1 to form sample groups A1A1, A1A2…A1A n ;

[0078] The average value of all sample groups in the test area is taken as standard group A.

[0079] For example, a single land type is selected, and a 20×20 square image of the test area is cropped. The sample group is constructed by sliding pixel by pixel, and the sliding step is set to 1. The fixed sample is a 5×5 sample starting from the upper left corner, which is defined as sample A1. The sample obtained by sliding the range of sample A1 to the right by 1 pixel is defined as A2. A3 is a 5×5 sample obtained by sliding A2 to the right by 1 pixel, and A2, A3, A4...A are obtained in sequence. 16 Similarly, the sample A1 is defined as the sample obtained by sliding 1 pixel downward. 17 , then slide to the right to get A 18 、A 19 、A 20 …A n ; Each sample is combined with sample A1 to form sample groups A1A1, A1A2…A1A n Select standard sample group A and take the average value of all sample groups in the test area as standard sample group A.

[0080] S32. Sample group feature information extraction: For each sample group, four spectral index values, namely, spectral angular distance, spectral information divergence, spectral similarity, and Pearson coefficient, are calculated as spectral index feature vectors. Four radiation index values, namely, information entropy, average gradient, contrast, and land enhancement index, are calculated as radiation index feature vectors.

[0081] S32 includes:

[0082] S321, use ENVI software to input sample A1 and sample A n , select the Compute Statistics from Rols tool to obtain the reflectance of all bands of the two samples and calculate the values ​​of the four spectral indicators. This is sample group A1A n The four spectral index values ​​of .

[0083] The calculation formula of spectral angular distance is:

[0084]

[0085] Among them, x and y represent two spectral curves x=(x1,x2,x3,…,x n ), y=(y1,y2,y3,…,y n ), N is the total number of bands, i, N are positive integers.

[0086] The calculation formula of spectral information divergence is:

[0087] SID(x,y)=D(x‖y)+D(y‖x)

[0088]

[0089] Where D(x‖y) is the relative entropy of y and x, D(y‖x) is the relative entropy of x and y, and i and N are positive integers.

[0090] The calculation formula for spectral similarity is:

[0091]

[0092]

[0093] Among them, σ x is the standard deviation of the spectral vector x, σ y is the standard deviation of the spectral vector y, is the mean of x, is the mean of y, i and N are positive integers.

[0094] The calculation formula of Pearson coefficient is:

[0095]

[0096] Among them, σ x and σ y is the standard deviation of the spectral vectors x and y, and is the mean of x and y, i and N are positive integers.

[0097] S322. Use ENVI software to enter the test area and use File>Save As>Save AS>(ENVI.NITF.TIFF.DTED)>Spectral Subset to export three single-band images of red, green, and blue respectively;

[0098] In the ENVI software, the NDVI index of the test area was calculated to obtain the NDVI image, and the average value of all pixel values ​​in the sample was taken as the radiation index NDVI value;

[0099] In Arcgis software, input the three single-band images of the test area, select ArcToolbox>SpatialAnalyst Tools>Map Algebra>Raster Calculator, use different coefficients for the red, green and blue bands, calculate the normalized grayscale image of the test area, and calculate the three radiation index values ​​of the sample.

[0100] The calculation formula of information entropy is:

[0101]

[0102] Among them, p i is the probability of gray level i appearing in the image, N is the total number of pixels in the image, and i and N are positive integers.

[0103] The calculation formula of the average gradient is:

[0104]

[0105] g′ i =x (i,j) -x (i,j+1)

[0106] g′ j =x (i,j) -x (i+1,j)

[0107] Among them, g′ i , g′ j They represent the horizontal and vertical gradients of the grayscale of the pixel at (i, j) in the discrete image, respectively. i, j, w, h, and N are positive integers.

[0108] The formula for calculating contrast is:

[0109] Contrast=∑ i＝1 ∑ j＝1 (ij) 2 P(ij)

[0110] Where P(ij) is the co-occurrence probability at position (ij) in the gray-level co-occurrence matrix. In this study, the contrast of two directions, 0° and 90°, was calculated and then the average was calculated.

[0111] S33, normalization of indicator feature vectors: normalizing the eight indicator feature vectors of each sample group;

[0112] The four spectral index values ​​are normalized to obtain the four normalized spectral characteristic values. For the four radiation index values, one value is assigned to each sample, and the ratio of the two values ​​of the sample group is calculated to obtain a normalized radiation index value for each group.

[0113] S34. Visualization of the multi-dimensional characteristic vectors of the sample group: The normalized 8 indicator characteristic vectors of each sample group are interactively arranged and combined according to the two major categories of spectral indicators and radiation indicators, and converted into characteristic scatter lines. A fan-shaped scatter line diagram of the fan-shaped distribution is generated through the polar coordinate system.

[0114] In the embodiment of the present invention, in step S34, for each normalized eigenvalue f k ∈(0,1)(k=1,2,…,8), define its polar coordinate mapping as:

[0115] r k =f k

[0116]

[0117] Among them, r k Indicates the characteristic scattered line length, θ k represents the angular position of the scattered line between 0 and 180°, and p represents the number of all eigenvectors;

[0118] Through this mapping, the eight indicator feature vectors are projected into eight characteristic scattered lines, which are distributed in a fan-shaped manner to complete the feature vector visualization.

[0119] S4. The fan-shaped scattered line images of the standard group and the fan-shaped scattered line images of each sample group are sequentially formed into image pairs, and the pairs are input into the twin convolutional network model to obtain a similarity evaluation result for each image pair; wherein the twin convolutional network model adopts a twin network and a convolutional neural network;

[0120] like Figure 4 As shown, in an embodiment of the present invention, the twin convolutional network model includes a feature extraction model and a similarity evaluation model; the feature extraction model includes two sets of identical convolutional networks and additional convolutional layers with shared weights, which are respectively used to input fan-shaped scattered line images of the standard group and the sample group, and output corresponding features; the similarity evaluation model includes a non-parametric square layer, a convolutional layer, a fully connected layer and a SoftMax function connected in sequence, which are used to input two features output by the feature extraction model and output a similarity evaluation result.

[0121] In the embodiment of the present invention, step S4 includes:

[0122] The fan-shaped scatter plots of the standard group and each sample group are sequentially formed into image pairs, which are input into the feature extraction model to extract high-dimensional features f1 and f2 of the image pairs;

[0123] Input high-dimensional features f1 and f2 into the similarity evaluation model and output tensor f s , f s =(f1-f2) 2 , after the nonparametric square layer, f s Input the convolution layer to obtain a series of feature maps, which are then input into the fully connected layer for extension processing to obtain a one-dimensional vector. This one-dimensional vector is input into the SoftMax function to obtain a two-dimensional probability distribution vector.

[0124] After the non-parametric square layer, add a convolution layer and SoftMax function to f s Embedded into a two-dimensional vector, which represents the predicted probability of the input feature image pair belonging to the same category. The SoftMax function is used to normalize the output of the final two-dimensional vector.

[0125] In the embodiment of the present invention, both the feature extraction model and the similarity evaluation model adopt cross entropy loss.

[0126] The cross entropy loss L used by the feature extraction model id Expressed as:

[0127]

[0128] Among them, * represents the convolution operation, f represents the feature expression output of the convolutional network, and θ I is the parameter of the additional convolutional layer, K is the total number of categories; t is the true target category, such as t = 2 represents the second category, and p is the one-hot encoding vector of the true target category (p t =1, other p i =0(i≠t)), is the probability of the predicted class.

[0129] The cross entropy loss L used by the similarity evaluation model ver Expressed as:

[0130]

[0131] Among them, f1 and f2 represent the two features of the input similarity evaluation model, θ s are the parameters of the convolutional layer connected to the square layer; s is a two-dimensional vector indicating whether the two images are from the same category, represents the predicted probability, q i Represents the similarity label of an image pair; for an input image pair, if the two are from the same category, then q1 = 1, q2 = 0; otherwise, q1 = 0, q2 = 1.

[0132] S5. Based on the similarity evaluation results of each image pair, the quality evaluation results of the hyperspectral image are obtained.

[0133] like Figure 3 As shown in the figure, the output result of the twin convolutional network model is 1 if they are similar, otherwise it is 0, thereby obtaining the similarity value of the sample set in the test area. The more 1 values ​​there are, the higher the image consistency and the better the image quality.

[0134] Example 1:

[0135] Acquire airborne hyperspectral image data, ensure that the image meets the spatial and spectral resolution requirements, perform preprocessing steps such as radiometric calibration, atmospheric correction, and orthorectification on the data, select the Yuncheng area in Shanxi Province, obtain and load hyperspectral images, select farmland as the test area, create standard groups and sample groups, input the created standard groups and sample groups into the model, and obtain image quality evaluation results. The specific steps include:

[0136] Step 1: Acquire airborne hyperspectral data with 660 bands, a spatial resolution of 5 meters, and a band range of 392 to 2493 nm.

[0137] Step 2: Preprocess the hyperspectral image, including radiometric calibration, atmospheric correction, and orthorectification.

[0138] Step 2.1: Radiometric Calibration. Use ENVI software to input the image to be calibrated and the image description file. In the Toolbox, select Radiometric Correction > Radiometric Calibration. Perform radiometric calibration to obtain the corrected surface reflectance.

[0139] Step 2.2: Atmospheric correction. Use ENVI software. In the Toolbox, select Radiometric Correction > Atmospheric Correction Module > FLAASH Atmospheric Correction. Enter the atmospheric model parameters according to the image header information, perform atmospheric correction, and obtain the surface reflectance after atmospheric correction.

[0140] Step 2.3: Orthorectification. Use ENVI software. In the Toolbox, select Geometric Correction > Orthorectification > RPC Orthorectification. Input the image to be orthorectified, the reference image, and the DEM data. Output the GCPs and perform orthorectification to generate a planar orthoimage.

[0141] Step 3, selecting a sample set and creating a feature image, includes the following steps:

[0142] 3.1. Select the farmland type and crop a 20×20 square image of the experimental area. Sample groups are constructed using a pixel-by-pixel sliding method with a sliding step of 1. A fixed sample is a 5×5 sample starting from the upper left corner, defined as sample A1. Sample A1 is slid one pixel to the right to obtain sample A2. A3 is the 5×5 sample of A2 slid one pixel to the right. A3 is the 5×5 sample of A2 slid one pixel to the right, and so on. Similarly, sample A is slid one pixel downward to obtain sample A17. Sliding right again to obtain sample A18, A19, A20, …An. Each sample forms sample groups A1A1, A1A2, …A1A256 with sample A1, sample groups A1A1, A1A2, …A1A256 with sample group A2, and sample groups AnA1, AnA2, …AnA256 with sample group An. After removing duplicate sample groups, all sample group sets were obtained, totaling 32,640 sample groups, and their average values ​​were calculated as standard sample group A.

[0143] 3.2.1. Use ENVI software to input the test area image, open the Region of Interest (ROI) tool, select the range of the sample set as the region of interest, use the Compute Statistics from Rols tool to obtain the reflectance of all bands of the two samples, and calculate the values ​​of the four spectral indicators.

[0144] Table 1 Values ​​of the first 10 spectral indicators of the sample set

[0145] Sample Group Spectral angular distance Spectral information divergence Spectral similarity Pearson coefficient AA01 0.027234802 0.000312318 0.042155921 0.984478231 AA02 0.016945461 0.000128975 0.022807587 0.994546083 AA03 0.020687495 0.000175121 0.020421048 0.990877634 AA04 0.023019184 9.36045E-06 0.028329976 0.989414578 AA05 0.014359657 9.0658E-05 0.013228548 0.995809935 AA06 0.01508789 9.68307E-05 0.01067738 0.994886562 AA07 0.039460736 0.000626159 0.031444163 0.985072362 AA08 0.077261532 0.002473026 0.124869552 0.936501114 AA09 0.057660926 0.001401479 0.070212727 0.965056589 AA010 0.079112838 0.00210399 0.154428868 0.920222339

[0146] 3.2.2. Using ENVI software, enter the test area and use File>Save As>Save AS>(ENVI.NITF.TIFF.DTED)>Spectral Subset to export three single-band images of red, green, and blue. Select Band 105 (639 nm) for red, Band 68 (550 nm) for green, and Band 34 (470 nm) for blue.

[0147] In ENVI software, calculate the NDVI index of the test area by using the tool Toolbox>Band Algebra>BandMath and entering the expression in the Enter an expression box:

[0148] (float(b4)-b3) / (b4+b3)

[0149] Among them, b4 is the near-infrared band, select Band181 (820nm), b3 is the red band, select Band105 (639nnm). Pay attention to the data type here. The data type used this time is integer, so data type conversion is required. Just convert one of the variables to floating point type.

[0150] Obtain the NDVI image of the experimental area, select the sample set range in Arcgis software, and calculate the two NDVI means for each group of samples.

[0151] In ArcGIS, input the three single-band images of the test area, select ArcToolbox>SpatialAnalyst Tools>Map Algebra>Raster Calculator, and enter the formula:

[0152] "R"*0.299+"G"*0.587+"R"*0.114

[0153] Among them, R represents the red single-band image, and 0.299 is the red band coefficient; G represents the green single-band image, and 0.587 is the green band coefficient; B represents the blue single-band image, and 0.114 is the blue band coefficient.

[0154] The normalized grayscale image of the test area is calculated and the radiation index value of the sample is calculated.

[0155] Table 2 Values ​​of the first 10 radiation indicators in the sample set

[0156] Category Group Information entropy Average gradient Contrast Vegetation Index AA01 4.221 2.009 16.411 0.383 AA02 4.734 2.606 17.506 0.439 AA03 3.951 3.648 31.060 0.194 AA04 5.954 7.399 77.506 0.512 AA05 3.910 4.420 33.030 0.463 AA06 4.730 2.610 21.500 0.299 AA07 4.020 2.236 19.928 0.471 AA08 4.406 3.162 23.064 0.383 AA09 4.159 2.236 26.173 0.430 AA010 4.091 2.550 17.882 0.740

[0157] 3.3. Normalize the spectral index values ​​obtained in 3.2 (Table 1) to obtain normalized spectral characteristic values. Each sample group corresponds to a spectral index value. For the radiation index values ​​(Table 2), the calculation method used will obtain the radiation index values ​​of the two samples in each group. Normalize the ratio of the two radiation index values ​​to obtain a radiation index value for each sample group.

[0158] The sample value f k ∈(0,1)(k=1,2,…,8), define its polar coordinate mapping as:

[0159] r k =f k

[0160]

[0161] Where r k Indicates the characteristic scattered line length, θ k represents the angular position of the scatter line between 0 and 180°, and p represents the total number of eigenvectors. Through this mapping, the 8-dimensional eigenvector is projected into eight characteristic scatter lines, which are distributed in a fan-shaped pattern. This completes the visualization of the eigenvectors and produces a fan-shaped scatter plot for the sample group. This process is implemented using a Python algorithm.

[0162] Step 4: The fan-shaped scatter plots of the standard group and the sample group are sequentially formed into image pairs, which are input into the twin convolutional network model to obtain the similarity evaluation results of the two. If the proportion of sample groups with a similarity of 1 related to sample A1 reaches 85% or above, the similarity result of sample A1 is judged to be 1, otherwise it is 0.

[0163] Step 5: Obtain the image quality evaluation result.

[0164] Figure 5The similarity output is displayed. Blue indicates a sample similarity of 1, and yellow indicates a sample similarity of 0. Of the 256 samples, 5 are 0, and the rest are 1. The result, r, is the ratio of the number of samples with a similarity of 1 to the total number of samples. A quality evaluation scale is set: r values ​​of 0.85 and above indicate excellent image quality; r values ​​between 0.6 and 0.85 indicate good image quality; and r values ​​below 0.6 indicate poor image quality. The test area is the selected farmland type, with r = 0.98, indicating that the imagery is relatively uniform, with low complexity, and high image quality for the farmland type.

[0165] The embodiment of the present invention further provides an airborne hyperspectral image quality assessment system based on a twin network architecture, comprising:

[0166] Acquisition module, used to acquire airborne hyperspectral images;

[0167] Preprocessing module, used to preprocess hyperspectral images;

[0168] The feature image extraction module is used to select a sample set from the pre-processed hyperspectral image, select the average value of the sample set as the standard set, extract the spectral index feature vector and the radiation index feature vector of each sample set, and convert them into a fan-shaped scattered line image;

[0169] A similarity assessment module is used to sequentially combine the fan-shaped scattered line images of the standard group and the fan-shaped scattered line images of each sample group into image pairs, input them into the Siamese convolutional network model, and obtain the similarity assessment results of each image pair; the Siamese convolutional network model uses a Siamese network and a convolutional neural network;

[0170] The image quality evaluation module is used to obtain the quality evaluation result of the hyperspectral image based on the similarity evaluation result of each group of image pairs.

[0171] This paper uses a feature-based imaging approach, selecting four spectral metrics—spectral angular distance, spectral information divergence, spectral similarity, and the Pearson coefficient—and four radiometric metrics—information entropy, average gradient, contrast, and land class enhancement index. It uses completely objective similarity as a specific evaluation parameter to assess the quality of airborne hyperspectral imagery. Similarity is determined using a twin network and deep learning to build a model that determines the similarity between samples. Within the same land class, the higher the similarity between samples, the better the image consistency, and the higher the image quality for that land class.

[0172] When applying deep learning techniques to no-reference image quality assessment, convolutional neural networks (CNNs) are typically used to obtain features at different scales, enabling model training and quality grade / score prediction. A convolutional neural network is a feedforward deep neural network consisting of an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. Within the network architecture, the convolutional and pooling layers are the core modules that implement feature extraction. The input layer first receives input data, and the convolutional layer extracts multi-level features from that data using convolution kernels of varying weights. The pooling layer, or downsampling layer, receives the feature maps output by the convolutional layer for feature selection and filtering. The fully connected layer nonlinearly combines the extracted features and outputs them.

[0173] The twin network was first proposed by Bromley et al. in the early 1990s. It is a "conjoined neural network", that is, the entire twin structure is composed of two neural networks. This "conjoining" is achieved by sharing weights between the two networks. Therefore, the twin network receives two inputs and passes them to the two weight-sharing neural networks to form their respective feature representations. Finally, the same loss function is used to calculate the metric between the feature representations of each network output, which can represent the correlation between the two inputs, thereby evaluating the similarity between them.

[0174] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. An airborne hyperspectral image quality assessment method based on a twin network architecture is characterized by: The method comprises: S1. Acquire airborne hyperspectral images; S2. Preprocessing the hyperspectral image; S3, selecting a sample set from the pre-processed hyperspectral image, selecting the average value of the sample set as a standard set, extracting the spectral index feature vector and the radiation index feature vector of each sample set, and converting them into a fan-shaped scattered line image; S4. Sequentially forming image pairs from the fan-shaped scattered line images of the standard group and the fan-shaped scattered line images of each sample group, and inputting the pairs into a twin convolutional network model to obtain a similarity evaluation result for each image pair; wherein the twin convolutional network model adopts a twin network and a convolutional neural network; S5. Based on the similarity evaluation results of each group of image pairs, obtain the quality evaluation results of the hyperspectral image.

2. The method according to claim 1, characterized in that In step S1, the spectral resolution of the acquired hyperspectral image is not less than 10 nm, the spectral range is 350 to 2500 nm, and the spatial resolution is better than 5 m.

3. The method according to claim 1, characterized in that In step S2, the hyperspectral image is preprocessed by the following method: S21, radiometric calibration: input the image to be radiometrically calibrated and the image description file into ENVI software, perform radiometric calibration, and obtain the surface reflectivity after radiometric correction; S22, atmospheric correction: After radiometric calibration, atmospheric correction is performed to obtain the atmospherically corrected surface reflectance; S23. Orthorectification: Input the image to be orthorectified, the reference image and the DEM data into the ENVI software, output the ground control points and perform orthorectification on them to generate a planar orthoimage.

4. The method according to claim 1, wherein Step S3 includes: S31, creating a sample group and a standard group: selecting a single land type from the pre-processed hyperspectral image, cutting out a test area, and dividing the test area into N sample A n (n=1,2,…,N), select the first sample A1 and the nth sample A n Composition of sample group A n A n , select the average value of all sample groups as the standard group A; where i, n, and N are all positive integers; S32. Sample group feature information extraction: For each sample group, four spectral index values, namely, spectral angular distance, spectral information divergence, spectral similarity, and Pearson coefficient, are calculated as spectral index feature vectors. Four radiation index values, namely, information entropy, average gradient, contrast, and land enhancement index, are calculated as radiation index feature vectors. S33, normalization of indicator feature vectors: normalizing the eight indicator feature vectors of each sample group; S34. Visualization of the multi-dimensional characteristic vectors of the sample group: The normalized 8 indicator characteristic vectors of each sample group are interactively arranged and combined according to the two major categories of spectral indicators and radiation indicators, and converted into characteristic scatter lines. A fan-shaped scatter line diagram of the fan-shaped distribution is generated through the polar coordinate system.

5. The method according to claim 4, characterized in that Step S31 includes: Set the image size of the test area to a×a, the sample size to b×b, and the sliding step size to l; Set the fixed sample A1 to be a b×b pixel range starting from the upper left corner, slide the sample A1 to the right by l pixels step by step, and obtain horizontal samples A2, A3, A4...A i Similarly, slide sample A1 down by 1 pixel step by step to obtain vertical samples A i+1 、A i+2 …A n ; Wherein, a, b, l, i, n are all positive integers; Combine each sample with sample A1 to form sample groups A1A1, A1A2…A1A n ; The average value of all sample groups in the test area is taken as standard group A.

6. The method according to claim 4, characterized in that In step S34, For each normalized eigenvalue f k ∈(0,1)(k=1,2,…,8), define its polar coordinate mapping as: Among them, r k Indicates the characteristic scattered line length, θ k represents the angular position of the scattered line between 0 and 180°, and p represents the number of all eigenvectors; Through this mapping, the eight indicator feature vectors are projected into eight characteristic scattered lines, which are distributed in a fan-shaped manner to complete the feature vector visualization.

7. The method according to claim 1, characterized in that The twin convolutional network model includes a feature extraction model and a similarity evaluation model; the feature extraction model includes two sets of identical convolutional networks with shared weights and an additional convolutional layer, which are respectively used to input the fan-shaped scattered line images of the standard group and the sample group and output corresponding features; The similarity evaluation model includes a non-parametric square layer, a convolutional layer, a fully connected layer and a SoftMax function connected in sequence, which is used to input the two features output by the feature extraction model and output a similarity evaluation result.

8. The method according to claim 7, characterized in that Step S4 includes: The fan-shaped scatter plots of the standard group and each sample group are sequentially formed into image pairs, which are respectively input into the feature extraction model to extract high-dimensional features f1 and f2 of the image pairs; Input high-dimensional features f1 and f2 into the similarity evaluation model and output tensor f s , f s =(f1-f2) 2 , after the nonparametric square layer, f s Input the convolution layer to obtain a series of feature maps, which are then input into the fully connected layer for extension processing to obtain a one-dimensional vector. This one-dimensional vector is input into the SoftMax function to obtain a two-dimensional probability distribution vector.

9. The method according to claim 8, characterized in that The feature extraction model and the similarity evaluation model both adopt cross entropy loss; The feature extraction model uses the cross entropy loss L id Expressed as: Among them, * represents the convolution operation, f represents the feature expression output of the convolutional network, θ I are the parameters of the additional convolutional layer, is the probability of the predicted category, K is the total number of categories, t is the true target category, and p t is the one-hot encoding vector of the true target category. For the true target category, there is p t =1, other p i =0(i≠t); The similarity evaluation model uses the cross entropy loss L ver Expressed as: Among them, f1 and f2 represent the two features of the similarity evaluation model, θ s are the parameters of the convolutional layer connected to the square layer, s is a two-dimensional vector indicating whether the two images are from the same category, represents the predicted probability, q i Represents the similarity label of an image pair; for an input image pair, if the two are from the same category, then q1 = 1, q2 = 0; otherwise, q1 = 0, q2 = 1.

10. An airborne hyperspectral image quality assessment system based on a twin network architecture, used to implement the method according to any one of claims 1 to 9, characterized in that: The system comprises: Acquisition module, used to acquire airborne hyperspectral images; A preprocessing module, used for preprocessing the hyperspectral image; a feature image extraction module, configured to select a sample set from the pre-processed hyperspectral image, select the average value of the sample set as a standard set, extract the spectral index feature vector and the radiation index feature vector of each sample set, and convert them into a fan-shaped scattered line image; a similarity assessment module, configured to sequentially form image pairs from the standard group of fan-shaped scattered line images and the fan-shaped scattered line images from each sample group, input the pairs into a twin convolutional network model, and obtain a similarity assessment result for each image pair; wherein the twin convolutional network model adopts a twin network and a convolutional neural network; The image quality evaluation module is used to obtain a quality evaluation result of the hyperspectral image based on the similarity evaluation result of each group of image pairs.

Images