Airborne hyperspectral image quality evaluation method and system based on twin network architecture

Abstract

The application discloses a method and system for evaluating the quality of airborne hyperspectral images based on a twin network architecture, comprising the following steps: S1, obtaining an airborne hyperspectral image; S2, preprocessing the hyperspectral image; S3, selecting a sample group set from the preprocessed hyperspectral image, taking the average value of the sample group set as a standard group, extracting the spectral index feature vector and the radiation index feature vector of each sample group, and converting them into fan-shaped scatter plot images; S4, sequentially grouping the fan-shaped scatter plot images of the standard group and the fan-shaped scatter plot images of each sample group into image pairs, inputting the image pairs into a twin convolutional network model, and obtaining the similarity evaluation results of each group of image pairs; wherein the twin convolutional network model adopts a twin network and a convolutional neural network; and S5, obtaining the quality evaluation results of the hyperspectral image based on the similarity evaluation results of each group of image pairs. The application provides a simple and convenient no-reference image quality evaluation method based on a twin convolutional network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hyperspectral image quality evaluation, in particular to an airborne hyperspectral image quality evaluation method and system based on a twin network architecture. BACKGROUND

[0002] In image quality evaluation, the mainstream methods are mainly divided into two categories. The first category is mainly based on feature indicators to construct a model, set a threshold to evaluate the image, and use fuzzy matrix and weighted synthesis algorithm to determine the evaluation threshold, and then evaluate the image quality. In this method, the weight is an important parameter, but the method of determining the weight is mainly based on the data set of human judgment, so there is a subjective error that cannot be eliminated, and it is difficult to maintain the consistency of the judgment. The invention patent 201110121214.0 discloses a method for evaluating the quality and application ability of hyperspectral data, which includes the following steps: (1) determining the constraint relationship among spatial resolution, spectral resolution and signal-to-noise ratio of hyperspectral data; (2) generating simulated data of different load performance parameters under the constraint condition of step (1) by using a simulation model based on radiation transmission and imaging process under standard atmospheric conditions; (3) applying different principles of hyperspectral data processing method for application analysis; (4) obtaining application processing results under different spatial resolution, spectral resolution and signal-to-noise ratio load performance parameters; (5) determining the index for evaluating the quality and application ability of hyperspectral data; (6) determining the relationship between the evaluation index and the spatial resolution, spectral resolution and signal-to-noise ratio load performance parameters by statistical analysis and fitting method, and establishing the data quality and application ability evaluation equation; (7) outputting the data quality and application ability evaluation results.

[0003] The second category of methods uses spatial-spectral joint features and 3D convolutional networks to construct a model. This method uses different levels of label data sets to train the model, and then gives the image quality level. The label data set is divided into levels based on artificial judgment of distortion degree, which has strong subjectivity and pertinence, and is not suitable for wide range of hyperspectral image quality evaluation. For example, the invention patent 202311848931.2 discloses a method for evaluating the quality level of hyperspectral remote sensing images, which includes the following steps: dividing the image into fixed-size blocks, then constructing a quality evaluation model, obtaining a feature map with spectral weight through a spatial-spectral attention module, and inputting it into a frequency spectrum integration embedding module to generate an embedding feature map of local adjacent spectral bands; after linear projection, input into the encoder, and set the jump connection mechanism to fuse the output of different encoders; input the output of the last encoder into the linear layer and the activation function layer to obtain the quality evaluation result of the image.

[0004] In summary, the two methods above: the first method sets the weight and threshold value depending on human setting, and the second method's label data set is also determined by human, which has strong subjectivity and poor practicability. SUMMARY

[0005] In view of the deficiencies in the prior art, the present application provides an airborne hyperspectral image quality evaluation method and system based on a twin network architecture, which uses a twin network structure and a convolutional neural network to construct a model, compares the image features of the same ground class test area samples with the image features of the standard samples in the hyperspectral image, and evaluates the image quality through the comparison results.

[0006] The present application provides an airborne hyperspectral image quality evaluation method based on a twin network architecture, which comprises:

[0007] S1, acquiring an airborne hyperspectral image;

[0008] S2, preprocessing the hyperspectral image;

[0009] S3, selecting a sample group set from the preprocessed hyperspectral image, taking the average value of the sample group set as a standard group, extracting the spectral index feature vector and the radiation index feature vector of each sample group, and converting them into fan scatter line images;

[0010] S4, sequentially forming image pairs from the fan scatter line images of the standard group and each sample group, inputting the image pairs into a twin convolutional network model, and obtaining the similarity evaluation results of each group of image pairs; wherein the twin convolutional network model uses a twin network and a convolutional neural network;

[0011] S5, obtaining the quality evaluation results of the hyperspectral image based on the similarity evaluation results of each group of image pairs.

[0012] Preferably, in step S1, the spectral resolution of the acquired hyperspectral image is not less than 10nm, the spectral range is 350-2500nm, and the spatial resolution is better than 5m.

[0013] Preferably, in step S2, the hyperspectral image is preprocessed by the following method:

[0014] S21, radiation calibration: input the image to be radiometrically calibrated and the image description file into ENVI software, perform radiation calibration, and obtain the surface reflectance after radiation correction;

[0015] S22, atmospheric correction: after radiation calibration, perform atmospheric correction to obtain the ground surface reflectance after atmospheric correction;

[0016] S23. Orthorectification: Input the image to be orthorectified, the reference image, and the DEM data into the ENVI software, output the ground control points, and perform orthophoto correction on them to generate a planar orthophoto.

[0017] Preferably, step S3 includes:

[0018] S31. Creating Sample and Standard Groups: Select a single land cover type from the preprocessed hyperspectral image, crop out the test area, and divide the test area into N sample groups A. n (n = 1, 2, ..., N), select the 1st sample A1 and the nth sample A n Forming sample group A n A n The average value of all sample groups is selected as the standard group A; where i, n, and N are all positive integers.

[0019] S32. Sample group feature information extraction: For two samples in each sample group, calculate the spectral angular distance, spectral information divergence, spectral similarity and Pearson coefficient as spectral index feature vectors, and calculate the information entropy, average gradient, contrast and land type enhancement index as radiation index feature vectors.

[0020] S33, Indicator Feature Vector Normalization: Normalize the 8 indicator feature vectors for each sample group;

[0021] S34. Visualization of multidimensional feature vectors of sample groups: The eight normalized indicator feature vectors in each sample group are arranged and combined in two categories: spectral indicators and radiometric indicators, and transformed into feature scatter lines. A fan-shaped scatter line plot with a fan-shaped distribution is generated through the polar coordinate system.

[0022] Preferably, step S31 includes:

[0023] Set the image size of the test area to a×a, the sample size to b×b, and the sliding step size to l;

[0024] Set a fixed sample A1 as a b×b pixel range starting from the top left corner. Slide sample A1 gradually to the right by l pixels to obtain horizontal samples A2, A3, A4…A… i Similarly, by sliding sample A1 downwards by l pixels, vertical samples A are obtained sequentially. i+1 A i+2 …A n Where a, b, l, i, and n are all positive integers;

[0025] Each sample is grouped with sample A1 to form sample groups A1A1, A1A2, ... A1A1. n ;

[0026] The average value of all sample groups in the test area is taken as the standard group A.

[0027] Preferably, in step S34,

[0028] For each normalized feature value f k ∈(0, 1) (k = 1, 2, …, 8), its polar coordinate mapping is defined as:

[0029] r k = f k

[0030]

[0031] wherein r k represents the feature scatter line length, θ k represents the angular position of the scatter line between 0-180°, and p represents the number of all feature vectors;

[0032] Through the mapping, the eight index feature vectors are projected as eight feature scatter lines, which are distributed in a fan-shaped manner, and the feature vector image is completed.

[0033] Preferably, the twin convolutional neural network model comprises a feature extraction model and a similarity evaluation model; the feature extraction model comprises two groups of identical convolutional networks sharing weights and an additional convolutional layer, which are respectively used for inputting the fan-shaped scatter line images of the standard group and the sample group set, and outputting corresponding features; the similarity evaluation model comprises a non-parametric square layer, a convolutional layer, a fully connected layer and a SoftMax function connected in sequence, which are used for inputting two features output by the feature extraction model, and outputting a similarity evaluation result.

[0034] Preferably, step S4 comprises:

[0035] The fan-shaped scatter line images of the standard group and each sample group are sequentially formed into image pairs, which are respectively input into the feature extraction model to extract high-dimensional features f1 and f2 of the image pairs;

[0036] The high-dimensional features f1 and f2 are input into the similarity evaluation model to output a tensor f s , f s = (f1-f2) 2 After the non-parametric square layer, f s is input into the convolutional layer to obtain a series of feature maps, then input into the fully connected layer for extension processing to obtain a one-dimensional vector, and the one-dimensional vector is input into the SoftMax function to obtain a two-dimensional probability distribution vector.

[0037] Preferably, the feature extraction model and the similarity evaluation model both adopt cross-entropy loss;

[0038] The cross-entropy loss L used by the feature extraction model id is expressed as:

[0039]

[0040] wherein * represents convolution operation, f represents the feature expression output of the convolution network, and θ I is the parameter of the additional convolution layer, is the possibility of the predicted category, K is the total number of categories, t is the real target category, and p t is the one-hot encoding vector of the real target category, for the real target category, p t =1, and p i =0 (i≠t).

[0041] The cross-entropy loss L used by the similarity evaluation model ver is expressed as:

[0042]

[0043] wherein f1 and f2 respectively represent two features input into the similarity evaluation model, and θ s is the parameter of the convolution layer connected with the square layer, and s is a two-dimensional vector representing whether two images come from the same category, represents the predicted probability, and q i represents the similarity label of the image pair; for an input image pair, if both come from the same category, q1=1 and q2=0; otherwise, q1=0 and q2=1.

[0044] Based on the same inventive concept, the application also provides an airborne hyperspectral image quality evaluation system based on a twin network architecture, which comprises:

[0045] An acquisition module for acquiring an airborne hyperspectral image;

[0046] A preprocessing module for preprocessing the hyperspectral image;

[0047] A feature image extraction module for selecting a sample group set from the preprocessed hyperspectral image, selecting the average value of the sample group set as a standard group, extracting the spectral index feature vector and the radiation index feature vector of each sample group, and converting them into fan-shaped scatter line images;

[0048] A similarity evaluation module for sequentially grouping the fan-shaped scatter line image of the standard group and the fan-shaped scatter line image of each sample group into image pairs, inputting them into a twin convolution network model, and obtaining the similarity evaluation result of each image pair; wherein the twin convolution network model adopts a twin network and a convolution neural network;

[0049] An image quality evaluation module is configured to obtain a quality evaluation result of the hyperspectral image based on the similarity evaluation result of each group of image pairs.

[0050] Compared with the prior art, the present application has the following advantages:

[0051] The present application adopts a feature image method, selects specific spectral indicators and radiation indicators, and uses completely objective similarity as a specific evaluation parameter to evaluate the quality of airborne hyperspectral images.

[0052] The present application adopts a twin network architecture and a convolutional neural network to construct a model, converts the spectral features and radiation features of images of the same land class into fan-shaped scatter line images, inputs a pair of fan-shaped scatter line images into a twin convolutional network model for comparison to make a level determination of the image quality, and the higher the similarity between the two, the higher the image spectral correlation, the better the resolution and uniformity, and the lower the same object and different spectrum, and the higher the image quality. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 A flowchart of the airborne hyperspectral image quality evaluation method based on the twin network architecture provided by the present application is shown in the figure.

[0054] Figure 2 A general flowchart of the airborne hyperspectral image quality evaluation method based on the twin network architecture provided by the present application is shown in the figure.

[0055] Figure 3 A verification process diagram of the airborne hyperspectral image quality evaluation method based on the twin network architecture provided by the present application is shown in the figure.

[0056] Figure 4 A framework diagram of the twin convolutional network model provided by the present application is shown in the figure.

[0057] Figure 5 An evaluation result diagram of the airborne hyperspectral image quality evaluation method based on the twin network architecture provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] The present invention will now be described in further detail with reference to the accompanying drawings:

[0060] like Figures 1-2 As shown, this embodiment of the invention provides an airborne hyperspectral image quality assessment method based on a twin network architecture, including:

[0061] S1. Acquire airborne hyperspectral imagery;

[0062] In this embodiment of the invention, the spectral resolution of the acquired hyperspectral image is not less than 10 nm, the spectral range is 350–2500 nm, and the spatial resolution is better than 5 m.

[0063] S2. Preprocess the hyperspectral image;

[0064] In this embodiment of the invention, in step S2, the hyperspectral image is preprocessed using the following method:

[0065] S21. Radiometric calibration: Input the image to be calibrated and the image description file into ENVI software, perform radiometric calibration, and obtain the surface reflectance after radiometric correction.

[0066] Using ENVI software, input the image to be radiometrically calibrated and the image description file. In the Toolbox, select Radiometric Correction > Radiometric Calibration. Perform radiometric calibration to obtain the radiometrically corrected surface reflectance.

[0067] S22. Atmospheric Correction: After radiometric calibration, atmospheric correction is performed to obtain the atmospherically corrected surface reflectance.

[0068] In the Toolbox, select Radiometric Correction > Atmospheric Correction Module > FLAASH Atmospheric Correction to perform atmospheric correction and obtain the atmospherically corrected surface reflectance.

[0069] S23, ortho correction: input the image to be ortho corrected, the reference image and DEM data into ENVI software, output ground control points and ortho projection correction is carried out to generate a plane ortho image.

[0070] Using ENVI software, input the image to be ortho corrected, the reference image and DEM data, output GCPs and ortho projection correction is carried out to eliminate the influence of terrain or deformation caused by camera orientation, etc., and generate a plane ortho image.

[0071] S3, select a sample group set from the pretreated hyperspectral image, select the average value of the sample group set as the standard group, extract the spectral index feature vector and the radiation index feature vector of each sample group, and convert them into a fan-shaped scatter plot image;

[0072] In the embodiment of the application, step S3 comprises:

[0073] S31, creating a sample group and a standard group: selecting a single land class from the pretreated hyperspectral image, cropping a test area, dividing the test area into N samples A n (n=1, 2, …, N), selecting the first sample A1 and the nth sample A n to form a sample group A n A n , and selecting the average value of all sample groups as a standard group A; wherein i, n and N are positive integers;

[0074] In the embodiment of the application, step S31 comprises:

[0075] Setting the image size of the test area as a x a, setting the sample size as b x b, and setting the sliding step as l;

[0076] Setting the fixed sample A1 as a b x b pixel range with the upper left corner as the starting point, sliding the sample A1 to the right by l pixels, and sequentially obtaining the horizontal samples A2, A3, A4…A i ; similarly, sliding the sample A1 downward by l pixels, and sequentially obtaining the vertical samples A i+1 , A i+2 …A n ; wherein a, b, l, i and n are positive integers;

[0077] Forming sample groups A1A1, A1A2…A1A n from each sample and the sample A1 respectively;

[0078] Taking the average value of all sample groups in the test area as the standard group A.

[0079] For example, a single land type is selected, and a 20×20 square image of the test area is cropped. A pixel-by-pixel sliding method is used to construct sample groups, with a sliding step size of 1. A fixed sample is a 5×5 sample starting from the top left corner, defined as sample A1. The sample obtained by sliding one pixel to the right of sample A1 is defined as A2, A3 is a 5×5 sample obtained by sliding one pixel to the right of A2, and so on, resulting in A2, A3, A4…A… 16 Similarly, the sample obtained by sliding down the range of sample A1 by 1 pixel is defined as A. 17 Then slide to the right to get A in sequence. 18 A 19 A 20 …A n Each sample is combined with sample A1 to form sample groups A1A1, A1A2...A1A1. n Select standard sample group A, and take the average value of all sample groups in the test area as standard sample group A.

[0080] S32. Sample group feature information extraction: For two samples in each sample group, calculate the spectral angular distance, spectral information divergence, spectral similarity and Pearson coefficient as spectral index feature vectors, and calculate the information entropy, average gradient, contrast and land type enhancement index as radiation index feature vectors.

[0081] S32 includes:

[0082] S321. Using ENVI software, input sample A1 and sample A. n The Compute Statistics from Rols tool was used to obtain the reflectance of all bands for both samples, and the values ​​of four spectral indices were calculated. This is sample group A1A. n The values ​​of the four spectral indicators.

[0083] The formula for calculating the spectral angular distance is:

[0084]

[0085] Where x and y represent two spectral curves x = (x1, x2, x3, ..., xy) n ), y = (y1, y2, y3, ..., y n ), where N is the total number of bands, and i and N are positive integers.

[0086] The formula for calculating spectral information divergence is:

[0087] SID(x,y)=D(x‖y)+D(y‖x)

[0088]

[0089] where D(x||y) is the relative entropy of y and x, D(y||x) is the relative entropy of x and y, i, N are positive integers.

[0090] The formula for calculating spectral similarity is:

[0091]

[0092]

[0093] where σ x is the standard deviation of the spectral vector x, σ y is the standard deviation of the spectral vector y, is the mean value of x, is the mean value of y, i, N are positive integers.

[0094] The formula for calculating Pearson coefficient is:

[0095]

[0096] where σ x and σ y are the standard deviations of the spectral vectors x and y, and are the mean values of x and y, i, N are positive integers.

[0097] S322, using ENVI software, input the test area, using File>Save As>Save AS>(ENVI.NITF.TIFF.DTED)>Spectral Subset, respectively export red, green, blue three single-band images;

[0098] In ENVI software, calculate the NDVI index of the test area, get the NDVI image, take the average value of all pixel values in the sample as the radiation index NDVI value;

[0099] In Arcgis software, input the three single-band images of the test area, select ArcToolbox>SpatialAnalyst tool>map algebra>grid calculator, use different coefficients for red, green and blue bands respectively, calculate the normalized gray scale image of the test area, calculate the 3 radiation index values of the sample.

[0100] The formula for calculating information entropy is:

[0101]

[0102] where p i is the probability of the gray level i appearing in the image, N is the total number of pixels in the image, i, N are positive integers.

[0103] The calculation formula of the average gradient is:

[0104]

[0105] g′ i =x( i,j )-x( i,j+1 )

[0106] g′ j =x( i,j )-x( i+1,j )

[0107] wherein g′ i , g′ j respectively represent the horizontal gradient and the vertical gradient of the pixel gray scale at the discrete image (i, j), i, j, w, h, N are positive integers.

[0108] The calculation formula of the contrast is:

[0109] Contrast=∑ i＝1 ∑ j＝1 (i-j) 2 P(i-j)

[0110] wherein P(i-j) is the co-occurrence probability at the position (i-j) in the gray scale co-occurrence matrix. In the present research, the contrast in two directions 0° and 90° is calculated, and then the average value is calculated.

[0111] S33, index characteristic vector normalization: the 8 index characteristic vectors of each sample group are normalized;

[0112] The 4 spectral index values are normalized to obtain the normalized 4 spectral characteristic values. For the 4 radiation index values, one value corresponding to each sample is calculated, and the two values of the sample group are calculated by ratio to obtain one normalized radiation index value of each group.

[0113] S34, sample group multi-dimensional characteristic vector imaging: the normalized 8 index characteristic vectors in each sample group are alternately arranged and combined according to the two categories of spectral index and radiation index, and are converted into characteristic scatter lines, and a fan-shaped scatter line graph of fan-shaped distribution is generated through a polar coordinate system.

[0114] In the embodiment of the present application, in step S34, for each normalized characteristic value f k ∈(0,1)(k=1,2,…,8), the polar coordinate mapping is defined as:

[0115] r k =f k

[0116]

[0117] Where, r k θ represents the length of the characteristic scatter line. k This indicates the angular position of the scatter line between 0 and 180°, and p represents the number of all eigenvectors.

[0118] Through this mapping, the 8 indicator feature vectors are projected into 8 feature lines, which are distributed in a fan-shaped manner, thus completing the visualization of the feature vectors.

[0119] S4. The fan-shaped scattering images of the standard group and the fan-shaped scattering images of each sample group are sequentially combined into image pairs, which are then input into the Siamese convolutional network model to obtain the similarity evaluation results of each image pair; wherein, the Siamese convolutional network model adopts Siamese network and convolutional neural network.

[0120] like Figure 4 As shown in the embodiment of the present invention, the Siamese convolutional network model includes a feature extraction model and a similarity evaluation model. The feature extraction model includes two identical convolutional networks with shared weights and an additional convolutional layer, which are used to input the fan-shaped scatter images of the standard group and the sample group, respectively, and output the corresponding features. The similarity evaluation model includes a non-parametric squared layer, a convolutional layer, a fully connected layer and a SoftMax function connected in sequence, which are used to input the two features output by the feature extraction model and output the similarity evaluation result.

[0121] In this embodiment of the invention, step S4 includes:

[0122] The pie charts of the standard group and each sample group are sequentially combined into image pairs, which are then input into the feature extraction model to extract the high-dimensional features f1 and f2 of the image pairs.

[0123] The high-dimensional features f1 and f2 are input into the similarity evaluation model, and the output tensor f is... s f s =(f1-f2) 2 After the nonparametric squared layer, f s The input to the convolutional layer yields a series of feature maps, which are then fed into the fully connected layer for stretching, resulting in a one-dimensional vector. This one-dimensional vector is then input into the SoftMax function to obtain a two-dimensional probability distribution vector.

[0124] After the nonparametric squared layer, add a convolutional layer and a SoftMax function to f s The features are embedded into a two-dimensional vector, which represents the predicted probability of input feature image pairs belonging to the same category. The SoftMax function is used to normalize the output of the final two-dimensional vector.

[0125] In this embodiment of the invention, both the feature extraction model and the similarity evaluation model employ cross-entropy loss.

[0126] The feature extraction model uses cross-entropy loss L id Represented as:

[0127]

[0128] Where * denotes convolution operation, f represents the feature representation output of the convolutional network, and θ I These are the parameters of the additional convolutional layer, K is the total number of classes; t is the true target class, such as t=2 indicating class 2, and p is the one-hot encoded vector of the true target class (p t =1, other p i =0 (i≠t)), It predicts the likelihood of a category.

[0129] The similarity evaluation model uses cross-entropy loss L ver Represented as:

[0130]

[0131] Where f1 and f2 represent two features of the input similarity evaluation model, θ s These are the parameters of the convolutional layer connected to the squared layer; s is a two-dimensional vector representing whether two images belong to the same category. q represents the predicted probability. i The similarity label represents the image pair; for an input image pair, if both images belong to the same category, then q1 = 1 and q2 = 0; otherwise, q1 = 0 and q2 = 1.

[0132] S5. Based on the similarity evaluation results of each image pair, the quality evaluation results of the hyperspectral image are obtained.

[0133] like Figure 3 As shown, the output of the twin convolutional network model is 1 if similar and 0 otherwise. This gives the similarity value of the sample set in the test area. The more 1s there are, the higher the consistency of the images and the better the image quality.

[0134] Example 1:

[0135] Airborne hyperspectral imagery data was acquired, meeting spatial and spectral resolution requirements. Preprocessing steps included radiometric calibration, atmospheric correction, and orthorectification. Hyperspectral images were acquired in Yuncheng, Shanxi Province, with farmland as the experimental area. Standard and sample sets were created and input into the model to obtain image quality evaluation results. The specific steps included:

[0136] Step 1: Acquire airborne hyperspectral data, 660 bands, spatial resolution 5 meters, band range 392–2493 nm.

[0137] Step 2, pre-processing of hyperspectral image, including radiometric calibration, atmospheric correction, orthorectification, step.

[0138] Step 2.1, radiometric calibration, using ENVI software, input the image to be radiometrically calibrated and the image description file, in the Toolbox, select Radiometric Correction > Radiometric Calibration. Perform radiometric calibration to obtain the surface reflectivity after radiometric correction.

[0139] Step 2.2, atmospheric correction, using ENVI software, in the Toolbox, select Radiometric Correction > Atmospheric Correction Module > FLAASH Atmospheric Correction, fill in the atmospheric model parameters according to the image header information, perform atmospheric correction, and obtain the ground surface reflectivity after atmospheric correction.

[0140] Step 2.3, orthorectification, using ENVI software, in the Toolbox, select Geometric Correction > Orthorectification > RPC Orthorectification, input the image to be orthorectified, the reference image and the DEM data, output GCPs and perform orthographic projection correction, generate planar orthographic image.

[0141] Step 3, select sample group set, make feature image, including the following steps:

[0142] 3.1, select farmland class, crop out 20x20 square test area image, use pixel-by-pixel sliding method to construct sample group, sliding step is 1. Fix the sample as the upper left corner of the 5x5 sample, define it as sample A1, the sample obtained by sliding the range of sample A1 to the right by 1 pixel is defined as A2, A3 is the 5x5 sample obtained by sliding the range of A2 to the right by 1 pixel, A3 is the 5x5 sample obtained by sliding the range of A2 to the right by 1 pixel, and so on A2, A3, A4…A16; Similarly, the sample obtained by sliding the range of sample A to the right by 1 pixel is defined as A17, and then sliding to the right in turn to obtain A18, A19, A20…An; Each sample respectively with sample A1 form sample group A1A1, A1A2…A1A256, with sample group A2 form sample group A1A1, A1A2…A1A256, with sample group An form sample group AnA1, AnA2…AnA256. After removing the duplicate sample groups, all sample group sets are obtained, a total of 32640 sample groups, calculate the average value as the standard sample group A.

[0143] 3.2.1, using ENVI software, input the test area image, open the Region of Interest (ROI) Tool, select the sample set range as the region of interest, and get all the band reflectance of the two samples through the Compute Statistics from Rols tool, and calculate the values of the four spectral indicators.

[0144] Table 1: Top 10 spectral indicator values of sample set

[0145] Sample group Spectral angular distance Spectral information divergence Spectral similarity Pearson coefficient AA01 0.027234802 0.000312318 0.042155921 0.984478231 AA02 0.016945461 0.000128975 0.022807587 0.994546083 AA03 0.020687495 0.000175121 0.020421048 0.990877634 AA04 0.023019184 9.36045E-06 0.028329976 0.989414578 AA05 0.014359657 9.0658E-05 0.013228548 0.995809935 AA06 0.01508789 9.68307E-05 0.01067738 0.994886562 AA07 0.039460736 0.000626159 0.031444163 0.985072362 AA08 0.077261532 0.002473026 0.124869552 0.936501114 AA09 0.057660926 0.001401479 0.070212727 0.965056589 AA010 0.079112838 0.00210399 0.154428868 0.920222339

[0146] 3.2.2, using ENVI software, input the test area, use File>Save As>Save AS>(ENVI.NITF.TIFF.DTED)>Spectral Subset to export three single-band images of red, green and blue respectively, red selects Band105 (639nm), green selects Band68 (550nm), and blue selects Band34 (470nm);

[0147] In ENVI software, calculate the NDVI index of the test area, through the tool Toolbox>Band Algebra>BandMath, input the expression in the Enter an expression (expression input box):

[0148] (float(b4)-b3) / (b4+b3)

[0149] Where b4 is the near-infrared band, Band181 (820nm) is selected, and b3 is the red band, Band105 (639nm) is selected. Note that the data type here, the data type used in this case is integer, so you need to convert one of the variables to float type.

[0150] Get the NDVI image of the test area, in Arcgis software, select the sample set range, and calculate the mean value of the two NDVI of each sample group respectively.

[0151] In Arcgis software, input the three single-band images of the test area, select ArcToolbox>SpatialAnalyst tool>map algebra>grid calculator, and input the formula:

[0152] “R”*0.299+“G”*0.587+“R”*0.114

[0153] Wherein, R represents the red single-band image, 0.299 is the red band coefficient; G represents the green single-band image, 0.587 is the green band coefficient; B represents the blue single-band image, 0.114 is the blue band coefficient.

[0154] The normalized gray image of the test area is calculated, and the radiation index value of the sample is calculated.

[0155] Table 2 Top 10 radiation index values of sample set

[0156] Class group Information entropy Average gradient Contrast Vegetation index AA01 4.221 2.009 16.411 0.383 AA02 4.734 2.606 17.506 0.439 AA03 3.951 3.648 31.060 0.194 AA04 5.954 7.399 77.506 0.512 AA05 3.910 4.420 33.030 0.463 AA06 4.730 2.610 21.500 0.299 AA07 4.020 2.236 19.928 0.471 AA08 4.406 3.162 23.064 0.383 AA09 4.159 2.236 26.173 0.430 AA010 4.091 2.550 17.882 0.740

[0157] 3.3, the spectral index value obtained in 3.2 (Table 1) is normalized to obtain the normalized spectral characteristic value, each sample group corresponds to a spectral index value. For the radiation index value (Table 2), the calculation method will obtain the radiation index value of two samples in each group, and the ratio normalization is carried out on the two groups of radiation index values to obtain a radiation index value of each sample group.

[0158] The sample value f k ∈(0,1)(k=1,2,…,8), define its polar coordinate mapping as:

[0159] r k =f k

[0160]

[0161] In the formula, r k represents the characteristic scatter line length, θ k represents the angular position of the scatter line between 0-180°, and p represents the number of all characteristic vectors. Through the mapping, the 8-dimensional characteristic vector is projected as 8 characteristic scatter lines, which are distributed in a fan shape, the characteristic vector is imaged, and the fan-shaped scatter diagram of the sample group is obtained. This process is realized through python algorithm.

[0162] Step 4, the fan-shaped scatter diagrams of the standard group and the sample group set are sequentially composed into image pairs, input into the twin convolution network model, and the similarity evaluation results of the two are obtained. If the proportion of sample groups related to sample A1 with a similarity of 1 reaches 85% or more, it is judged that the similarity result of sample A1 is 1, otherwise it is 0.

[0163] Step 5, the image quality evaluation result is obtained.

[0164] Figure 5The similarity output results are shown, blue represents that the sample similarity result is 1, yellow represents that the sample similarity result is 0, there are 256 samples in total, 5 of which are 0, and the rest are 1. The result r is represented as the ratio of the number of samples with a similarity of 1 to the total number of samples. The quality evaluation classification is set: the r value is 0.85 and above, the image quality evaluation is excellent; the r value is between 0.6 and 0.85, the image quality is good; the r value is below 0.6, the image quality is poor. The test area is the selected farmland type, r = 0.98, which indicates that for the farmland type, the test area image is relatively uniform and has low complexity, and the image quality is high.

[0165] The embodiment of the present application also provides an airborne hyperspectral image quality evaluation system based on a twin network architecture, comprising:

[0166] The acquisition module is used to acquire the airborne hyperspectral image.

[0167] The preprocessing module is used to preprocess the hyperspectral image.

[0168] The feature image extraction module is used to select a sample group set from the preprocessed hyperspectral image, select the average value of the sample group set as a standard group, extract the spectral index feature vector and the radiation index feature vector of each sample group, and convert them into a fan-shaped scatter plot image.

[0169] The similarity evaluation module is used to sequentially form image pairs of the fan-shaped scatter plot image of the standard group and the fan-shaped scatter plot image of each sample group, input the image pairs into a twin convolutional network model, and obtain the similarity evaluation result of each image pair; wherein the twin convolutional network model adopts a twin network and a convolutional neural network.

[0170] The image quality evaluation module is used to obtain the quality evaluation result of the hyperspectral image based on the similarity evaluation result of each image pair.

[0171] The present application adopts a feature image method, selects four spectral indexes of spectral angle distance, spectral information divergence, spectral similarity and Pearson coefficient, and four radiation indexes of information entropy, average gradient, contrast and class enhancement index, adopts a completely objective similarity as a specific evaluation parameter, and evaluates the quality of the airborne hyperspectral image. The similarity is determined by constructing a model by using a twin network and deep learning, so as to determine the similarity between samples. In the same class, the higher the similarity between samples, the better the image consistency, and it is considered that the image quality of the class is high.

[0172] In the aspect of no-reference image quality assessment using deep learning techniques, a Convolutional neural network (CNN) is usually used to obtain features of different scales to realize the training of the model and the prediction of quality grades / scores. The convolutional neural network is a kind of feedforward deep neural network, which is composed of an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer. In the whole network structure, the convolutional layer and the pooling layer are the core modules for realizing the feature extraction function of the whole network. Firstly, the input layer receives the input data, and the convolutional layer extracts multi-level features of the data using convolution kernels with different weight values. The pooling layer, i.e. the down-sampling layer, receives the feature map output by the convolutional layer to perform feature selection and filtering, and the fully connected layer performs nonlinear combination on the extracted features and outputs.

[0173] The twin network was first proposed by Bromley et al. in the early 1990s, which is a kind of "twin neural network", i.e. the whole twin structure is composed of two neural networks, and the "twin" is realized by sharing the weights of the two networks, so the twin network receives two inputs and transmits them to the two neural networks sharing the weights to form respective feature representations, and finally the metric between the feature representations output by each network is calculated by using the same loss function, which can represent the correlation between the two inputs, so as to evaluate the similarity between them.

[0174] The above merely describes preferred embodiments of the present application but should not be used to limit the present application. Various modifications and changes can be made by those skilled in the art based on the spirit and principles of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall fall within the protection scope of the present application.

Claims

1. A method for airborne hyperspectral image quality assessment based on a twin network architecture, characterized in that, The method includes: S1. Acquire airborne hyperspectral imagery; S2. Preprocess the hyperspectral image; S3. Select a sample set from the preprocessed hyperspectral image, select the average value of the sample set as the standard set, extract the spectral index feature vector and radiation index feature vector of each sample set, and convert them into a fan-shaped scatter image. Specifically, it includes: S31. Creating Sample and Standard Groups: Select a single land cover type from the preprocessed hyperspectral image, crop out the test area, and divide the test area into N sample groups A. n (n = 1, 2, ..., N), select the 1st sample A1 and the nth sample A n Composition of sample group A1A n The average value of all sample groups is selected as the standard group A; where i, n, and N are all positive integers. S32. Sample group feature information extraction: For two samples in each sample group, calculate the spectral angular distance, spectral information divergence, spectral similarity and Pearson coefficient as spectral index feature vectors, and calculate the information entropy, average gradient, contrast and land type enhancement index as radiation index feature vectors. S33, Indicator Feature Vector Normalization: Normalize the 8 indicator feature vectors for each sample group; S34. Visualization of multidimensional feature vectors of sample groups: The eight normalized indicator feature vectors in each sample group are arranged and combined in an interactive manner according to two major categories: spectral indicators and radiometric indicators, and transformed into feature scatter lines. A fan-shaped scatter line plot with a fan-shaped distribution is generated through the polar coordinate system. S4. The fan-shaped scatter line images of the standard group and the fan-shaped scatter line images of each sample group are sequentially combined into image pairs, and input into the Siamese convolutional network model to obtain the similarity evaluation results of each image pair; wherein, the Siamese convolutional network model adopts Siamese network and convolutional neural network. S5. Based on the similarity evaluation results of each image pair, the quality evaluation results of the hyperspectral image are obtained.

2. The method according to claim 1, characterized in that, In step S1, the acquired hyperspectral image has a spectral resolution of not less than 10 nm, a spectral range of 350–2500 nm, and a spatial resolution better than 5 m.

3. The method according to claim 1, characterized in that, In step S2, the hyperspectral image is preprocessed using the following method: S21. Radiometric calibration: Input the image to be calibrated and the image description file into ENVI software, perform radiometric calibration, and obtain the surface reflectance after radiometric correction. S22. Atmospheric Correction: After radiometric calibration, atmospheric correction is performed to obtain the atmospherically corrected surface reflectance. S23. Orthorectification: Input the image to be orthorectified, the reference image, and the DEM data into the ENVI software, output the ground control points, and perform orthophoto correction on them to generate a planar orthophoto.

4. The method according to claim 1, characterized in that, Step S31 includes: Set the image size of the test area to a×a, the sample size to b×b, and the sliding step size to l; Set a fixed sample A1 as a b×b pixel range starting from the top left corner. Slide sample A1 gradually to the right by l pixels to obtain horizontal samples A2, A3, A4…A… i Similarly, by sliding sample A1 downwards by l pixels, vertical samples A are obtained sequentially. i+1 A i+2 …A n Where a, b, l, i, and n are all positive integers; Each sample is grouped with sample A1 to form sample groups A1A1, A1A2, ... A1A1. n ; The average value of all sample groups in the experimental area was taken as the standard group A.

5. The method according to claim 1, characterized in that, In step S34, For each normalized eigenvalue f k ∈(0,1)(k=1,2,…,8), its polar coordinate mapping is defined as: Where, r k θ represents the length of the characteristic scatter line. k This indicates the angular position of the scatter line between 0 and 180°, and p represents the number of all eigenvectors. Through this mapping, the 8 indicator feature vectors are projected into 8 feature lines, which are distributed in a fan-shaped manner, thus completing the visualization of the feature vectors.

6. The method according to claim 1, characterized in that, The twin convolutional network model includes a feature extraction model and a similarity evaluation model; the feature extraction model includes two identical convolutional networks with shared weights and an additional convolutional layer, which are used to input the fan-shaped scatter images of the standard group and the sample group, respectively, and output the corresponding features; The similarity evaluation model includes a non-parametric quadratic layer, a convolutional layer, a fully connected layer, and a SoftMax function connected in sequence. It is used to input two features output by the feature extraction model and output a similarity evaluation result.

7. The method according to claim 6, characterized in that, Step S4 includes: The pie charts of the standard group and each sample group are sequentially combined into image pairs, which are then input into the feature extraction model to extract the high-dimensional features f1 and f2 of the image pairs. The high-dimensional features f1 and f2 are input into the similarity evaluation model, and the output tensor f is... s f s =(f1-f2) 2 After the nonparametric squared layer, f s The input to the convolutional layer yields a series of feature maps, which are then fed into the fully connected layer for stretching, resulting in a one-dimensional vector. This one-dimensional vector is then input into the SoftMax function to obtain a two-dimensional probability distribution vector.

8. The method according to claim 7, characterized in that, Both the feature extraction model and the similarity evaluation model employ cross-entropy loss. The feature extraction model uses cross-entropy loss L id Represented as: Where * denotes convolution operation, f represents the feature representation output of the convolutional network, and θ I These are the parameters of the additional convolutional layer. It predicts the probability of a category, where K is the total number of categories, t is the true target category, and p is the probability of the predicted category. k It is the one-hot encoded vector of the true target class. For the true target class t, we have p t =1, other p i =0 (i≠t), p t p k A specific value at k = t; For a sample with a true label t, its true label distribution p k Defined by a one-hot vector, its formula is: The similarity evaluation model uses cross-entropy loss L ver Represented as: f s =(f1-f2) 2 Where f1 and f2 represent two features input to the similarity evaluation model, θ s These are the parameters of the convolutional layer connected to the squared layer, and s is a two-dimensional vector representing whether two images belong to the same category. q represents the predicted probability. i Similarity labels for image pairs; For an input image pair, its true similarity label distribution q i Defined by the following formula: If both belong to the same category, then q1 = 1 and q2 = 0; otherwise, q1 = 0 and q2 = 1.

9. An airborne hyperspectral image quality assessment system based on a twin network architecture, used to implement the method according to any one of claims 1-8, characterized in that, The system includes: The acquisition module is used to acquire airborne hyperspectral images; A preprocessing module is used to preprocess the hyperspectral image; The feature image extraction module is used to select a sample set from the preprocessed hyperspectral image, select the average value of the sample set as the standard set, extract the spectral index feature vector and the radiometric index feature vector of each sample set, and convert them into a fan-shaped scatter image. The similarity evaluation module is used to sequentially form image pairs from the fan-shaped scatter images of the standard group and the fan-shaped scatter images of each sample group, input them into the Siamese convolutional network model, and obtain the similarity evaluation results of each image pair; wherein, the Siamese convolutional network model adopts Siamese network and convolutional neural network. The image quality assessment module is used to obtain the quality assessment result of the hyperspectral image based on the similarity assessment result of each image pair.

Images