Hyperspectral image aircraft wake detection method based on spatial-spectral combination

Abstract

The invention discloses a hyperspectral image aircraft wake detection method based on spatial-spectral combination, and the method comprises the steps: carrying out the edge extraction of the spatial dimension of a hyperspectral image band by band, carrying out the full-band result statistical merging of a generated edge image, and carrying out the threshold screening of a merging result; eigenvalue decomposition is carried out on the spectral dimension of the hyperspectral image, eigenvectors are screened according to the eigenvalues, and the retained eigenvectors are selected to reconstruct the image; an RX operator is adopted to calculate a pixel abnormal score of the reconstructed image, and threshold screening is carried out on an abnormal score result; and performing bidirectional verification combination on two groups of results obtained in the spatial dimension and the spectral dimension by adopting a bidirectional verification model, screening spectral dimension feature detection results by taking a spatial dimension feature detection result as a guide image, and finally completing the detection of the wake of the aircraft. The spatial structure information and the spectral information of the hyperspectral image are utilized to the greatest extent, and the detection effects of high accuracy and low false alarm rate are achieved.

Description

Technical Field

[0001] The present invention relates to the field of hyperspectral remote sensing imaging technology, and particularly to a method for detecting aircraft wakes in hyperspectral images based on joint spatial-spectral processing. Background Art

[0002] Compared with traditional multispectral scanners, imaging spectrometers can obtain continuous images with hundreds of bands, and a spectral curve can be extracted from each image pixel, which has significant advantages for the fine recognition of ground objects. The hyperspectral images obtained by imaging spectrometers have the characteristic of "integration of image and spectrum", containing rich spatial, radiation, and spectral triple information of ground objects, and can truly and meticulously record the characteristics of ground objects. Compared with airborne hyperspectral remote sensing, spaceborne hyperspectral imaging technology not only maintains a high spectral resolution, but also has the advantages of a large detection range and small limitations. Specifically, a large detection range means that the swath width of spaceborne hyperspectral imaging is larger and the imaging speed is fast, and it only takes a few minutes to obtain the spectral information of ground objects within a range of hundreds of thousands of square kilometers; small limitations mean that the spaceborne platform can cover observation areas that are difficult for airborne platforms to reach globally.

[0003] The fine spectral information in hyperspectral images has great advantages for the accurate recognition of ground objects, can identify ground objects that are difficult to distinguish in panchromatic and multispectral images, and has inherent advantages in target detection tasks. However, the rich ground object information and wide swath width make the data volume of the original hyperspectral image very large, and due to the physical meaning of the three-dimensional data structure of hyperspectral images, the spatial correlation between different pixels makes the spectral vectors not independent, which also poses challenges to existing image processing methods and data analysis methods.

[0004] Hyperspectral remote sensing target detection is a binary classification problem that divides an image into background and target of interest. According to whether the spectral information of the target is known, it can be divided into the detection of known target spectral information and the anomaly detection of unknown target spectral information. However, in practical applications, affected by various factors such as the environment, illumination, imaging system, and surrounding ground objects, the target spectrum is usually uncertain and needs to be estimated from the image. Currently, the anomaly detection algorithms based on model optimization at home and abroad can be roughly divided into three categories: methods based on statistical models, methods based on representation models, and methods based on decomposition models.

[0005] The anomaly detection algorithm based on statistical models is the earliest developed method in anomaly detection. It has a simple model and good performance, and realizes anomaly detection by utilizing the statistical characteristics of the target and the background. The most classic one is the Reed-Xiaoli (RX) algorithm proposed by I.S. Reed and Xiaoli Yu. The RX algorithm is also generally regarded as the benchmark algorithm in statistical models. The RX algorithm is a constant false alarm rate algorithm based on the generalized likelihood ratio test. It assumes that the background follows a Gaussian distribution and estimates the probability density function by calculating the covariance and mean vector of all background samples. The detection pixels where the anomaly targets are located will deviate from the main statistical characteristics represented by the covariance matrix, thereby effectively detecting the anomaly targets. However, since the background in actual remote sensing images often does not follow a single multivariate Gaussian distribution, its detection effect is greatly affected by the background distribution.

[0006] The method based on the representation model considers anomaly detection starting from a single pixel. Its basic idea is as follows: Select a set of representative background spectra from the image as the background dictionary. These spectra in the background dictionary are also called atoms. Based on the assumption that the background pixels can be well represented by the atoms in the background dictionary while the anomaly pixels cannot, image reconstruction is carried out.

[0007] The method based on the decomposition model focuses on describing the overall characteristics of the image from a global perspective and performing anomaly detection accordingly. There is a certain degree of correlation between background pixels in both space and spectrum, which is reflected in the data as the low-rank property, while the similarity between anomaly targets and the background is low, showing the sparse property. However, either of these two methods requires iterative calculations and has a high time complexity, or ignores the relationship between the spatial correlation and spectral correlation of pixels in hyperspectral images, and it is difficult to achieve good results in the detection of aircraft wakes. Summary of the Invention

[0008] Aiming at the above problems, the present invention provides a method for detecting aircraft wakes in hyperspectral images based on the joint spatial-spectral information, which can break through the limitations of the background, utilize the joint spatial-spectral information for aircraft wake detection, make the most of the spatial structure information and spectral information of hyperspectral images, obtain detection results with high accuracy and low false alarm rate, and have strong robustness under various complex background distributions and good detection effects.

[0009] A method for detecting aircraft wakes in hyperspectral images based on the joint spatial-spectral information includes the following steps:

[0010] (1) Extract the edges of the spatial dimension of the hyperspectral image band by band, perform full-band result statistics and merging on the generated edge images, and perform threshold screening on the merged results to obtain the spatial dimension feature detection results;

[0011] (2) Perform eigenvalue decomposition on the spectral dimension of the hyperspectral image, screen the eigenvectors according to the eigenvalue magnitudes, and use the retained eigenvectors to reconstruct the image; calculate the pixel anomaly scores of the reconstructed image using the RX operator, and perform threshold screening on the anomaly score results to obtain the spectral dimension feature detection results;

[0012] (3) Use a two-way verification model to perform two-way verification and merging on the two sets of results obtained from the spatial dimension and the spectral dimension. According to the idea of guided filtering, use a sliding window to perform cross-side comparison. The side with a higher outlier statistical result is defined as the plume, and the other side is defined as the background. Use the spatial dimension feature detection results as a guiding image to screen the spectral dimension feature detection results, and finally complete the detection of the aircraft wake.

[0013] In step (1), the optimal edge detection operator, the Canny operator, is used for edge extraction. The specific process is as follows:

[0014] Convert the feature band image into a grayscale image I band by band, then perform Gaussian convolution smoothing filtering on the grayscale image I, and then calculate the gradient:

[0015]

[0016] where h is the Gaussian filter, are the differential filters in the x and y directions of the spatial dimension. The magnitude of the image gradient is G, and the calculated image gradient map is S;

[0017] Subsequently, along the gradient direction, suppress the points with non-maximum gradient values to refine the edges and achieve the edge detection effect with a single-pixel width; use double-threshold detection and connection to ensure a low error rate of the detection results, and obtain a series of edge images of the original image's feature bands.

[0018] Perform full-band result statistics and merging on the generated edge images, and perform threshold screening on the merging results. The specific process is as follows:

[0019] After obtaining the edge images, perform Hough line detection on them; in the line detection results obtained from the same-band images, use the Bresenham algorithm to perform single-pixel connection and merging on the lines with the same parameters to maintain the accuracy of the single-pixel width edge detection; finally, use a voting mechanism to merge the detection results of all feature bands, and set a threshold function f(t) to process the detection results:

[0020]

[0021] threshold 1 =ε 1 ·max(t), 0 < ε 1 <1

[0022] Wherein, t represents the number of times the pixel is counted as an edge in the line detection, threshold 1 is the threshold number of times finally retained as an edge, ε 1 is a constant, and f(t) is the statistical result at the corresponding position after screening;

[0023] Through the above steps, the voting result of the spatial - dimensional line pixels is obtained, that is, the spatial - dimensional feature detection result.

[0024] In step (2), eigenvalue decomposition is performed on the spectral dimension of the hyperspectral image, and the formula is as follows:

[0025]

[0026] Wherein, S is the hyperspectral image, V is the eigenvalue vector matrix, ∧ is the eigenvalue matrix corresponding to the eigenvector, v 1 ……v N represents the eigenvalue vector, λ 1 ……λ N represents the eigenvalue corresponding to the eigenvector.

[0027] According to the eigenvalue magnitudes, the eigenvectors are screened, and the retained eigenvectors are used to reconstruct the image. Specifically:

[0028] Analyze the eigenvalue histogram, retain the eigenvectors corresponding to the larger eigenvalues, suppress noise, and reconstruct the image

[0029]

[0030] Wherein, represents the reconstructed eigenvector matrix, and m is the number of deleted eigenvectors; then the reconstructed image is expressed as:

[0031]

[0032] Wherein, S is the hyperspectral image, μ H is the mean of the image.

[0033] The RX operator is used to calculate the pixel anomaly score of the reconstructed image, and the result of the anomaly score is screened by a threshold. The specific process is as follows:

[0034] Apply the RX detection operator D RX (z) to the reconstructed image to calculate the pixel anomaly score:

[0035]

[0036] Wherein, z is the image pixel, Σ 0 is the covariance matrix, μ 0 is the mean;

[0037] Finally, considering the background components in the retained eigenvalues during eigenvalue selection, threshold processing is performed on the results of the anomaly scores:

[0038] threshold 2 = ε 2 max(z), 0 ≤ ε 2 <1

[0039] In the formula, threshold 2 is the threshold of the finally retained outliers, ε 2 is a constant, and z is the image pixel;

[0040] After the processing of the above steps, an anomaly score matrix is obtained, that is, the spectral dimension feature detection result.

[0041] The specific process of step (3) is as follows:

[0042] Based on the assumption that the anomaly score on the plume side is higher than that on the background side, according to the idea of guided filtering, a sliding window is used, and the center of the window moves along the target position of the spatial dimension feature detection result. During the process, the anomaly degree of the corresponding local range in the spectral dimension feature detection result is dynamically compared;

[0043] Assume that the final result matrix is Σ, represents the sub-matrix segmented by the window with coordinates (i, j) as the center and an odd number k as the dimension. Matrix A is the spatial dimension feature detection result, and matrix B is the spectral dimension feature detection result. Both A and B have the same spatial dimension size as the original three-dimensional hyperspectral image; when the element a at position (i, j) in matrix A i,j > 0, the calculation is performed. The two-way verification model is expressed in matrix operations as follows:

[0044]

[0045] f(X, Y) = H(X·Y - X·Y T )

[0046] Among them, the * operation is the Hadamard product of matrices, which is the result of multiplying the corresponding elements of two matrices of the same size; J k,k is a k-order all-ones matrix; I k is the identity matrix; G k is a k-order matrix with 1 only at the center position and 0 at the remaining positions; T k is a k-order upper triangular matrix with all values being 1; for simplicity in representing the operation relationship, the parameters X and Y in the formula represent matrices of the same size; σ is a constant, and the definition of the unit step function H(l) is:

[0047]

[0048] In the formula, l represents the result of matrix comparison; after the two-way verification process is completed, the final detection result is obtained.

[0049] In the case of non-plume, there is no gradient trend in the outliers on both sides, while in the case of plume, there must be a gradient trend in the local areas on both sides. The two-way verification improves the positioning accuracy while solving the problem of directional determination on the plume side.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. The present invention fully considers the spatial-spectral correlation of hyperspectral images, uses a spatial dimension feature extraction unit improved based on the Hough transform to extract important edge information in the image, and effectively reduces the false alarm rate by using spatial dimension features.

[0052] 2. The present invention adopts a two-way verification model and proposes a guided filtering window, which effectively improves the accuracy of the detection result while reducing the calculation amount.

[0053] 3) The method proposed by the present invention does not target a certain background distribution and can effectively detect various complex backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flowchart of a method for detecting aircraft wakes in hyperspectral images based on spatial-spectral joint in an embodiment of the present invention.

[0055] Figure 2 It is a schematic diagram of the fusion process of the two-way verification model in an embodiment of the present invention.

[0056] Figure 3 It is a comparison result diagram of the detection results by the method of the present invention and other methods. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] The following further describes the present invention in detail with reference to the drawings and embodiments. It should be noted that the following embodiments are intended to facilitate the understanding of the present invention and do not limit it in any way.

[0058] In this embodiment, an image acquired by the visible short-wave infrared hyperspectral camera of the Gaofen-5 satellite is selected and cropped into an image of 200×200×180 as an example.

[0059] As Figure 1 shown, a method for detecting aircraft wakes in hyperspectral images based on spatial-spectral joint includes the following steps:

[0060] Step 1: Extract the edges of the spatial dimension of the hyperspectral image band by band, perform full-band result statistics and merging on the generated edge images, and perform threshold screening on the merged results to obtain the spatial dimension feature detection result.

[0061] Select a hyperspectral image that has been cropped and has a size of 200×200×180, where both the number of rows and columns is 200 and the number of bands is 180, and perform spatial - dimensional straight - line extraction. The straight - line extraction includes the following steps: For the k - th (1 ≤ k ≤ 180) spectrum, first convert it into a grayscale image, perform edge detection using the Canny operator to obtain 180 gradient maps of the image. Perform Hough straight - line detection on each gradient map to obtain 180 edge response maps, that is, 180 binary matrices of size 200×200. Sum these 180 binary matrices, and set the ε 1 (0 < ε 1 < 1) times the threshold for screening. The screening result is the spatial - dimensional detection result.

[0062] Among them, the operation process of the Canny operator for calculating the gradient map is as follows:

[0063]

[0064] Among them, h is the Gaussian filter, is the difference filter in the x and y directions of the spatial dimension. The amplitude of the image gradient is G, and the calculated image gradient map is S.

[0065] Step 2: Perform eigenvalue decomposition on the spectral dimension of the hyperspectral image, screen the eigenvectors according to the eigenvalue magnitudes, and use the retained eigenvectors to reconstruct the image; Calculate the pixel anomaly scores of the reconstructed image using the RX operator, and perform threshold screening on the anomaly score results to obtain the spectral - dimensional feature detection results.

[0066] Regard the spectral dimension of the hyperspectral image as 200×200 spectral vectors with a dimension of 180×1, and perform eigenvalue decomposition. S is the hyperspectral image, V is the eigenvector matrix of eigenvalues, and Λ is the eigenvalue matrix corresponding to the eigenvectors:

[0067]

[0068] Obtain the eigenvalues and eigenvectors, discard the eigenvectors with smaller eigenvalues, and use the remaining eigenvectors for image reconstruction. m is the number of discarded eigenvectors. The mean of the image is μ H , then the reconstructed image can be expressed as:

[0069]

[0070] Perform anomaly detection on the reconstructed image using the RX operator D RX (z) to obtain the anomaly - value image, and set a threshold for screening the anomaly - value image:

[0071]

[0072] threshold 2 = ε 2 max(z), 0 ≤ ε 2 <1

[0073] After the above steps of processing, an anomaly score matrix is obtained, which is the detection result in the spectral dimension.

[0074] Step 3: Use a two-way verification model to perform two-way verification and merging on the two sets of results obtained in the spatial dimension and the spectral dimension. Use the detection result of the spatial dimension features as the guiding image to screen the detection result of the spectral dimension features, and finally complete the detection of the aircraft wake.

[0075] Spatial-spectral two-way verification. Based on the idea of guided filtering, use a sliding window, and the center of the window moves along the target position of the spatial dimension feature extraction result. During the process, dynamically compare the anomaly degree of the local range at the corresponding position in the spectral dimension detection result. The simplified process is as Figure 2 shown. Assume that the final result matrix is Σ, denotes the sub-matrix segmented by a window with an odd dimension centered at the coordinate (i, j) (here k = 5, and (i, j) corresponds to the center position of the window). Matrix A is the boundary response matrix obtained by the spatial dimension feature extraction module, that is, the detection result in the spatial dimension. Matrix B is the outlier matrix obtained by the spectral dimension feature extraction module, that is, the detection result in the spectral dimension. Both A and B have the same spatial dimension size as the original three-dimensional hyperspectral image. When the position (i, j) in matrix A is at the center position of the window and the element a i,j > 0, the calculation is performed. The two-way verification model can be expressed by matrix operations as follows:

[0076]

[0077] f(X, Y) = H(X·Y - X·Y T )

[0078] where the * operation is the Hadamard product of matrices, which is the result of multiplying the corresponding elements of two matrices of the same size; J k,k is a k-order all-ones matrix; I k is the identity matrix; G k is a k-order matrix with 1 only at the center position and 0 at the other positions; T k is a k-order upper triangular matrix with all values being 1; To simplify the representation of the operation relationship, the parameters X and Y in the formula represent matrices of the same size. σ is a constant. Among them, the definition of the unit step function H(l) is:

[0079]

[0080] After the two-way verification process ends, the final detection result is obtained.

[0081] The evaluation index table of the detection result of the method of the present invention compared with the results of other methods is shown in Table 1 below.

[0082] Table 1

[0083]

[0084] Figure 3 It is a comparison result diagram of the detection result of the method of the present invention and other methods. In the figure, (a) is a single-band image, (b) is a labeled ground truth image, (c) is the experimental result using the RX algorithm, (d) is the experimental result using the LRX algorithm, (d) is the experimental result using the LRASR algorithm, (f) is the experimental result using the KIFD algorithm, and (g) is the experimental result using the method of the present invention.

[0085] According to Table 1 and Figure 3 , it can be seen that the detection result using the present invention is closest to the ground truth image. While the accuracy rate of the detection result is relatively high, the false alarm rate is relatively low, and the performance of the detection method is excellent.

[0086] The above-described embodiments have detailed the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, supplements, and equivalent replacements made within the scope of the principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for detecting aircraft wakes based on hyperspectral images of space and spectrum, characterized in that: The following steps are involved: (1) Extract the edge of the hyperspectral image band by band in the spatial dimension, statistically merge the generated edge image in all bands, perform threshold screening on the merged result, and obtain the spatial dimension feature detection result; (2) Perform eigenvalue decomposition on the spectral dimension of the hyperspectral image, screen the eigenvectors according to the eigenvalue size, and select the retained eigenvectors to reconstruct the image; use the RX operator to calculate the pixel anomaly score of the reconstructed image, perform threshold screening on the anomaly score results, and obtain the spectral dimension feature detection results; (3) A bidirectional verification model is used to perform bidirectional verification and merging of the two sets of results obtained from the spatial dimension and the spectral dimension. Based on the idea of ​​guided filtering, a sliding window is used for bilateral comparison. The side with a higher outlier statistical result is defined as the plume, and the other side is defined as the background. The spatial dimension feature detection result is used as a guide image to screen the spectral dimension feature detection result, and finally the detection of the aircraft wake is completed.

2. The method for detecting aircraft wakes based on hyperspectral images of space-spectrum joint according to claim 1 is characterized in that: In step (1), the optimal edge detection operator Canny operator is used to extract the edge. The specific process is as follows: The characteristic band image is converted into a grayscale image I band by band, and then the grayscale image I is subjected to Gaussian convolution smoothing filtering, and then the gradient is calculated: Where h is a Gaussian filter, is a differential filter in the x and y directions of the spatial dimension, the amplitude of the image gradient is G, and the calculated image gradient map is S; Subsequently, along the gradient direction, points with non-maximum gradient values ​​are suppressed to achieve edge refinement and single-pixel-wide edge detection effects; dual threshold detection and connection are used to ensure a low error rate in the detection results, and a series of edge images of the characteristic bands of the original image are obtained.

3. The method for detecting aircraft wakes based on hyperspectral images of space-spectrum joint according to claim 1 is characterized in that: In step (1), the generated edge image is statistically merged for all bands, and the merged result is thresholded. The specific process is as follows: After the edge image is obtained, Hough line detection is performed on it; in the line detection results obtained from the same band image, the Bresenham algorithm is used to merge single-pixel connections for the lines with the same parameters to maintain the accuracy of single-pixel wide edge detection; finally, the detection results of all feature bands are merged using a voting mechanism, and the threshold function f(t) is set to process the detection results: threshold1=ε1·max(t),0<ε1<1 In the formula, t represents the number of times the pixel is counted as an edge in the straight line detection, threshold1 is the threshold number of times it is finally retained as an edge, ε1 is a constant, and f(t) is the statistical result of the corresponding position after screening; After the above steps, the spatial dimension line pixel voting result is obtained, that is, the spatial dimension feature detection result.

4. The method for detecting aircraft wakes based on hyperspectral images of space-spectrum joint according to claim 1 is characterized in that: In step (2), the spectral dimension of the hyperspectral image is decomposed by eigenvalues ​​using the following formula: Where S is the hyperspectral image, V is the eigenvalue vector matrix, ∧ is the eigenvalue matrix corresponding to the eigenvector, v1…v N represents the eigenvalue vector, λ1…λ N Denotes the eigenvalue corresponding to the eigenvector.

5. The method for detecting aircraft wakes based on hyperspectral images of space-spectrum joint according to claim 4 is characterized in that: In step (2), the feature vectors are screened according to the size of the feature values, and the retained feature vectors are selected to reconstruct the image, specifically: Analyze the eigenvalue histogram, retain the eigenvectors corresponding to the larger eigenvalues, suppress the noise, and reconstruct the image In the formula, represents the reconstructed eigenvector matrix, m is the number of deleted eigenvectors; then the reconstructed image It is expressed as: Where S is the hyperspectral image, μ H is the mean value of the image.

6. The method for detecting aircraft wakes based on hyperspectral images of space-spectrum joint according to claim 1 is characterized in that: In step (2), the RX operator is used to calculate the pixel anomaly score of the reconstructed image, and the result of the anomaly score is thresholded. The specific process is as follows: Apply the RX detection operator D to the reconstructed image RX (z) Calculate pixel anomaly score: Where z is the image pixel, Σ0 is the oblique variance matrix, and μ0 is the mean; Finally, the anomaly score result is thresholded, taking into account the background components in the features retained during feature selection: threshold2=ε2max(z),0≤ε2<1 In the formula, threshold2 is the outlier threshold finally retained, ε2 is a constant, and z is the image pixel; After the above steps, the anomaly score matrix is ​​obtained, that is, the spectral dimension feature detection result.

7. The method for detecting aircraft wakes based on hyperspectral images of space-spectrum combination according to claim 1, characterized in that: The specific process of step (3) is as follows: Based on the assumption that the abnormal value score on the plume side is higher than that on the background side, and according to the idea of ​​guided filtering, a sliding window is used, and the center of the window moves along the target position of the spatial dimension feature detection result. During the process, the abnormal degree of the local range of the corresponding position in the spectral dimension feature detection result is dynamically compared; Assume that the final result matrix is ​​∑, It represents the submatrix divided by the window with the coordinate (i, j) as the center and the odd number k as the dimension. Matrix A is the spatial dimension feature detection result, and matrix B is the spectral dimension feature detection result. Both A and B have the same spatial dimension as the original three-dimensional hyperspectral image. When the element a at position (i, j) in matrix A i,j When >0, the calculation is performed, and the two-way validation model is expressed by matrix operation as follows: f(X,Y)=H(X·Y-X·Y T ) The * operation is the Hadamard product of matrices, which is the result of multiplying the elements at corresponding positions of two matrices of the same size; J k,k is a k-order all-1 matrix; I k is the unit matrix; G k is a k-order matrix with 1 only in the center and 0 in the rest of the positions; T k is a k-order upper triangular matrix with all values ​​being 1; to simplify the expression of the operation relationship, the parameters X and Y in the formula represent matrices of the same size; σ is a constant, and the unit step function H(l) is defined as: In the formula, l represents the result of matrix comparison; after the two-way verification process is completed, the final detection result is obtained.

8. The method for detecting aircraft wakes based on hyperspectral images of space-spectrum joint according to claim 7 is characterized in that: In the absence of plume, there is no gradient trend in the outliers on both sides, but in the presence of plume, there must be a gradient trend in the local areas on both sides. The two-way verification can improve the positioning accuracy and solve the problem of directional determination on the plume side.

Images