Hyperspectral anomaly detection method based on Hadamard projection and Kalman filtering

Abstract

The invention relates to the field of multi-hyperspectral anomaly detection, in particular to a hyperspectral anomaly detection method based on Hadamard projection and Kalman filtering. The method takes efficient dimension reduction and online modeling as the core, and comprises the following steps: firstly, carrying out rapid dimension reduction on input high-dimensional spectral data by utilizing Hadamard random projection to greatly reduce the calculation burden of subsequent processing; then, in a low-dimensional space, a Kalman filter is adopted to perform dynamic and smooth tracking on a background mean value, and a robust covariance online updating mechanism based on quantile screening and diagonal contraction is combined, so that pollution of abnormal points to a background model is effectively inhibited; and finally, calculating a mahalanobis distance as an abnormal score based on a stable background model. The method realizes unification of calculation efficiency, detection precision and model stability, and is especially suitable for real-time online anomaly detection tasks of airborne platforms.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral anomaly detection, specifically to a hyperspectral anomaly detection method based on Hadamard projection and Kalman filtering. Background Technology

[0002] Hyperspectral anomaly detection utilizes hyperspectral imaging technology to acquire detailed spectral information of ground features, enabling the automatic identification and location of targets or features significantly different from their surroundings in complex backgrounds without prior knowledge. Due to its unsupervised nature and powerful ability to distinguish subtle differences in materials, this technology has broad application prospects in environmental monitoring, precision agriculture, camouflaged target identification, and disaster search and rescue.

[0003] However, anomaly detection faces significant challenges in real-time or near-real-time applications such as pushbroom hyperspectral imaging. First, the extremely high dimensionality of hyperspectral data results in a massive computational burden for estimating and calculating background statistical models (such as the covariance matrix), making it difficult to meet real-time requirements. Second, pushbroom imaging data is acquired row by row, and the background environment dynamically changes as the scanning process progresses, requiring detection algorithms to possess online learning and adaptive update capabilities. However, the traditional Reed-Xiaoli (RX) algorithm and its variants are extremely sensitive to the accuracy of background model estimation. Even a small number of anomalous points in the background can distort the covariance matrix estimation, leading to numerous false alarms or missed detections.

[0004] To address these issues, existing research has explored various methods. For example, traditional dimensionality reduction methods such as Principal Component Analysis (PCA) reduce computational costs, but may lose subtle spectral features crucial for anomaly detection. Some online update algorithms employ simple exponential moving averages (EMA) to update the background model, but these are sensitive to noise and abrupt changes, exhibiting insufficient stability. While some studies have introduced robust statistics, these are often computationally complex and difficult to integrate into real-time processing workflows. Therefore, a computationally efficient, model-robust, and dynamic real-time anomaly detection method is urgently needed to solve the two core challenges of "curse of dimensionality" and "model drift" in high-dimensional data stream processing. Summary of the Invention

[0005] This invention addresses the problems of high computational complexity, susceptibility to background model estimation interference, and poor adaptability to dynamic environments in pushbroom hyperspectral real-time anomaly detection. It proposes a hyperspectral anomaly detection method based on Hadamard projection and Kalman filtering. The core of this method is efficient dimensionality reduction and online modeling: First, Hadamard random projection is used to rapidly reduce the dimensionality of the input high-dimensional spectral data, significantly reducing the computational burden of subsequent processing. Then, in the low-dimensional space, a Kalman filter is used to dynamically and smoothly track the background mean, combined with a robust online covariance update mechanism based on quantile screening and diagonal shrinkage to effectively suppress the contamination of the background model by anomalies. Finally, Mahalanobis distance is calculated as the anomaly score based on a stable background model. This invention achieves a balance between computational efficiency, detection accuracy, and model stability, making it particularly suitable for real-time online anomaly detection tasks on airborne platforms.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by the present invention is as follows:

[0007] A hyperspectral anomaly detection method based on Hadamard projection and Kalman filtering includes the following steps:

[0008] Step 1: Structured random projection dimensionality reduction.

[0009] Current scan line data of the received hyperspectral image By applying a sparse random Hadamard transform (SRHT) projector, the data is projected onto a lower-dimensional subspace to obtain the dimensionality-reduced data. The dimensions after dimensionality reduction It was set to 5.

[0010] Step 2: Online background model initialization and update.

[0011] In the low-dimensional subspace, the background statistical model is initialized and updated online:

[0012] a) If processing the first row of data, then directly calculate the current row. mean With covariance , as the initial background model.

[0013] b) If it is not the first row of data, then update the background model:

[0014] b1) Mean Update: Calculate the mean of the dimensionality-reduced data for the current row as the observation value, and update the background mean using a Kalman filter. The process noise covariance matrix of the Kalman filter is... Set as Measure the noise covariance matrix Set as A new background mean is obtained through the update step. .

[0015] b2) Robust Covariance Update: Centered on the updated mean, calculate the bias vector and its square norm of each pixel in the current row in the reduced-dimensional space, and apply the result to a preset quantile threshold. Pixels with excessive bias are removed, and the sample covariance of the current row is estimated using only the remaining pixels; finally, the momentum coefficient is used. Exponential moving average and contraction coefficient Updating the global covariance matrix using a combination of diagonal contraction and other methods .

[0016] Step 3: Buffer check.

[0017] Set a buffer length The value is 99. When the cumulative number of rows processed is less than 99, only steps 1 and 2 are executed to continuously update the background mean and covariance, without outputting anomaly detection results, so as to ensure that the background model gradually converges and stabilizes with sufficient sample support.

[0018] Step 4: Calculate the outlier score.

[0019] When the cumulative number of processed rows reaches or exceeds 99, each pixel in the current row is processed. Based on the mean and covariance of the current background model Calculate its Mahalanobis distance To ensure numerical stability, Cholesky decomposition is used to solve the linear equation system instead of direct matrix inversion.

[0020] Step 5: Output the anomaly score graph.

[0021] Combine the Mahalanobis distances of all pixels in the current row into a one-dimensional vector, and output it as the anomaly detection result for that scan row.

[0022] Furthermore, step 2 includes the following sub-steps:

[0023] Step 2.1: When processing the first row, directly use the dimensionality-reduced data of that row for initialization; let the dimensionality-reduced matrix of the first row be... , its first 1 pixel The initial mean is The initial estimate of the covariance is: in It is the identity matrix. For a very small positive number (e.g.) This is used to ensure matrix invertibility. Simultaneously, the initial state of the Kalman filter is set to... Its internal covariance matrix is ​​set as a diagonal large number matrix to indicate that the mean estimate is uncertain at the beginning;

[0024] Step 2.2: Online update of subsequent rows; when processing the first row When traversing a row, first calculate the simple mean of the current row based on the dimensionality-reduced data of that row: ;

[0025] Step 2.2.1: Kalman update of the mean; change the background mean of the previous row. The row mean of the current row, representing the previous state of the Kalman filter. As an observation; since it is assumed that the background mean changes relatively smoothly between adjacent rows, the state transition can be simplified to "the mean of the previous row is directly passed to the next row," that is... The observation model is Among the observations that is Kalman filtering internally utilizes a pre-defined process noise covariance matrix. and measurement noise covariance matrix Calculate the Kalman gain and update the state to obtain the new background mean.

[0026] Step 2.2.2: First, calculate the pixel values ​​in the current row. To the new mean square distance Then, set a quantile threshold. Remove all Pixels larger than this threshold are considered part of the "normal" pixel set. The number of them is ; using sets Calculate the sample covariance of the current row for each cell in the array: Subsequently, a momentum coefficient was used. (Pick ) The global covariance of the previous row and the current line Fusion: To prevent the covariance matrix from becoming too "sharp" or numerically unstable in certain directions, a slight diagonal contraction is introduced: first, the mean variance of the current covariance is calculated. ,in The sum of the diagonal elements of the matrix is ​​expressed as the sum of the diagonal elements, and finally expressed as the contraction coefficient. (Pick Convert the covariance matrix to a scalar matrix Slightly zoom in: This significantly reduces the risk of numerical instability while preserving the statistical structure.

[0027] Compared with existing methods, the advantages of the present invention are:

[0028] This invention achieves a balance between computational efficiency, model robustness, and detection accuracy in pushbroom hyperspectral real-time detection tasks through a collaborative framework of structured random projection and online covariance update: 1) SRHT is used for dimensionality reduction, which has a much lower computational complexity than traditional methods such as PCA, and does not require storing a large projection matrix, resulting in extremely low memory usage, making it very suitable for hardware-constrained real-time processing platforms. 2) A Kalman filter is introduced to dynamically track the background mean, which, compared to simple EMA, can better suppress observation noise and provide a smoother and more accurate background estimate. 3) A robust covariance update mechanism of "quantile screening + diagonal shrinkage" is proposed, which can effectively eliminate the interference of outliers on the background model, and ensure the good state of the covariance matrix through shrinkage regularization, significantly improving the stability and reliability of the algorithm in complex backgrounds. 4) The entire process is processed online row by row, without iteration, with low latency, fully meeting the real-time requirements of pushbroom imaging. Attached Figure Description

[0029] Figure 1 This is a flowchart of the detection method of the present invention;

[0030] Figure 2 This is a schematic diagram of the online background mean and covariance update module in this invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and examples.

[0032] like Figure 1 As shown, a hyperspectral anomaly detection method based on Hadamard projection and Kalman filtering includes the following steps:

[0033] Step 1: Structured random projection dimensionality reduction.

[0034] (1a) Let the hyperspectral data of the current scan row be... ,in The number of pixels per row. For the number of bands, This represents the spectral vector of a single pixel. An SRHT projector is used for... Dimensionality reduction is performed to obtain the dimensionality-reduced matrix:

[0035] like Figure 2 As shown, step 2: online background model initialization and update.

[0036] (2a) When processing the first row, the dimensionality-reduced data of that row is used directly for initialization. Let the dimensionality-reduced matrix of the first row be... , its first Behavior The initial mean is The initial estimate of covariance is in It is the identity matrix. For a very small positive number (e.g.) This is used to ensure matrix invertibility. Simultaneously, the initial state of the Kalman filter is set to... Its internal covariance matrix is ​​set to a diagonal large number matrix to indicate that the mean estimate is uncertain at the beginning.

[0037] (2b) Online updates for subsequent rows. When processing the first row... When traversing a row, first calculate the simple mean of the current row based on the dimensionality-reduced data of that row:

[0038] (2b-1) Kalman update of the mean. Replace the background mean from the previous row... The row mean of the current row, representing the previous state of the Kalman filter. As an observation, since it is assumed that the background mean changes relatively smoothly between adjacent rows, the state transition can be simplified to "the mean of the previous row is directly passed to the next row," that is... The observation model is Among the observations that is Kalman filtering internally utilizes a pre-defined process noise covariance matrix. and measurement noise covariance matrix Calculate the Kalman gain and update the state to obtain the new background mean.

[0039] (2b-2) Robust covariance update:

[0040] First, calculate each pixel in the current row. To the new mean square distance Then, a quantile threshold is set. Remove all Pixels larger than this threshold are considered part of the "normal" pixel set. The number of them is Using sets Calculate the sample covariance of the current row for each cell in the array: Then, using a momentum coefficient (Pick ) The global covariance of the previous row and the current line Fusion: To prevent the covariance matrix from becoming too "sharp" or numerically unstable in certain directions, a slight diagonal contraction is introduced: first, the mean variance of the current covariance is calculated. ,in The sum of the diagonal elements of the matrix is ​​expressed as the sum of the diagonal elements, and finally expressed as the contraction coefficient. (Pick Convert the covariance matrix to a scalar matrix Slightly zoom in: This approach preserves the statistical structure while significantly reducing the risk of numerical instability.

[0041] Step 3: Buffer check.

[0042] Total number of lines processed .like If the result is not found, then no further steps will be executed; only an empty value will be returned, and the subsequent steps will be stopped. Add 1.

[0043] Step 4: Calculate the outlier score.

[0044] when At that time, for each pixel in the current row Based on the current background average Covariance Calculate the Mahalanobis distance as the outlier score.

[0045] (4a) Calculate the deviation vector:

[0046] (4b) Numerically stable Mahalanobis distance calculation. Direct calculation. The values ​​are prone to instability, therefore we should first... Perform Cholesky decomposition to obtain the lower triangular matrix. ,satisfy: Then, for each pixel, solve the linear equation. get Then, its length is taken as the Mahalanobis distance: Numerically, this is equivalent to .

[0047] Step 5: Output the anomaly score graph.

[0048] The Mahalanobis distance of all pixels in the current row Arranged into a one-dimensional vector in pixel order This output serves as the anomaly detection result for that scan line.

[0049] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the implementation methods of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of the present invention.

Claims

1. A hyperspectral anomaly detection method based on Hadamard projection and Kalman filtering, characterized in that, Includes the following steps: Step 1: Structured random projection dimensionality reduction; Current scan line data of the received hyperspectral image By applying a sparse random Hadamard transform projector, the data is projected onto a lower-dimensional subspace to obtain the dimensionality-reduced data. The dimensions after dimensionality reduction It was set to 5; Step 2: Online background model initialization and update; In the low-dimensional subspace, the background statistical model is initialized and updated online: If processing the first row of data, then directly calculate the current row. mean With covariance As the initial background model; If the data is not from the first row, the background model is updated: the mean of the reduced-dimensional data in the current row is calculated as the observation value, and the background mean is updated using a Kalman filter; the updated mean is then used as the basis for further updates. Centered on the target cell, calculate the bias vector and its square norm of each cell in the current row in the reduced-dimensional space to obtain the global covariance matrix. ; Step 3: Buffer check; Set a buffer length The cumulative number of rows processed is less than 99; when the cumulative number of rows processed is less than 99, only steps 1 and 2 are executed to continuously update the background mean and covariance, without outputting anomaly detection results, so as to ensure that the background model gradually converges and stabilizes with sufficient sample support. Step 4: Calculate outlier scores; When the cumulative number of processed rows reaches or exceeds 99, the i-th pixel of the current row is processed. Based on the mean and covariance of the current background model Calculate its Mahalanobis distance To ensure numerical stability, Cholesky decomposition is used to solve the linear equation system instead of direct matrix inversion. Step 5: Output the anomaly score graph; Combine the Mahalanobis distances of all pixels in the current row into a one-dimensional vector, and output it as the anomaly detection result for that scan row.

2. The hyperspectral anomaly detection method based on Hadamard projection and Kalman filtering according to claim 1, characterized in that: Step 2 includes the following sub-steps: Step 2.1: When processing the first row, directly use the dimensionality-reduced data of that row for initialization; let the dimensionality-reduced matrix of the first row be... , its first 1 pixel The initial mean is ; The initial estimate of covariance is ,in It is the identity matrix. It is a small positive number to ensure matrix invertibility; at the same time, the initial state of the Kalman filter is set to... Its internal covariance matrix is ​​set as a diagonal large number matrix to indicate that the mean estimate is uncertain at the beginning; Step 2.2: Online update of subsequent rows; when processing the first row When traversing a row, first calculate the simple mean of the current row based on the dimensionality-reduced data of that row: .

3. The hyperspectral anomaly detection method based on Hadamard projection and Kalman filtering according to claim 2, characterized in that: Step 2.2 includes the following sub-steps: Step 2.2.1: Kalman update of the mean; change the background mean of the previous row. As the previous state of the Kalman filter, the row mean of the current row. As an observation; since it is assumed that the background mean changes relatively smoothly between adjacent rows, the state transition is simplified to "the mean of the previous row is directly passed to the next row," that is... The observation model is Among the observations that is Kalman filtering internally utilizes a pre-defined process noise covariance matrix. and measurement noise covariance matrix Calculate the Kalman gain and update the state to obtain the new background mean. ; Step 2.2.2: First, calculate the pixel values ​​in the current row. To the new mean square distance ; Then, set a quantile threshold. Remove all Pixels larger than this threshold are considered part of the "normal" pixel set. The number of them is ; using sets Calculate the sample covariance of the current row for each cell in the array: ; Then, using a momentum coefficient ,Pick The global covariance of the previous row and the current line Fusion: Introduce a slight diagonal contraction: first calculate the average variance of the current covariance. ,in The sum of the diagonal elements of the matrix is ​​expressed as the sum of the diagonal elements, and finally expressed as the contraction coefficient. ,Pick Convert the covariance matrix to a scalar matrix Slightly zoom in: .

Images