Hyperspectral anomaly detection method and device based on factor group sparse regularization

Abstract

The application discloses a hyperspectral anomaly detection method and device based on factor group sparse regularization and belongs to the technical field of remote sensing image processing. The method reduces three-dimensional hyperspectral data into a two-dimensional matrix form, decomposes the matrix into a background part and an anomaly part based on a background dictionary; utilizes a Schatten-p norm to regularize and constrain the low-rank characteristics of the background part, utilizes a 2,1 norm to regularize and constrain the column sparse characteristics of the anomaly part; converts the Schatten-p norm into a factor group sparse regularization form, introduces an auxiliary matrix to replace variables after performing singular value decomposition on a coefficient matrix; iteratively solves the replaced variables by using an alternating direction multiplier method to obtain an anomaly matrix and calculates an anomaly detection value of each pixel. The application avoids a complex calculation process, guarantees detection accuracy and significantly improves detection speed.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, specifically relating to a hyperspectral anomaly detection method and apparatus based on factor group sparse regularization, applicable to anomaly detection tasks in hyperspectral remote sensing images. Background Technology

[0002] Hyperspectral images contain hundreds of continuous spectral bands, capable of recording the unique spectral characteristics of different ground features, providing rich data support for tasks such as ground feature identification and target detection. Hyperspectral anomaly detection aims to identify pixels with significantly different spectral features from the background in an unsupervised manner, and has important application value in fields such as military reconnaissance and environmental monitoring.

[0003] Early anomaly detection methods were mostly based on statistical models, such as the Reed-Xiaoli method and its improved algorithms. These methods are theoretically mature and computationally efficient, but they rely heavily on pre-defined background distribution assumptions and are difficult to adapt to heterogeneous and complex backgrounds composed of multiple types of land cover.

[0004] To address the aforementioned bottlenecks, scholars have proposed a series of anomaly detection methods based on low-rank features, with representative algorithms including robust principal component analysis, low-rank representation methods, and low-rank sparse decomposition methods. These methods abandon explicit assumptions about the background distribution and utilize the inherent structural features of hyperspectral data to separate anomalies from the background, exhibiting better adaptability. However, existing methods generally use the nuclear norm to characterize the low-rank features of the hyperspectral data background. As a convex relaxation of the rank function, the nuclear norm cannot adequately approximate the essence of low-rank features, leading to limited detection accuracy. Furthermore, the solution process of these methods involves complex singular value decomposition operations, resulting in high computational complexity and difficulty in meeting the real-time requirements of practical applications.

[0005] In recent years, some researchers have attempted to use the Schatten-p norm instead of the nuclear norm to obtain more accurate low-rank approximations. However, the direct solution of the Schatten-p norm still faces the problem of low computational efficiency. The factor group sparse regularization method provides a new approach to achieve fast solution of the Schatten-p norm, but this method has not yet been effectively applied in the field of hyperspectral anomaly detection.

[0006] In summary, how to fully utilize the low-rank features of hyperspectral data background to improve detection accuracy while accelerating the solution speed to meet practical application needs is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a hyperspectral anomaly detection method and apparatus based on factor group sparse regularization. By introducing factor group sparse regularization into hyperspectral anomaly detection, and transforming the solution of the Schatten-p norm into a threshold pruning operation after matrix decomposition, singular value decomposition is avoided, computational complexity is reduced, and more accurate characterization of low-rank background features is achieved.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A hyperspectral anomaly detection method based on factor group sparse regularization, the method comprising:

[0010] Step 1: Acquire three-dimensional hyperspectral data and unfold it into a two-dimensional matrix along the band dimension;

[0011] Step 2: Decompose the two-dimensional matrix into a background part and an anomaly matrix part, wherein the background part is represented as the product of a preset background dictionary and a coefficient matrix;

[0012] Step 3: Apply the Schatten-p norm to the low-rank feature of the coefficient matrix for regularization constraints, and apply the 2,1 norm to the column sparsity feature of the abnormal matrix for regularization constraints.

[0013] Step 4: Perform singular value decomposition on the coefficient matrix after the Schatten-p norm regularization constraint, introduce a first auxiliary matrix and a second auxiliary matrix for variable substitution, and apply group sparsity constraint to the row vectors of the first auxiliary matrix.

[0014] Step 5: Iteratively solve the replaced variables using the alternating direction multiplier method to obtain the anomaly matrix;

[0015] Step 6: Calculate the anomaly detection value for each pixel based on the L2 norm of each column vector in the anomaly matrix.

[0016] Furthermore, in step 4, after performing singular value decomposition on the coefficient matrix, the first auxiliary matrix is ​​set to be the product of the left singular matrix and the singular value matrix raised to the power of 1 / 2, and the second auxiliary matrix is ​​set to be the product of the right singular matrix and the singular value matrix raised to the power of 1 / 2. The coefficient matrix is ​​then expressed as the product of the transpose of the first auxiliary matrix and the second auxiliary matrix.

[0017] Furthermore, in step 5, the replaced variables are iteratively solved using the alternating direction multiplier method. Specifically, this includes: fixing the second auxiliary matrix, the outlier matrix, and the Lagrange multipliers of the current iteration step; calculating the gradient descent update value of the first auxiliary matrix using the linearized gradient descent method; and then performing a proximal operator operation on the update value to constrain its row sparsity, thereby obtaining the updated first auxiliary matrix.

[0018] Furthermore, step 5, which iteratively solves the replaced variables using the alternating direction multiplier method, also includes: fixing the updated first auxiliary matrix, the anomaly matrix of the current iteration step, and the Lagrange multipliers; obtaining the update expression of the second auxiliary matrix by solving the closed-form solution; and using Cholesky decomposition to solve the closed-form solution to obtain the updated second auxiliary matrix.

[0019] Furthermore, step 5, which iteratively solves the replaced variables using the alternating direction multiplier method, also includes: fixing the updated first auxiliary matrix, the updated second auxiliary matrix, and the Lagrange multipliers of the current iteration step; calculating the intermediate variable matrix; and using the proximal operator method to constrain the column sparsity of the anomaly matrix to obtain the updated anomaly matrix.

[0020] Furthermore, step 5, which iteratively solves the replaced variables using the alternating direction multiplier method, also includes: after each iteration, updating the Lagrange multipliers and penalty parameters based on the update results of this iteration, calculating the original residual and the dual residual, and stopping the iteration when both the original residual and the dual residual are less than a preset threshold; if the iteration stopping condition is not met, clearing the all-zero rows in the first auxiliary matrix and the all-zero columns corresponding to the all-zero rows in the second auxiliary matrix to achieve automatic rank adjustment.

[0021] Furthermore, in step 6: for each column in the anomaly matrix, the arithmetic square root of the sum of squares of the elements in that column vector is calculated, and the calculation result is used as the anomaly detection value of the corresponding pixel.

[0022] On the other hand, the present invention provides a hyperspectral anomaly detection device based on factor group sparsity regularization, comprising:

[0023] The unfolding module is used to acquire three-dimensional hyperspectral data and unfold it into a two-dimensional matrix along the band dimension.

[0024] The decomposition module is used to decompose the two-dimensional matrix into a background part and an anomaly matrix part, wherein the background part is represented as the product of a preset background dictionary and a coefficient matrix;

[0025] The constraint module is used to perform regularization constraints on the low-rank features of the coefficient matrix using the Schatten-p norm, and to perform regularization constraints on the column sparsity features of the abnormal matrix using the 2,1 norm.

[0026] The replacement module is used to perform singular value decomposition on the coefficient matrix after the Schatten-p norm regularization constraint, introduce a first auxiliary matrix and a second auxiliary matrix for variable replacement, and apply group sparsity constraint to the row vector of the first auxiliary matrix.

[0027] A solution module, configured to iteratively solve the replaced variables by the alternating direction multiplier method to obtain an anomaly matrix;

[0028] A calculation module, configured to calculate an anomaly detection value of each pixel according to the two-norm of each column vector in the anomaly matrix.

[0029] In a third aspect, the present invention provides an electronic device, including: one or more processors; a memory, configured to store one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the foregoing hyperspectral anomaly detection method based on factor group sparse regularization.

[0030] In a fourth aspect, the present invention provides a computer-readable storage medium, on which executable instructions are stored, and when the instructions are executed by a processor, the processor is enabled to implement the foregoing hyperspectral anomaly detection method based on factor group sparse regularization.

[0031] The beneficial effects of the present invention are as follows:

[0032] First, more accurate characterization of low-rank features is achieved. Existing methods use the nuclear norm to constrain the low-rank features of the background part of hyperspectral data, which is a convex relaxation of the rank function and has limited characterization accuracy. The present invention uses the Schatten-p norm (0 < p < 1) to replace the nuclear norm, which can more closely approximate the rank function of the matrix and make more full use of the low-rank features of the hyperspectral data background, thereby significantly improving the detection ability of anomaly targets.

[0033] Second, the calculation efficiency is greatly improved. In the solution process of traditional low-rank methods, singular value decomposition of the matrix is required, and the computational complexity is high. The present invention transforms the singular value decomposition operation into a threshold pruning operation on the decomposed matrix through factor group sparse regularization, and the computational complexity is greatly reduced. Theoretically, it is ensured that the present invention has higher computational efficiency than traditional methods and can meet the real-time requirements of practical applications.

[0034] Third, a complete fast solution scheme is provided. The present invention gives a complete algorithm flow for solving the hyperspectral anomaly detection problem based on factor group sparse regularization using the alternating direction multiplier method. In the solution process, relevant parameters can be automatically updated and the iteration can be automatically stopped according to the detection threshold, which has good robustness and practicability. Description of the Drawings

[0035] Figure 1 It is a flowchart of the hyperspectral anomaly detection method based on factor group sparse regularization of the present invention. Detailed Embodiments

[0036] The present invention will be further described below with reference to the drawings and embodiments.

[0037] like Figure 1 As shown, this invention provides a hyperspectral anomaly detection method based on factor group sparse regularization. It uses the Schatten-p norm to characterize the low-rank features of hyperspectral data, fully utilizing these features to complete the anomaly detection task. In the specific solution process, the alternating direction multiplier method is used to solve the regularization problem, thus accelerating the detection speed. Specifically, the method includes:

[0038] Step 1: Acquire three-dimensional hyperspectral data and unfold it into a two-dimensional matrix along the band dimension;

[0039] Step 2: Decompose the two-dimensional matrix into a background part and an anomaly part, wherein the background part is represented as the product of a preset background dictionary and a coefficient matrix;

[0040] Step 3: Apply the Schatten-p norm to the low-rank feature of the coefficient matrix for regularization constraints, and apply the 2,1 norm to the column sparsity feature of the outlier part for regularization constraints.

[0041] Step 4: Perform singular value decomposition on the coefficient matrix after the Schatten-p norm regularization constraint, introduce a first auxiliary matrix and a second auxiliary matrix for variable substitution, and apply group sparsity constraint to the row vectors of the first auxiliary matrix.

[0042] Step 5: Iteratively solve the replaced variables using the alternating direction multiplier method to obtain the anomaly matrix;

[0043] Step 6: Calculate the anomaly detection value for each pixel based on the L2 norm of each column vector in the anomaly matrix.

[0044] Furthermore, in step 1:

[0045] Three-dimensional hyperspectral remote sensing data (rows is the data length, cols is the data width, and bands is the number of spectral bands) Dimensionality reduction to two-dimensional hyperspectral remote sensing data (bands is the number of spectral bands, N is the number of pixels).

[0046] Furthermore, in step 2:

[0047] Because of the strong correlation between background pixels, matrix X can be decomposed into a background part and an anomalous part:

[0048] (1)

[0049] in, This represents the background portion in hyperspectral data. is a background dictionary composed of background pixels (m is the number of atoms in the dictionary), represents the coefficient matrix corresponding to the background atoms, represents the remaining part, which the model regards as the abnormal part.

[0050] Furthermore, in step 3:

[0051] Based on the low-rank feature of the background part and the sparse feature of the abnormal part, formula (1) can be transformed into formula (2):

[0052] (2)

[0053] Where, represents the rank of the coefficient matrix, represents the 2,1 norm of the abnormal part E, and its definition is , where i represents the i-th column of the abnormal part E, j represents the j-th row of the abnormal part E, this norm encourages the column sparsity of the abnormal part E, and λ is the weight coefficient used to balance the two parts.

[0054] Among many hyperspectral anomaly detection methods based on low-rank features, (the value of the coefficient matrix S) is relaxed to (the nuclear norm of the coefficient matrix S) for further solution. To solve the problems of insufficient representation of the background low-rank feature by the nuclear norm and high computational complexity, the present invention uses the Schatten-p norm to relax , and its definition is , where represents the singular value of the coefficient matrix S, represents the i-th largest singular value of the coefficient matrix S, represents the p-th power of the i-th largest singular value. When the power exponent p = 1, this norm is the nuclear norm. When p approaches 0, this norm is the rank of the coefficient matrix. Therefore, when 0 < p < 1, this norm is closer to the essence of the rank function than the nuclear norm and its variant forms, and has more convenient solution conditions, does not rely on artificial priors, and has more stability and universality.

[0055] Based on the Schatten-p norm, formula (2) is relaxed into the following form:

[0056] (3)

[0057] The present invention uses the factor group sparse regularization (FGSR) method to solve the regularization problem of the Schatten-p norm in formula (3). Considering the number of non-smooth terms and the solution difficulty in the formula after factor group sparse decomposition, take This allows for a more accurate representation of the essence of the rank function and enables a faster solution. Therefore, formula (3) can be transformed into the following form:

[0058] (4)

[0059] Furthermore, in step 4:

[0060] According to the factor group sparsity regularization theory, the coefficient matrix S can be decomposed, and singular value decomposition can be performed on the coefficient matrix S to obtain... , , , Let represent the left singular matrix, the singular value matrix, and the right singular matrix, respectively. , Therefore, formula (4) can be transformed into the following form:

[0061] (5)

[0062] Here, α represents the F-norm penalty strength of matrix B, which can balance the column sparsity constraints of matrix A with the numerical constraints of matrix B.

[0063] Next, we solve formula (5). The augmented Lagrange form of this problem is:

[0064] (6)

[0065] Where Q is the Lagrange multiplier and μ is the penalty parameter. The problem (6) is solved using the Alternating Direction Multiplier Method (ADMM). This method fixes other variables during the solution process, then minimizes one of the variables, and alternately updates all variables to obtain the optimal solution. Assuming the iteration reaches the k-th (index) iteration, the solution steps can be divided into the following steps:

[0066] 1) Fix B, E, Q, update A, the objective function is:

[0067] (7)

[0068] make Then, formula (7) can be simplified to the following form:

[0069] (8)

[0070] Equation (8) is solved using the linearized gradient descent method and the proximal operator method, and the objective function is obtained. The gradient with respect to A is Updating A using the linearized gradient descent method yields:

[0071] (9)

[0072] in, This is a safe step size based on the Lipschitz constant, which ensures that gradient descent does not oscillate. Next, the intermediate result matrix... By performing proximal operators and constraining row sparsity, the final result is obtained. :

[0073] (10)

[0074] in, Representation matrix The L2 norm of the i-th row, (i,:) represents The i-th row.

[0075] 2) Fix A, E, and Q, update B, the objective function is:

[0076] (11)

[0077] make Then formula (11) can be simplified to the following form:

[0078] (12)

[0079] Formula (12) has a closed-form solution, for Differentiate and set it equal to zero:

[0080] (13)

[0081] Further analysis reveals:

[0082] (14)

[0083] According to formula (14), the optimal Cholesky decomposition can be obtained. .

[0084] 3) Fix A, B, and Q, update E, the objective function is:

[0085] (15)

[0086] Similar to the proximal operator pruning operation when updating A, let Then formula (15) can be transformed into:

[0087] (16)

[0088] By constraining the column sparsity of matrix E, we can obtain the final result. :

[0089] (17)

[0090] (:,j) indicates The j-th column; (:,j) indicates The j-th column.

[0091] 4) Fix A, B, and E, update Q:

[0092] (18)

[0093] The algorithm shown in Table 1 below provides a specific solution to problem (6).

[0094] When the coefficient matrix The computational complexity of performing singular value decomposition is O(n log n). After using the alternative methods shown in Table 1, the complexity is reduced to Where r is the initial rank preset by the algorithm, based on the low-rank characteristics of hyperspectral data, This theoretically explains the core reason why the present invention has higher computational efficiency compared to traditional methods.

[0095] Table 1

[0096]

[0097]

[0098] This method was tested on five publicly available real-world datasets and compared with five common hyperspectral anomaly detection algorithms, including RX, RPCA, LSMAD, LSDM_MoG, and LRASR. The RX algorithm is a foundational algorithm and standard baseline in the field of hyperspectral anomaly detection, while the other four algorithms are representative anomaly detection algorithms based on low-rank features of hyperspectral data. The AUC values ​​of each algorithm are shown in Table 2, and the detection times are shown in Table 3. Experimental results demonstrate that the proposed method exhibits the best detection performance on all five datasets, with the highest AUC values, and is the fastest among low-rank feature-based algorithms, demonstrating excellent detection efficiency.

[0099] Table 2

[0100] Table 3

[0101] On the other hand, the present invention provides a hyperspectral anomaly detection device based on factor group sparse regularization, which includes modules capable of implementing the steps of the aforementioned method, specifically including:

[0102] The unfolding module is used to acquire three-dimensional hyperspectral data and unfold it into a two-dimensional matrix along the band dimension.

[0103] The decomposition module is used to decompose the two-dimensional matrix into a background part and an anomaly matrix part, wherein the background part is represented as the product of a preset background dictionary and a coefficient matrix;

[0104] The constraint module is used to perform regularization constraints on the low-rank features of the coefficient matrix using the Schatten-p norm, and to perform regularization constraints on the column sparsity features of the abnormal matrix using the 2,1 norm.

[0105] The replacement module is used to perform singular value decomposition on the coefficient matrix after the Schatten-p norm regularization constraint, introduce a first auxiliary matrix and a second auxiliary matrix for variable replacement, and apply group sparsity constraint to the row vector of the first auxiliary matrix.

[0106] The solver module is used to iteratively solve the replaced variables using the alternating direction multiplier method to obtain the anomaly matrix;

[0107] The calculation module is used to calculate the anomaly detection value of each pixel based on the L2 norm of each column vector in the anomaly matrix.

[0108] Thirdly, the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned hyperspectral anomaly detection method based on factor group sparse regularization.

[0109] Fourthly, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enable the processor to implement the aforementioned hyperspectral anomaly detection method based on factor group sparsity regularization.

[0110] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A hyperspectral anomaly detection method based on factor group sparse regularization, characterized in that, The method includes: Step 1: Acquire three-dimensional hyperspectral data and unfold it into a two-dimensional matrix along the band dimension; Step 2: Decompose the two-dimensional matrix into a background part and an anomaly matrix part, wherein the background part is represented as the product of a preset background dictionary and a coefficient matrix; Step 3: Apply the Schatten-p norm to the low-rank feature of the coefficient matrix for regularization constraints, and apply the 2,1 norm to the column sparsity feature of the abnormal matrix for regularization constraints. Step 4: Perform singular value decomposition on the coefficient matrix after the Schatten-p norm regularization constraint, introduce a first auxiliary matrix and a second auxiliary matrix for variable substitution, and apply group sparsity constraint to the row vectors of the first auxiliary matrix. Step 5: Iteratively solve the replaced variables using the alternating direction multiplier method to obtain the anomaly matrix; Step 6: Calculate the anomaly detection value for each pixel based on the L2 norm of each column vector in the anomaly matrix.

2. The hyperspectral anomaly detection method based on factor group sparse regularization according to claim 1, characterized in that, In step 4, after performing singular value decomposition on the coefficient matrix, let the first auxiliary matrix be the product of the left singular matrix and the singular value matrix raised to the power of 1 / 2, let the second auxiliary matrix be the product of the right singular matrix and the singular value matrix raised to the power of 1 / 2, and express the coefficient matrix as the product of the transpose of the first auxiliary matrix and the second auxiliary matrix.

3. The hyperspectral anomaly detection method based on factor group sparse regularization according to claim 1, characterized in that, In step 5, the replaced variables are iteratively solved using the alternating direction multiplier method. Specifically, this includes fixing the second auxiliary matrix, the outlier matrix, and the Lagrange multipliers of the current iteration step, calculating the gradient descent update value of the first auxiliary matrix using the linearized gradient descent method, and then performing a proximal operator operation on the update value to constrain its row sparsity, thereby obtaining the updated first auxiliary matrix.

4. The hyperspectral anomaly detection method based on factor group sparse regularization according to claim 3, characterized in that, Step 5, which iteratively solves the replaced variables using the alternating direction multiplier method, also includes: fixing the updated first auxiliary matrix, the anomaly matrix of the current iteration step, and the Lagrange multipliers; obtaining the update expression of the second auxiliary matrix by solving the closed-form solution; and using Cholesky decomposition to solve the closed-form solution to obtain the updated second auxiliary matrix.

5. The hyperspectral anomaly detection method based on factor group sparse regularization according to claim 4, characterized in that, Step 5, which iteratively solves the replaced variables using the alternating direction multiplier method, also includes: fixing the updated first auxiliary matrix, the updated second auxiliary matrix, and the Lagrange multipliers of the current iteration step; calculating the intermediate variable matrix; and using the proximal operator method to constrain the column sparsity of the anomaly matrix to obtain the updated anomaly matrix.

6. The hyperspectral anomaly detection method based on factor group sparse regularization according to claim 1, characterized in that, Step 5, which iteratively solves the replaced variables using the alternating direction multiplier method, further includes: after each iteration, updating the Lagrange multipliers and penalty parameters based on the update results of this iteration, calculating the original residual and the dual residual, and stopping the iteration when both the original residual and the dual residual are less than a preset threshold; if the iteration stopping condition is not met, clearing the all-zero rows in the first auxiliary matrix and the all-zero columns corresponding to the all-zero rows in the second auxiliary matrix to achieve automatic rank adjustment.

7. The hyperspectral anomaly detection method based on factor group sparse regularization according to claim 1, characterized in that, In step 6: For each column in the anomaly matrix, calculate the arithmetic square root of the sum of squares of the elements in that column vector, and use the calculation result as the anomaly detection value of the corresponding pixel.

8. A hyperspectral anomaly detection device based on factor group sparse regularization, characterized in that, include: The unfolding module is used to acquire three-dimensional hyperspectral data and unfold it into a two-dimensional matrix along the band dimension. The decomposition module is used to decompose the two-dimensional matrix into a background part and an anomaly matrix part, wherein the background part is represented as the product of a preset background dictionary and a coefficient matrix; The constraint module is used to perform regularization constraints on the low-rank features of the coefficient matrix using the Schatten-p norm, and to perform regularization constraints on the column sparsity features of the abnormal matrix using the 2,1 norm. The replacement module is used to perform singular value decomposition on the coefficient matrix after the Schatten-p norm regularization constraint, introduce a first auxiliary matrix and a second auxiliary matrix for variable replacement, and apply group sparsity constraint to the row vector of the first auxiliary matrix. The solver module is used to iteratively solve the replaced variables using the alternating direction multiplier method to obtain the anomaly matrix; The calculation module is used to calculate the anomaly detection value of each pixel based on the L2 norm of each column vector in the anomaly matrix.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When one or more programs are executed by the one or more processors, the one or more processors implement the hyperspectral anomaly detection method based on factor group sparse regularization as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, enable the processor to implement the hyperspectral anomaly detection method based on factor group sparse regularization as described in any one of claims 1-7.

Images