An infrared dim small target detection method based on deep sparse low-rank neural network

Abstract

The application discloses an infrared dim small target detection method based on a deep sparse low-rank neural network, and belongs to the field of infrared data processing in remote sensing digital image processing. Due to the small volume and low brightness of the infrared dim small target, the target is difficult to be detected from an image, and therefore, the application is proposed to improve the performance of an infrared dim small target detection algorithm in small target detection and solve the problem of inaccurate target detection result in a clutter background. Specifically, the original infrared image is segmented into a series of infrared image blocks by using a sliding window; a target detection model based on target sparse representation and background low-rank constraint is established; each variable of the target detection model is solved by using an alternating direction multiplier method after the infrared image block is input; the model is expanded into a convolutional neural network, and relevant parameters in the model are continuously updated; a target detection result in the obtained reconstructed infrared image block is obtained; and the target detection result of the infrared image is output. The application can obtain good detection results for infrared targets with different attributes in different background environments.

Description

Technical Field

[0001] This invention belongs to the field of infrared data processing in remote sensing digital image processing, specifically relating to an algorithm for detecting weak targets in infrared images based on a deep sparse low-rank neural network. Background Technology

[0002] Infrared target detection with nighttime detection capabilities is crucial in target search and tracking systems, which are widely used in military surveillance and precision-guided weapons. The currently accepted definition of an infrared target as one with a size between 2×2 and 9×9 pixels, or occupying less than 0.15% of the entire image, is a weak infrared target. Infrared search and tracking systems rely on the accurate detection results of target detection methods. However, because infrared targets are often small and have low brightness, they typically lack spatial characteristics. Furthermore, the targets to be detected are often in complex backgrounds or affected by sea clutter or cloud clutter, leading to noise and background frequently being mistakenly detected as targets.

[0003] In recent years, many scholars have dedicated themselves to improving the performance of infrared weak target detection methods and have proposed various detection algorithms for this task. Generally, the proposed infrared weak target detection methods can be broadly classified into two categories: pre-detection tracking algorithms and pre-tracking detection algorithms. Pre-detection tracking methods aim to track infrared weak targets in multiple frames of images based on the assumption of a consistent trajectory. Many pre-detection tracking algorithms are based on 3D matched filtering and maximum likelihood estimation. These algorithms typically require processing hundreds of frames of data, thus generally having high time complexity. Pre-tracking detection models aim to suppress the background and highlight small targets. These methods are currently divided into four types: background spatial consistency-based methods, neural network-based learning models, target saliency-based methods, and infrared block image-based methods. These algorithms all require calculating the characteristics of the target and background, detecting images through target enhancement and background suppression. However, because weak targets lack specific spatial features, pre-tracking detection methods are often affected by background and noise. Summary of the Invention

[0004] To address the challenge of accurate detection of small targets in infrared images under varying environmental conditions, this invention proposes a method for detecting small infrared targets based on a deep sparse low-rank neural network. The method detects small infrared targets by constructing a sparse infrared image patch model, iteratively updates the variables in the model using the alternating direction multiplier method, and then unfolds the constructed model into a convolutional neural network to update the model parameters, thereby obtaining the final target detection result.

[0005] To solve the above problems, the technical solution adopted in this invention, as described in the license, is as follows:

[0006] The key technical points of this invention are as follows: An infrared image to be detected is input, and a sliding window moving from the upper left to the lower right is set in the infrared image to extract local infrared image patches. These local image patches are vectorized to form an image patch matrix. Because background information typically has low-rank characteristics, and weak targets have sparse characteristics in each image patch, we construct an optimization equation to solve the infrared weak target detection problem. Furthermore, a learned sparse transformation is used in the sparse regularization term to suppress edges and corners. The proposed optimization model can be efficiently solved using the alternating direction multiplier method. Finally, the iterative steps of the proposed weak target detection model are expanded into a deep network. The proposed deep sparse low-rank neural network consists of four layers: a sparse prior layer, a low-rank prior layer, a reconstruction layer, and a multiplier update layer. Multiple convolutional neural network layers are designed to train the learned sparse transformation. The parameters in the network are trained and shared between different layers.

[0007] The technical solution adopted by this invention to solve the problem of infrared weak target detection is as follows:

[0008] Step 1: Input an infrared image f with spatial dimensions of M×N. D Design a sliding window w that moves from the infrared image f in fixed steps s. D Move the top left corner to the bottom right corner and extract the infrared image f D The extracted local image patches are then vectorized to form an infrared block image D with a spatial size of m×n.

[0009] Step 2: Utilizing the low-rank and sparse characteristics of the background and target in the infrared patch image, a convex optimization equation is established to detect weak infrared targets. Based on the background consistency assumption, it is deduced that the background B of the infrared patch image is a low-rank matrix. Simultaneously, the difference between the target and the background is used to deduce that the target infrared patch image T is a sparse matrix.

[0010] Step 3: Enhance the objective sparse constraint term in the convex optimization equation using sparse learning, and reconstruct the established convex optimization problem using the augmented Lagrange algorithm. An auxiliary variable h is introduced for solving the target image T. Based on this, the alternating direction multiplier method is used to iteratively solve for the target infrared block image T, the infrared block background image B, the auxiliary variable h, and the Lagrange multiplier U, respectively.

[0011] Step 4: Unfold the established sparse object detection model into a convolutional neural network, and design sparse prior layers, low-rank prior layers, reconstruction layers, and multiplier update layers to update the parameters involved in solving for each variable and auxiliary variable. By constructing a neural network, these parameters are shared and optimized to find the optimal solution.

[0012] Step 5: Determine whether the infrared block image D and the iteratively output target infrared block image T and infrared block background image B satisfy the convergence condition. If they do, output the target infrared block image T and infrared block background image B; otherwise, return to step 3 to continue the iteration loop.

[0013] Step Six: Using the vectorization method described in Step One, reconstruct each column of the output target infrared block image T and infrared block background image B into a local image block. Then, stitch the local image blocks together according to the step size of the sliding window. Take the median pixel value within each local image block corresponding to the same pixel position in the original image as the final grayscale value in the output image, thus obtaining the infrared weak target detection result.

[0014] The steps in step two above are as follows:

[0015] By constructing a convex optimization equation, the infrared weak target detection problem can be expressed as follows:

[0016]

[0017] Where D is the infrared patch image, N is noise, λ is the regularization parameter, ||·||0 represents the l0 norm, and rank(·) represents the rank of the matrix. Since solving for the l0 norm is an NP-hard problem and cannot be solved directly, the l1 norm is used instead of the l0 norm, and the rank of the matrix is ​​solved using the kernel norm. Then, formula (1) can be rewritten as follows:

[0018]

[0019] Among them, ||·|| * Represents the nuclear norm, ||·||1 represents the l1 norm, ||·|| F δ represents the Frobrnius norm, which is a constant close to 0.

[0020] The specific process of step three is as follows:

[0021] The targets detected using formula (2) are usually affected by image background and noise; therefore, by using sparse transformation to enhance the sparsity of the detected targets, formula (2) can be expressed as follows:

[0022]

[0023] in, It is a sparse transformation; by optimizing formula (3) using the augmented Lagrange multiplier method, the following expression can be obtained.

[0024]

[0025] Where ρ is the penalty parameter, Y is the Lagrange multiplier, and λ1 and λ2 are regularization parameters; B and T in formula (4) are solved using the alternating direction multiplier method, as shown below.

[0026]

[0027] in, This refers to the update rate; the singular value thresholding algorithm is used to solve for B, as shown below.

[0028]

[0029] Where, σ i u i and v i It is the i-th value obtained after performing singular value decomposition on D-T+U, and max(·) represents finding the maximum value; B t Let B represent the value of B generated in the t-th iteration; solving for variable T is a linear inverse problem, which is solved using an iterative shrinking threshold algorithm; specifically, the auxiliary variable h is solved alternately using the iterative shrinking threshold algorithm. t and T t As shown below

[0030]

[0031]

[0032] in It is the update rate, T t The soft thresholding method can be used to solve this problem.

[0033] The specific process of step four is as follows:

[0034] Design reconstruction layer h t According to formula (7), the reconstruction result of the t-th iteration is related to the output of the (t-1)-th iteration, i.e., B t-1 U t-1 and T t-1 In the first iteration, B0, U0, and T0 are set to D; hyperparameters It can be updated in each iteration; The initial value is 0.1; then, a sparse prior layer T is designed. t This layer enhances the target T by continuously updating according to formula (8). t To leverage the sparsity properties of sparse neural networks, a 6-layer convolutional neural network was designed. Through learning, training, and validation on hundreds of sets of training data, a complete sparse transformation matrix was obtained. To expand network capacity, in formula (8) and Each layer utilizes a three-layer convolutional neural network for learning and obtains... and Design a low-rank prior layer B t According to formula (6), B t The solution is obtained through singular value decomposition with a soft thresholding scheme; in a deep sparse low-rank neural network, the threshold is learned to obtain an optimal value, as shown below.

[0035]

[0036] Design the multiplier update layer U t The Grand multipliers are updated to formula (5) at this layer. These are the hyperparameters trained in a deep sparse low-rank neural network, initialized to 0.1; parameters as well as and The designed network is continuously learned and updated, and parameters are shared among the various networks, ultimately outputting a complete object detection network.

[0037] The present invention has the following beneficial technical effects:

[0038] This invention proposes an infrared weak target detection method based on a deep sparse low-rank neural network (LS-IPINet). This method establishes a convex optimization equation for target detection using low-rank constraint regularization terms on the background space and sparse constraint regularization terms on the weak target. The alternating direction multiplier method is used to accelerate the solution of the constructed convex optimization equation. The iterative steps are then expanded into a deep learning network, and multi-layer deep convolutional modules are established to automatically iteratively optimize the model parameters. The negative feedback mechanism and loss function of deep learning are used to update and constrain the parameters, ensuring optimal output parameters and thus obtaining optimal target detection results. Compared to weak target detection methods based on densely nested attention networks (DNANet), attention pyramid context networks (AGPCNet), log tensor kernel norm networks (LogTFNN), partial tensor kernel norm networks (PSTNN), and infrared image patches (IPI), the algorithm of this invention can achieve accurate detection of different types of weak targets under various environmental background conditions.

[0039] Due to their small size and low brightness, infrared targets are difficult to detect in images. Therefore, this invention aims to improve the performance of infrared target detection algorithms for small targets and address the problem of inaccurate target detection results in cluttered backgrounds. Specifically, it includes: segmenting the original infrared image into a series of infrared image patches using a sliding window; establishing a target detection model based on sparse target representation and low-rank background constraints; inputting the infrared image patches and solving for the variables of the target detection model using the alternating direction multiplier method; unfolding the proposed model into a convolutional neural network and continuously updating the relevant parameters; reconstructing the target detection results in the obtained infrared image patches; and outputting the target detection results of the infrared image. This invention achieves good detection results for infrared targets with different properties under various background environments. Attached Figure Description

[0040] Figure 1 This is a flowchart of the algorithm of the present invention (flowchart of the infrared weak target detection method based on LS-IPINet).

[0041] Figure 2 This is a schematic diagram of infrared block image generation.

[0042] Figure 3 This is a schematic diagram of the low-rank background of an infrared block image.

[0043] Figure 4 This is a schematic diagram illustrating the conversion of the output infrared block image into detection results.

[0044] Figure 5 The image shows the detection results of various weak target detection methods.

[0045] Figure 6 This provides a 3D visualization of the detection results from various weak target detection methods.

[0046] Figure 7 A diagram illustrating the expansion of objectives, point objectives, and weak objectives.

[0047] Figure 8 (a) shows the ROC curves of multiple point target images.

[0048] Figure 8 (b) shows the ROC curves of multiple weak targets and point targets.

[0049] Figure 8 In the middle (c), there are multiple ROC curves for all types of images.

[0050] Figure 9 This is the object detection result of LS-IPINet in the NUST-SIRST dataset. Specific implementation methods

[0051] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0052] like Figure 1 As shown, the specific implementation steps of the present invention are as follows:

[0053] (1) Input an infrared image f with spatial dimensions of M×N. D Design a sliding window w that moves from the infrared image f in fixed steps s. D Move the top left corner to the bottom right corner and extract the infrared image f D Local image patches, such as Figure 2 As shown in the figure. All extracted local image patches are then vectorized to form an infrared patch image D with a spatial size of m×n.

[0054] (2) Utilizing the low-rank and sparse characteristics of the background and target in infrared block images, a convex optimization equation is established to detect weak infrared targets. Due to the influence of thermal radiation and diffraction, the extracted local image blocks are correlated with the surrounding image blocks, exhibiting non-local autocorrelation characteristics. Since the background usually changes relatively smoothly and the correlation between pixels is high, the background B of the infrared block image is a low-rank matrix. Conversely, the pixel intensity of weak targets usually differs significantly from the intensity of surrounding pixels, so the infrared block image T of the target is a sparse matrix.

[0055] (3) Sparse learning is used to enhance the objective sparse constraint term in the convex optimization equation, and the augmented Lagrange algorithm is used to reconstruct the established convex optimization problem. An auxiliary variable h is introduced for solving the target image T. Based on this, the alternating direction multiplier method is used to iteratively solve the target infrared block image T, the infrared block background image B, the auxiliary variable h, and the Lagrange multiplier U.

[0056] (4) The established sparse target detection model is unfolded into a convolutional neural network. A sparse prior layer, a low-rank prior layer, a reconstruction layer, and a multiplier update layer are designed to update the parameters involved in solving for each variable and auxiliary variable. By constructing a neural network, these parameters are shared and optimized to seek the optimal solution.

[0057] (5) Determine whether the infrared block image D and the target infrared block image T and infrared block background image B output by the iteration meet the convergence condition. If they meet the condition, output the target infrared block image T and the infrared block background image B; if they do not meet the condition, return to step (3) to continue the iteration loop.

[0058] (6) Restore each column of the output target infrared block image T and infrared block background image B to a local image block using the vectorization method in step (1). Then, stitch the local image blocks together according to the step size of the sliding window. Figure 4As shown, the median value of the pixel values ​​in each local image block corresponding to the same position in the original image is taken as the final calculated value in the output image, thus obtaining the infrared weak target detection result.

[0059] In the steps of the above infrared weak target detection method, the specific operation of step (2) is as follows:

[0060] Due to the influence of thermal radiation and diffraction, the extracted local image patches are correlated with their surrounding image patches, exhibiting nonlocal autocorrelation characteristics. For example... Figure 3 As shown, the first row of images in the figure are the original infrared images, and the second row of images are the singular value curves of the corresponding infrared patch images. All curves decay rapidly to 0, indicating that the background B of the infrared patch image is a low-rank matrix. Conversely, the pixel intensity of small targets usually differs significantly from the intensity of surrounding pixels, thus the target infrared patch image T is a sparse matrix. Therefore, the infrared small target detection problem can be represented as...

[0061]

[0062] Where D is the infrared patch image, N is the noise, λ is the regularization parameter, ||·||0 represents the l0 norm, and rank(·) represents the rank of the matrix. However, solving for the l0 norm is an NP-hard problem and cannot be solved directly. Therefore, we use the l1 norm instead of the l0 norm and use the kernel norm to solve for the rank of the matrix. Then, formula (1) can be rewritten as follows:

[0063]

[0064] Among them, ||·|| * Represents the nuclear norm, ||·||1 represents the l1 norm, ||·|| F δ represents the Frobrnius norm, which is a constant close to 0.

[0065] The specific steps for step (3) are as follows:

[0066] The targets detected using formula (2) are often affected by image background and noise. Therefore, we utilize sparse transformation. To enhance the sparsity of the detection targets, formula (2) can be expressed as follows:

[0067]

[0068] in, It is a sparse transformation.

[0069] By optimizing formula (3) using the augmented Lagrange multiplier method, the following expression can be obtained.

[0070]

[0071] Where ρ is the penalty parameter, Y is the Lagrange multiplier, and λ1 and λ2 are regularization parameters. Then, B and T in formula (4) are solved using the alternating direction multiplier method, as shown below.

[0072]

[0073] in, This refers to the update rate. The singular value thresholding algorithm is used to solve for B, as shown below.

[0074]

[0075] Where, σ i u i and v i It is the i-th value in the result obtained after performing singular value decomposition on D-T+U, and max(·) represents finding the maximum value. B t Let represent the value of B produced in the t-th iteration. Solving for variable T is a linear inverse problem, which is solved using an iterative shrinking threshold algorithm. Specifically, we use the iterative shrinking threshold algorithm to alternately solve for the auxiliary variable h. t and T t As shown below

[0076]

[0077]

[0078] in It is the update rate, T t The soft thresholding method can be used to solve this problem.

[0079] The specific steps for step (4) are as follows:

[0080] All iterations of the object detection model described in step (3) are expanded into the designed deep network to obtain a deep sparse low-rank neural network. This network is designed to learn and update some parameters in the object detection model, including... Simultaneously, we train and learn coefficient transformations in the sparse prior layer. Solving Equation (8) involves... and We treat these as two coefficient transformations for training and learning respectively. The design of each layer in the deep sparse low-rank neural network is described below:

[0081] 1) Design the reconstruction layer h t According to formula (7), the reconstruction result of the t-th iteration is related to the output of the (t-1)-th iteration, i.e., B t-1 Ut-1 and T t-1 In the first iteration, the initial values ​​of B0, U0, and T0 are set to D. Hyperparameters It can be updated in each iteration. The initial value is 0.1.

[0082] 2) Design of sparse prior layer T t This layer enhances the target T by continuously updating according to formula (8). t To leverage the sparsity properties of sparse transformation matrices, we designed a 6-layer convolutional neural network. Through learning, training, and validation on hundreds of sets of training data, we obtained a complete sparse transformation matrix. To expand network capacity, in formula (8) and Each layer utilizes a three-layer convolutional neural network for learning and obtains... and

[0083] 3) Design the low-rank prior layer B t According to formula (6), B t The solution is obtained through singular value decomposition with a soft thresholding scheme. In a deep sparse low-rank neural network, the threshold is learned to obtain the optimal value, as shown below.

[0084]

[0085] 4) Design the multiplier update layer U t The Lagrange multipliers are updated to Equation (5) at this layer. These are hyperparameters trained in a deep sparse low-rank neural network, initialized to 0.1.

[0086] To demonstrate the effectiveness of this invention, the following experiment was designed:

[0087] The effectiveness of this invention will be demonstrated through the following experiments.

[0088] 1. Experimental data:

[0089] We evaluate a target detection model based on a deep sparse low-rank neural network on the publicly available infrared target detection dataset NUAA-SIRST, which contains 427 representative images and includes 480 different target types. In the NUAA-SIRST dataset, approximately 90% of the images contain a single target, approximately 55% of the targets occupy only 0.02% of the image, approximately 65% ​​of the targets are dark, and only 35% of the targets can be detected based on brightness. In our experiments, we first resize the image spatial dimensions to 300×300, and then extract 50%, 20%, and 30% of the images from the original dataset for training, validation, and testing, respectively. To verify the robustness of the target detection model, we conduct robustness validation experiments on the publicly available infrared target detection dataset NUST-SIRST. NUST-SIRST contains 10,000 infrared small target images with a spatial dimension of 128×128. The backgrounds in this dataset are diverse, including clouds, cities, rivers, and roads. The small targets in this dataset are either separated from other real-world infrared small target datasets or synthesized using a two-dimensional Gaussian function. However, some infrared targets in the NUST-SIRST dataset are too large to meet the definition of small targets. Therefore, we selected 50 images containing different types of weak small targets for robustness experiments.

[0090] 2. Experimental Results and Analysis

[0091] To verify the superiority of the proposed algorithm in infrared weak target detection, we compared weak target detection methods based on densely nested attention networks (DNANet), attention pyramid context networks (AGPCNet), log tensor kernel norm networks (LogTFNN), partial tensor kernel norm networks (PSTNN), and infrared image patches (IPI). All data-driven models were retrained on the NUAA-SIRST dataset.

[0092] exist Figure 5 In this paper, we present the results of weak target detection in seven images representing seven typical scenes. To unify the output format of different target detection methods, we segment the target and background of the grayscale images into binary images with an adaptive threshold, T. adp The selection is as follows:

[0093] T adp =max[max(M)×0.7,0.5×σ] M +μ M (10)

[0094] Where M represents the detection result of any target, max(·) represents the maximum value in the matrix, and σ Mμ is the standard deviation of matrix M. M This is the average gray value of matrix M. Figure 5 In the diagram, the target detection results for the same image are listed in the same row, and the target detection results for the same method are listed in the same column.

[0095] For the first image, LogTFNN and PSTNN failed to fully extract the target, and IPI also failed to detect it. The target shape detected by AGPCNet differed slightly from the ground truth image, which might affect the application of subsequent detection results. LS-IPINet and DNANet preserved the shape of the detected target well. The second input image was designed to test the multi-target detection capabilities of these models. The detection results showed that each algorithm could accurately locate the target. We magnified the detected targets. Experimental results showed that among all models, the algorithm proposed in this invention preserved the target shape best, especially the target highlighted by the second magnified rectangle in the image. The target in the third image was a weak target. The target had low contrast with the background, making detection difficult. Due to the low contrast, IPI could not separate the target from the background using low-rank regularization, as could AGPCNet. LogTFNN and PSTNN could detect some pixels of the target. DNANet and LS-IPINet detected the target well. The target in the fourth image was a weak, isolated point target in a complex background. Because its intensity differed from the surrounding pixels, each algorithm could accurately locate the target. Compared to the targets detected by DNANet and AGPCNet, the targets detected by IPI, LogTFNN, and PSTNN algorithms are relatively small. The target in the fifth image is difficult to observe. IPI and DNANet cannot locate the target in this image. PSTNN can detect the target's location. LogTFNN, AGPCNet, and LS-IPINet achieve good detection results. However, LogTFNN has a high false alarm rate in its target detection results. The sixth and seventh images are used to test the ability of these models to preserve the shape of the detected targets. The IPI model can hardly detect the target in both images. LogTFNN, PSTNN, and AGPCNet can detect the location of two targets, but cannot completely preserve the shape of the targets. DNANet and LS-IPINet can preserve the shape of two targets very well. The performance of DNANet and LS-IPINet is not affected by background environmental information. Overall, our proposed LS-IPINet can accurately detect various types of targets and preserve the target shape in different backgrounds.

[0096] The 3D visualization results of all models are as follows Figure 6As shown in the diagram, the false positives and false negatives can be directly observed from the 3D visualization results. It can be observed that the IPI and LogTFNN models consistently misdetect background elements as targets. Due to the weak targets and complex backgrounds in the third and fifth images, most target detection models fail to detect targets effectively in these two images. The targets detected by our proposed LS-IPINet are clearly observable in the 3D visualization results, with fewer false positives.

[0097] like Figure 7 As shown, based on target type, we categorize the infrared images in the NUAA-SIRST dataset into three classes: extended target images, point target images, and weak target images. The results of different algorithms are plotted as follows: Figure 8 The ROC curve shown is from... Figure 8 (a) and Figure 8 As shown in (b), the performance of the IPI, LogTFNN, and PSTNN algorithms decreases to some extent with the increase in the proportion of weak targets. Because weak targets are darker and smaller than point targets, they are more difficult to detect, especially against cluttered backgrounds. Conversely, the performance of the DNANet and LS-IPINet algorithms is improved. Figure 8 Figure (c) shows that the DNANet and LS-IPINet algorithms are robust to various scenarios, while our proposed LS-IPINet achieves the highest detection accuracy.

[0098] like Figure 9 As shown, we compared the detection results of the manually labeled NUST-SIRST ground truth map with those of the LS-IPINet proposed in this invention. We observed that the shape of the detected target in the LS-IPINet detection result is closer to the ground truth shape than the shape in the ground truth map. This also verifies that the object detection method proposed in this invention can achieve good object detection results on different training datasets.

[0099] We tested the performance of the proposed LS-IPINet using different amounts of training data on the NUAA-SIRST dataset. The training data amounts were 50, 100, 150, 200, 250, and 300. We selected 127 images as test data, which were not included in the training data. The quantitative results are listed in Table 1, where the best values ​​in each column are indicated in bold, and the second-best values ​​are marked with an underline.

[0100] Table 1 Comparison of performance evaluation metrics of LS-IPINet algorithm under different training data volumes

[0101]

[0102] In Table 1, we observed that the IoU increased as the training data increased from 50 to 250. However, the IoU dropped to 0.6758 with 300 training data, lower than the IoU with 100 training data. nIoU reached its highest point with 100 training data. Although the nIoU value declined with increasing training data, the nIoU for training data of 100, 150, and 200 was higher than the nIoU for other training data groups. The AUC values ​​did not change significantly with different training data amounts. The AUC reached its highest value with 300 training data, and the AUC values ​​for different training data amounts were all above 0.93. SCRGs were all greater than 83 for different training data amounts, indicating that these trained LS-IPINets could effectively highlight targets in infrared images. However, the optimal values ​​for different evaluation metrics were distributed across models with different training data amounts, making it difficult to determine the optimal amount of training data. We observed that most suboptimal values ​​were found in the results with 200 training data sets, indicating that LS-IPINet trained with 150 to 200 images has more comprehensive performance. Meanwhile, we observed that the IoU, nIoU, and AUC with 50 training data sets were only 8.19%, 8.7%, and 3.2% lower than the optimal values, respectively. Compared to models trained with other datasets, LS-IPINet trained with 50 training data sets also achieved good performance. Furthermore, the performance of LS-IPINet at this scale is close to that of DNANet and AGPCNet. In addition, compared to the LS-IPINet model trained with 300 training data sets, LS-IPINet trained with 50 sets of data saves significantly more training time.

Claims

1. A method for detecting weak infrared targets based on a deep sparse low-rank neural network, characterized in that, The method includes the following steps: Step 1: Enter a space with dimensions of [size missing] infrared images Design a sliding window According to a fixed step size From infrared images Move the top left corner to the bottom right corner to extract the infrared image. Local image patches are extracted; and all extracted local image patches are vectorized to form a spatial size of Infrared block image ; Step 2: Utilizing the low-rank and sparse characteristics of the background and target in infrared block images, establish a convex optimization equation to detect weak infrared targets; based on the background consistency assumption, deduce the background of the infrared block image. It is a low-rank matrix; simultaneously, it uses the difference between the target and the background to derive the infrared patch image of the target. It is a sparse matrix; Step 3: Enhance the objective sparse constraint terms in the convex optimization equation using sparse learning, and reconstruct the established convex optimization problem using the augmented Lagrangian algorithm; for the target image The solution introduces auxiliary variables Based on this, the alternating direction multiplier method is used to iteratively solve the target infrared block image. Infrared block background image Auxiliary variables and Lagrange multipliers ; Step 4: Unfold the established sparse target detection model into a convolutional neural network, and design sparse prior layers, low-rank prior layers, reconstruction layers, and multiplier update layers to update the parameters involved in solving each variable and auxiliary variable; by constructing a neural network, these parameters are shared and optimized to seek the optimal solution. Step 5: Determine the infrared block image With the target infrared block image of the iterative output Infrared block background image Check if the convergence condition is met; if so, output the target infrared block image. and infrared block background image If the condition is not met, return to step three and continue the iteration loop. Step 6: Output the target infrared block image and infrared block background image Each column in the image is restored to a local image patch using the vectorization method in step one; the local image patches are then stitched together according to the step size of the sliding window; the median value of the pixel values ​​in each local image patch corresponding to the same position in the original image is taken as the final gray value in the output image, thus obtaining the infrared weak target detection result.

2. The infrared weak target detection method based on a deep sparse low-rank neural network according to claim 1, characterized in that: The specific process of step two is as follows: Step 21: Due to the influence of thermal radiation and diffraction, the extracted local image patches are correlated with their surrounding image patches, exhibiting non-local autocorrelation characteristics. By solving for singular values ​​in the original infrared image, it can be found that there are very few non-zero singular values, indicating that the background image of the infrared patch image is... It is a low-rank matrix; conversely, the pixel intensity of a weak target usually differs significantly from the intensity of surrounding pixels in the infrared block image of the target. It is a sparse matrix; therefore, the infrared small target detection problem can be expressed as: (1) in, It is an infrared block image. It's noise. It is a regularization parameter. represent Norm, Represents the rank of the matrix; Step Two Two Solving for the norm is an NP-hard problem and cannot be solved directly; therefore, we use... Norm substitution If we take the nuclear norm and use it to find the rank of the matrix, then equation (1) can be rewritten as follows: (2) in, Represents the nuclear norm number. represent Norm, Represents the Frobrnius norm. It is a constant close to 0.

3. The infrared weak target detection method based on a deep sparse low-rank neural network according to claim 2, characterized in that: The specific process of step three is as follows: Step 3. Targets detected using formula (2) are usually affected by image background and noise; therefore, by using sparse transformation to enhance the sparsity of target detection, formula (2) can be expressed as: (3) in, It is a sparse transformation; Step 3.2: Optimize formula (3) using the augmented Lagrange multiplier method to obtain the following expression. (4) in, It is a penalty parameter. It is a Lagrange multiplier. and It is a regularization parameter; Step 3: Solve the equation (4) using the alternating direction multiplier method. and As shown below (5) in, , It is the update rate; Steps three and four, for The solution uses the singular value thresholding algorithm, as shown below. (6) in, , and Yes The result obtained after performing singular value decomposition is the first A number, This represents finding the maximum value; Representing the The next iteration generated The value; Step 3.5, Variables Solving this problem is a linear inverse problem, which is solved using an iterative shrinking threshold algorithm; specifically, the auxiliary variables are solved alternately using the iterative shrinking threshold algorithm. and As shown below (7) (8) in It's the update rate. The soft thresholding method can be used to solve this problem. .

4. The infrared weak target detection method based on a deep sparse low-rank neural network according to claim 3, characterized in that: The specific process of step four is as follows: Step 41: Design the Reconstruction Layer According to formula (7), the reconstruction result of the t-th iteration is related to the output of the (t-1)-th iteration, that is... , and ; In the first iteration, , and Set as Hyperparameters It can be updated in each iteration; The initial value is 0.1; Step 4.2: Design a sparse prior layer ; This layer enhances the target based on the continuous updates of formula (8). To leverage the sparsity properties of sparse neural networks, a 6-layer convolutional neural network was designed. Through learning, training, and validation on hundreds of sets of training data, a complete sparse transformation matrix was obtained. In order to expand network capacity, in formula (8) and Each layer utilizes a three-layer convolutional neural network for learning and obtains... and ; Step 4.3: Design a low-rank prior layer ; According to formula (6), Solved by singular value decomposition with a soft threshold scheme; In deep sparse low-rank neural networks, the threshold is learned to obtain an optimal value, as shown below. (9) Step 4: Design the Multiplier Update Layer The Grand multipliers are updated to formula (5) at this layer. These are hyperparameters trained in a deep sparse low-rank neural network, initialized to 0.1; Steps four and five, parameters as well as and The designed network is continuously learned and updated, and parameters are shared among the various networks, ultimately outputting a complete object detection network.

Images