Deep learning matrix decomposition-based infrared small target detection generalization enhancement method
Through the deep learning-based matrix decomposition method, infrared images are decomposed into background low-rank matrix and target sparse matrix, which solves the generalization problem of infrared small object detection and improves the generalization performance and detection effect of the detection network.
Patent Information
- Application Number
- CN202411099202.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-12-06
AI Technical Summary
Existing infrared small object detection methods perform well on training datasets, but their performance declines on untrained datasets, resulting in generalization problems, especially due to the lack of texture and structural information of infrared images, based on the domain shift assumption The method is not suitable for infrared small object detection.
A matrix decomposition method based on deep learning is designed. By constructing a feature matrix decomposition module and optimization loss function, the infrared image is decomposed into a background low-rank matrix and a target sparse matrix. Combined with the deep learning optimization process, the essential features of small infrared targets are learned, thereby Improve the generalization performance of the detection network.
It effectively improves the generalization of infrared small object detection method on different data domains, improves detection effect, reduces overfitting, and enhances detection capabilities in complex situations.
Smart Images

Figure CN119091148A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for enhancing the generalization of infrared small target detection, and relates to the field of image processing, and in particular to a method for enhancing the generalization of infrared small target detection based on deep learning matrix decomposition. Background Art
[0002] Small target detection is widely used in marine rescue, military surveillance, aviation attack and defense, and other fields. Due to the sensitivity of visible light images to lighting conditions, infrared small target detection has attracted great interest. However, infrared small targets are often point-like, lack texture and structural information, and are easily obscured by complex background clutter. Therefore, high-precision detection of infrared small targets remains a challenge.
[0003] Previous work on this challenge can be roughly divided into traditional methods and deep learning-based methods. Traditional methods, such as filtering-based methods, local contrast-based methods, and low-rank decomposition-based methods, are often ineffective in complex situations due to their reliance on hand-crafted features. In recent years, deep learning-based methods have made significant progress. However, directly adopting existing popular detectors does not produce satisfactory performance. Therefore, the current approach to the problem of infrared small target detection first designs a U-Net-based network and achieves a breakthrough in infrared small target detection, which promotes more U-Net-based methods, and these well-optimized methods achieve satisfactory performance.
[0004] However, the current work performs well on the training dataset, but the performance decreases in the untrained dataset, which makes it difficult for the current work to cope with complex and changeable actual situations. In fact, this detection generalization problem has been widely studied in other detection task fields. The core of these works is to decouple domain variation and invariant features based on the domain shift hypothesis. However, due to the lack of texture and structural information in infrared images, the domains of data for different infrared small target detections are almost the same. This makes the current detection generalization method based on the domain shift hypothesis not applicable to the generalization problem of infrared small target detection. Therefore, solving the problem of generalization of infrared small target detection requires a solution for infrared small target features. However, current research has not paid attention to the generalization problem of infrared small target detection. Summary of the invention
[0005] In order to solve the problems existing in the background technology, the present invention provides a method for enhancing the generalization of infrared small target detection based on deep learning matrix decomposition. The present invention analyzes the causes of the generalization problem of infrared small targets. Inspired by the phenomenon that traditional methods do not have performance degradation across data sets, the present invention analyzes the difference between traditional methods and methods based on deep learning, and finds that the key difference lies in the optimization target. The optimization target of the traditional method pays more attention to the essential characteristics of infrared small targets, while the optimization object (loss function) of the deep learning-based method only regards infrared small target detection as a classification problem. The design of the loss function leads to overfitting of the infrared small target detection method based on deep learning. To this end, the present invention designs a new feature matrix decomposition module, combined with the optimization target of the traditional method, to enhance the generalization of the current infrared small target detection method based on deep learning method.
[0006] The technical solution adopted by the present invention is:
[0007] The generalization enhancement method of infrared small target detection based on deep learning matrix decomposition of the present invention includes:
[0008] S1: Construct feature matrix decomposition module and its optimization loss function.
[0009] S2: Collect several infrared images to be detected and construct them into a training set after target annotation. The training set is input into the infrared small target detection network for training. The feature map output by the backbone network of the infrared small target detection network is input into the feature matrix decomposition module for synchronous training. The loss function of the infrared small target detection network and the optimized loss function of the feature matrix decomposition module are added together and used as the overall loss function until the overall loss function converges to obtain the trained infrared small target detection network.
[0010] S3: Collect the infrared image of the target to be detected and input it into the trained infrared small target detection network for processing. After processing, the infrared small target detection network outputs the target detection result image with enhanced generalization and displays it on the monitor, thereby realizing the enhanced generalization of infrared small target detection.
[0011] The method of the present invention performs matrix decomposition on the features of infrared images based on deep learning, decomposes the image into a low-rank matrix containing image background information and a sparse matrix containing image target information, and optimizes the matrix decomposition using the optimization process of deep learning, so that the deep learning neural network learns the essential characteristics of small infrared targets in the process of optimizing matrix decomposition, thereby improving the generalization performance of the network. In the network reasoning process, the feature matrix decomposition module is no longer used.
[0012] In the step S1, the feature matrix decomposition module includes a background low-rank matrix construction head and a target sparse matrix construction head. The background low-rank matrix construction head and the target sparse matrix construction head have the same structure, both of which include sequentially connected upsampling or downsampling operations, convolution layers and matrix size transformation operations.
[0013] In step S1, the optimization loss function L of the feature matrix decomposition module is as follows:
[0014] L=||B|| * +||T||1+||ITB|| F ,
[0015] Among them, ||·|| * ,||·||1and||·|| F They represent the nuclear norm, L1 norm and F norm of the calculation matrix respectively; B and T represent the background features and target features output by the feature matrix decomposition module respectively, and I represents the infrared image input to the infrared small target detection network.
[0016] In the step S2, the infrared image to be detected and the target to be marked specifically includes marking the position of the target area in the infrared image.
[0017] In the step S2, the feature map output by the backbone network of the infrared small target detection network is specifically that the infrared image I is first input into the backbone network of the infrared small target detection network, and the output after processing by any convolution layer in the backbone network and then down-sampling processing is used as the feature map.
[0018] In the step S2, the size of the result of the feature map after being processed by the matrix size transformation operation of the feature matrix decomposition module is consistent with the size of the infrared image input to the infrared small target detection network.
[0019] The present invention takes into account the generalization problem of the current infrared small target detection method based on deep learning in different data domains. The present invention uses deep learning to perform matrix decomposition on the features of the infrared image, decomposes the image into a low-rank matrix containing image background information and a sparse matrix containing image target information, and optimizes the matrix decomposition using the optimization process of deep learning. The deep learning neural network learns the essential characteristics of the infrared small target in the process of optimizing the matrix decomposition, thereby improving the generalization performance of the network. Compared with the existing methods, the present method can effectively improve the generalization of the existing methods in different data domains.
[0020] The beneficial effects of the present invention are:
[0021] The method of the present invention considers the generalization problem of infrared small target detection. The current infrared small target detection method has the problem of performance degradation in other data domains after fitting one data domain. Starting from the optimization problem of infrared small target detection, infrared small target detection is regarded as a decomposition problem of low-rank matrix and sparse matrix, and is solved and optimized in combination with deep learning to complete the detection of infrared small targets. The application of this method can effectively improve the generalization of existing infrared small target detection methods based on deep learning, so that other existing infrared small target detection methods have better infrared small target detection effects in different data domains after applying this method. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 It is a structural diagram of the characteristic matrix decomposition module of the method of the present invention;
[0023] Figure 2 4 is a diagram showing the generalization enhancement effect of an embodiment of the present invention. DETAILED DESCRIPTION
[0024] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0025] The generalization enhancement method of infrared small target detection based on deep learning matrix decomposition of the present invention is as follows:
[0026] S1: Construct feature matrix decomposition module and its optimization loss function.
[0027] like Figure 1 As shown, the feature matrix decomposition module includes a background low-rank matrix construction head and a target sparse matrix construction head. The background low-rank matrix construction head and the target sparse matrix construction head have the same structure, both of which include sequentially connected upsampling or downsampling operations, convolution layers and matrix size transformation operations.
[0028] The optimization loss function L of the feature matrix decomposition module is as follows:
[0029] L=||B|| * +||T||1+||ITB|| F ,
[0030] Among them, ||·|| * ,||·||1and||·|| F They represent the nuclear norm, L1 norm and F norm of the calculation matrix respectively; B and T represent the background features and target features output by the feature matrix decomposition module respectively, and I represents the infrared image input to the infrared small target detection network.
[0031] Using the feature map F obtained by the infrared small target detection network, the feature matrix decomposition module first obtains the background features from the feature map F using upsampling or downsampling and convolution operations according to the downsampling rate n. and target features The size of the background feature B' and the target feature T' are both 64×H / 8×W / 8, and the size of the feature map F is C×H / n×W / n. Then, the background feature B' and the target feature T' are processed by matrix size transformation to obtain the background matrix B and the target matrix T, and the size becomes 1×H×W, where C represents the number of channels of the feature map, H and W represent the height and width of the original infrared image I, respectively, and n represents the downsampling rate. Feature Map Finally, the matrix decomposition optimization loss function L is constructed.
[0032] By optimizing the loss function L through matrix decomposition, the low rank of the background matrix is maximized and the sparsity of the target matrix is maximized. From the perspective of optimization problem, the model can learn the essential characteristics of the infrared small target task during training, thereby improving the generalization of the network.
[0033] S2: Collect several infrared images to be detected and construct them into a training set after target annotation. The training set is input into the infrared small target detection network for training. The feature map output by the backbone network of the infrared small target detection network is input into the feature matrix decomposition module for synchronous training. The loss function of the infrared small target detection network and the optimized loss function of the feature matrix decomposition module are added together and used as the overall loss function until the overall loss function converges to obtain the trained infrared small target detection network.
[0034] The infrared image to be detected and the target to be marked specifically includes marking the target area position in the infrared image.
[0035] The feature map output by the backbone network of the infrared small target detection network is specifically that the infrared image I is first input into the backbone network of the infrared small target detection network, and the output after processing by any convolutional layer in the backbone network and then down-sampling is used as the feature map.
[0036] The size of the result of the feature map after the matrix size transformation operation of the feature matrix decomposition module is consistent with the size of the infrared image input to the infrared small target detection network.
[0037] During the matrix size transformation operation, the background feature B' and the target feature T' are regarded as feature encoding the original image I. That is, the process can be regarded as evenly dividing the image with the original size of H×W into H / 8×W / 8 grids, and each grid contains 8×8 feature codes. Therefore, the size of the background feature B' and the target feature T' can be regarded as the original image divided into H / 8×W / 8 8×8 feature codes, which is changed to 64×H / 8×W / 8 after a certain matrix size transformation. Therefore, a pixel position on the background feature B' and the target feature T' contains the 8×8 feature codes of the pixel area position corresponding to the original image I. On this basis, the matrix transformation described in S12 converts the background feature B' and the target feature T' into the size of the original image I, and obtains the background matrix and the target matrix
[0038] S3: Collect the infrared image of the target to be detected and input it into the trained infrared small target detection network for processing. After processing, the infrared small target detection network outputs the target detection result image with enhanced generalization and displays it on the monitor, thereby realizing the enhanced generalization of infrared small target detection.
[0039] The method of the present invention performs matrix decomposition on the features of infrared images based on deep learning, decomposing the image into a low-rank matrix containing image background information and a sparse matrix containing image target information, and optimizes the matrix decomposition using the optimization process of deep learning, so that the deep learning neural network can learn the essential characteristics of infrared small targets in the process of optimizing matrix decomposition, thereby improving the generalization performance of the network. The infrared small target detection network can use the existing detection network, such as MRF 3 Net, DNANet, and ALCNet. During network inference, the feature matrix decomposition module is no longer used.
[0040] like Figure 2 As shown, the results of small target detection based on two infrared images using the method of the present invention and only using the infrared small target detection network are compared with the annotated images. It can be seen that an erroneous detection target appears in the detection results when the method of the present invention is not applied, while the erroneous detection target is eliminated in the detection results when the method of the present invention is applied. Starting from the essence of the infrared small target detection task, this method regards the infrared small target detection task as a decomposition problem of the background low-rank matrix and the target sparse matrix, and optimizes the decomposition problem in combination with deep learning, so that the network can learn the essence of the infrared small target detection task during the training process, effectively improving the generalization of the current deep learning-based infrared small target detection method in different infrared small target data domains.
Claims
1. A method for enhancing the generalization of infrared small target detection based on deep learning matrix decomposition, characterized in that: include: S1: Construct feature matrix decomposition module and optimize its loss function; S2: Collect several infrared images to be detected and construct them into a training set after target annotation. Input the training set into the infrared small target detection network for training. Input the feature map output by the backbone network of the infrared small target detection network into the feature matrix decomposition module for synchronous training. Add the loss function of the infrared small target detection network and the optimized loss function of the feature matrix decomposition module together as the overall loss function until the overall loss function converges to obtain the trained infrared small target detection network. S3: Collect the infrared image of the target to be detected and input it into the trained infrared small target detection network for processing. After processing, the infrared small target detection network outputs the target detection result image with enhanced generalization and displays it on the monitor, thereby realizing the enhanced generalization of infrared small target detection.
2. The method for enhancing generalization of infrared small target detection based on deep learning matrix decomposition according to claim 1 is characterized in that: In the step S1, the feature matrix decomposition module includes a background low-rank matrix construction head and a target sparse matrix construction head. The background low-rank matrix construction head and the target sparse matrix construction head have the same structure, both of which include sequentially connected upsampling or downsampling operations, convolution layers and matrix size transformation operations.
3. The generalization enhancement method for infrared small target detection based on deep learning matrix decomposition according to claim 1 is characterized in that: In step S1, the optimization loss function L of the feature matrix decomposition module is as follows: L=||B||*+||T||1+||I-T-B|| F , Among them, ||·|| * ,||·||1and||·|| F They represent the nuclear norm, L1 norm and F norm of the calculation matrix respectively; B and T represent the background features and target features output by the feature matrix decomposition module respectively, and I represents the infrared image input to the infrared small target detection network.
4. The method for enhancing generalization of infrared small target detection based on deep learning matrix decomposition according to claim 1, characterized in that: In the step S2, the infrared image to be detected and the target to be marked specifically includes marking the position of the target area in the infrared image.
5. The method for enhancing generalization of infrared small target detection based on deep learning matrix decomposition according to claim 1, characterized in that: In the step S2, the feature map output by the backbone network of the infrared small target detection network is specifically that the infrared image I is first input into the backbone network of the infrared small target detection network, and the output after processing by any convolution layer in the backbone network and then down-sampling processing is used as the feature map.
6. The method for enhancing generalization of infrared small target detection based on deep learning matrix decomposition according to claim 2, characterized in that: In the step S2, the size of the result of the feature map after being processed by the matrix size transformation operation of the feature matrix decomposition module is consistent with the size of the infrared image input to the infrared small target detection network.
Citation Information
Patent Citations
Lightweight convolutional neural network image recognition method based on low rank and sparse decomposition
CN111079781A
Infrared weak and small target detection method based on deep sparse low-rank neural network
CN115510660A
Infrared image small target detection method and system based on transform domain tensor depth expansion
CN118172543A
