Infrared remote sensing image weak and small target detection method based on nested context

Abstract

The invention discloses an infrared remote sensing image weak and small target detection method based on nested context. The method comprises the following steps: firstly, an input image is sent to a three-dimensional dense nesting module TDNM to aggregate information from multiple scales, and features from different semantic levels are adaptively enhanced through an asymmetric context module ACM to generate multi-level features; secondly, in order to make full use of information of different depth levels, the DNACNet fuses shallow features rich in space detail information and deep features containing deep semantic information, and subsequently, the fused features are clustered and merged through an eight-connected neighborhood clustering module, so that the center point of each target area is identified and positioned; the method provided by the invention effectively improves the detection precision of the weak and small target in the infrared remote sensing image, and has a wide application prospect in the aspects of military reconnaissance, civil monitoring, precision guidance and the like.

Description

Technical Field

[0001] This invention relates to the problem of target detection in remote sensing images in the field of deep learning, and in particular to a method for detecting weak targets in infrared remote sensing images based on nested context. Background Technology

[0002] Infrared remote sensing imagery for detecting weak targets is crucial for the safety and efficiency of key areas such as maritime surveillance, early warning systems, and precision guidance. Detecting these targets not only challenges existing image processing techniques but also exhibits unique characteristics in practical applications. These characteristics include: small target size, occupying only a few pixels in the image; low signal-to-clutter ratio, making them easily lost amidst complex noise and background clutter; irregular shape, limiting the effectiveness of detection using traditional shape features; and the variability of target characteristics in different scenarios, further increasing the difficulty of detection.

[0003] Traditional image processing-based methods achieved some success in the early stages, but they often relied on hand-designed features, which proved inadequate when faced with the diverse scenarios in real-world applications. In recent years, convolutional neural network (CNN)-based methods, with their powerful data-driven learning capabilities, have demonstrated superior performance by learning the complex features of weak targets in infrared remote sensing images from large datasets. Although CNN-based methods have shown significant performance advantages in weak target detection in infrared remote sensing images, they still face inherent limitations. The most prominent problem is that weak target features in infrared remote sensing images are easily lost or weakened during multi-layered information transmission, thus affecting the final detection results. This insufficient ability to preserve deep feature information has become a key bottleneck restricting further improvements in the performance of CNN-based methods for weak target detection in infrared remote sensing images. Summary of the Invention

[0004] The purpose of this invention is to provide a weak target detection method for infrared remote sensing images based on nested contexts, called DNACNet. The core of this method lies in the construction of a three-way densely nested module (TDNM) and its effective combination with an asymmetric context module (ACM) strategy. This combination strategy achieves hierarchical interaction and adaptive enhancement of features, greatly improving the accuracy of capturing weak target features. Through the continuous feature fusion and enhancement mechanism of the TDNM module, DNACNet can maintain high sensitivity to weak target features in infrared remote sensing images at a deep network level, significantly improving the accuracy of target detection and the robustness of the system.

[0005] The present invention specifically adopts the following technical solution:

[0006] A method for detecting weak targets in infrared remote sensing images based on nested contexts is proposed. The main features of this method are:

[0007] a. A three-way densely nested module TDNM is proposed and cleverly combined with an asymmetric context module ACM to effectively extract features of weak targets in infrared remote sensing images.

[0008] b. A method for detecting weak targets in infrared remote sensing images, DNACNet, is proposed, which combines TDNM and ACM.

[0009] like Figure 1 As shown, the method mainly includes the following steps:

[0010] (1) The input image is first fed into the three-dimensional densely nested module TDNM to aggregate information from multiple scales, and features from different semantic levels are adaptively enhanced by the asymmetric context module ACM to generate multi-level features.

[0011] (2) The multi-level features output in step (1) are fed into the subsequent feature pyramid for upsampling and connection, aiming to fuse the output features of multiple levels.

[0012] (3) The fusion features output in step (2) are processed by 1×1 convolution and channel adjustment, and then clustered and merged by the eight-connected neighborhood clustering module to facilitate the identification and location of the center point of each target region. Attached Figure Description

[0013] Figure 1 This is a framework diagram of the infrared remote sensing image weak target detection method based on nested context according to the present invention.

[0014] Figure 2 This is a structural diagram of the three-way densely nested module TDNM of the present invention.

[0015] Figure 3 This is a structural diagram of the Asymmetric Context Module (ACM) of the present invention.

[0016] Figure 4 This is a structural diagram of the TDNM (Three-way Dense Nested Module) that incorporates ACM in this invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the following embodiments are only used to further illustrate the present invention and should not be construed as limiting the scope of protection of the present invention. Those skilled in the art can make some non-essential improvements and adjustments to the present invention based on the above-described invention, and these improvements and adjustments should still fall within the scope of protection of the present invention.

[0018] The specific explanation of the Balanced Multimodal Feature Fusion (BFM) module in step (1) is as follows:

[0019] like Figure 2 As shown, assume Q i,j Let L(i,j) represent the output of node L(i,j), where i corresponds to the i-th downsampling layer in the encoder, and j represents the convolutional layer in the j-th dense block on the standard skip connection path. When j=0, the node captures features only through standard dense skip connections, and the output feature map expression of node L(i,j) is:

[0020] Q i,j =P max (F(Q i-1,j (1)

[0021] Where F(·) represents a multi-level cascaded convolutional layer with the same convolutional block. P max (·) indicates max pooling with a step size of 2.

[0022] When j > 0, each node receives output from three directions, including dense ordinary skip connections and nested bidirectional interactive skip connections. The output feature map expression of node L(i,j) is as follows:

[0023]

[0024] Where U(·) represents the upsampling layer, and [·,·] represents the cascaded layer.

[0025] like Figure 4 As shown, ACM is used to enhance the features of nodes L(i-1,j) and L(i+1,j) in TDNM, and then perform subsequent feature fusion with node L(i,j-1). The specific structure of ACM is as follows: Figure 3 As shown, the expression for the above process can be summarized as follows:

[0026]

[0027] Where F(·) represents a multi-level cascaded convolutional layer with the same convolutional block, P max (·) represents max pooling with a stride of 2, G(·) represents top-down attention modulation, and M(·) represents bottom-up context modulation.

[0028] The specific explanation of the DNACNet infrared remote sensing image weak target detection method in steps (2) and (3) is as follows:

[0029] like Figure 1As shown, DNACNet aggregates information from multiple scales through a three-way densely nested module (TDNM), and features from different semantic levels are adaptively enhanced through an asymmetric context module (ACM) to generate multi-level features. Subsequently, DNACNet employs a feature pyramid fusion module to integrate features from different levels to enhance the model's expressive power. To fully utilize information at different depths, DNACNet fuses shallow features rich in spatial detail with deep features containing deep semantic information, thereby constructing an informative and comprehensive global feature map. Following the feature pyramid fusion module, DNACNet introduces an eight-connected neighborhood clustering module to cluster pixels belonging to the same target and calculate the centroid of each target.

[0030] This embodiment uses the publicly available datasets NUDT-SIRST and NUAA-SIRST for training and testing, and employs the detection accuracy P0. d And false alarm rate F a Its localization capability was evaluated, and its shape description capability was assessed using IoU. The training iterations were 1500. Quantitative detection results are shown in Tables 1 and 2, demonstrating the superior detection performance of this invention. Bold text indicates the best experimental results, and underlined text indicates the second-best.

[0031] Table 1. Experimental results of DNACNet and other algorithms on the NUDT-SIRST dataset.

[0032]

[0033] Table 2. Experimental results of DNACNet and other algorithms on the NUAA-SIRST dataset.

[0034]

Claims

1. A method for detecting weak targets in infrared remote sensing images based on nested contexts, characterized in that: a. A three-way densely nested module (TDNM) is proposed and cleverly combined with an asymmetric context module (ACM) to effectively extract features of weak targets in infrared remote sensing images; b. A method for detecting weak targets in infrared remote sensing images, DNACNet, is proposed, which combines TDNM and ACM. This method mainly includes the following steps: (1) The input image is first fed into the three-dimensional densely nested module TDNM to aggregate information from multiple scales, and features from different semantic levels are adaptively enhanced by the asymmetric context module ACM to generate multi-level features. (2) The multi-level features output in step (1) are fed into the subsequent feature pyramid for upsampling and connection, aiming to fuse the output features of multiple levels; (3) The fusion features output in step (2) are processed by 1×1 convolution and channel adjustment, and then clustered and merged by the eight-connected neighborhood clustering module to facilitate the identification and location of the center point of each target region.

2. The three-way densely nested module TDNM as described in claim 1, characterized in that... In step (1), a TDNM for extracting features of weak targets in infrared remote sensing images is designed. The design method is as follows: Since the optimal receptive field differs for targets of different sizes, TDNM stacks multiple sub-networks to construct a densely nested structure, allowing sub-networks of different depths to extract features of targets of different sizes. At the same time, multiple interconnected nodes are added between the sub-networks, and each node fuses its own features and those of its adjacent layers to form a densely nested structural module. This can preserve information about weak targets in infrared remote sensing images, thereby achieving better detection results.

3. The combination of the three-way densely nested module TDNM and the asymmetric context module ACM as described in claim 1, characterized in that... The method for combining the two modules is as follows: The features of nodes L(i-1,j) and L(i+1,j) in the three-way densely nested module TDNM are input into the asymmetric context module ACM. By utilizing top-down global attention modulation and bottom-up local attention modulation, ACM exchanges the multi-scale context of nodes L(i-1,j) and L(i+1,j), which can encode the semantic information and spatial details of the target more richly. Then, the features output by ACM are further fused with the features of node L(i,j-1) to obtain the features of node L(i,j).

4. The DNACNet method for detecting weak targets in infrared remote sensing images combining TDNM and ACM according to claim 1, characterized in that... DNACNet is designed by combining TDNM and ACM. The design method is as follows: By constructing a three-way densely nested module TDNM and combining it with the ACM strategy, this combination effectively achieves layer-by-layer interaction and adaptive enhancement of features, thereby capturing the features of weak targets more accurately. By leveraging the continuous feature fusion and enhancement mechanism of TDNM, DNACNet can maintain its sensitivity to weak target features in infrared remote sensing images deep within the network, effectively improving the accuracy and robustness of weak target detection in infrared remote sensing images. Experimental verification on the publicly available infrared remote sensing image datasets NUDT-SIRST and NUAA-SIRST demonstrates the significant effectiveness of the proposed DNACNet method, fully proving its excellent performance in detecting weak targets in infrared remote sensing images.

Images