An infrared small target detection method and device based on a pyramid morphological network

Abstract

The application discloses an infrared small target detection method and device based on a pyramid morphological network, and the method comprises the following steps: inputting an infrared image to be detected into a feature extraction network, wherein the network comprises a learnable direction interaction attention module and a spatial pyramid pooling morphological module; the learnable direction interaction attention module performs multi-direction feature enhancement on an input feature map and fuses different branch features through learnable direction rotation and energy attention cooperation; and the spatial pyramid pooling morphological module firstly performs multi-scale spatial pyramid pooling to obtain aggregated features, and then performs morphological operation on the aggregated features to realize spatial structure purification. The application breaks through the limitation of fixed direction attention through learnable direction rotation, compensates for the defect of losing detailed information in pure pooling operation through a cascaded structure of pooling first and then morphological operation, and effectively improves the detection precision and robustness of the infrared small target under the condition of extremely low signal-to-noise ratio.

Description

Technical Field

[0001] This invention relates to the field of infrared image target detection technology, and in particular to an infrared small target detection method and apparatus based on a pyramid morphological network. Background Technology

[0002] Infrared detection technology, with its unique advantages of all-weather operation, strong anti-interference capabilities, and good concealment, plays an irreplaceable role in military reconnaissance, precision guidance, low-altitude defense, and civil aviation safety. Compared with visible light sensors, infrared imaging systems can capture the temperature radiation information of targets and are not limited by lighting conditions, making them a core technology for long-range target detection and tracking.

[0003] Infrared aircraft, as typical high-value aerial targets, are crucial for airspace security and national defense capabilities, and their reliable detection is directly related to these factors. However, in actual detection scenarios, due to the long detection distance and high target altitude, infrared aircraft targets often occupy only a very small number of pixels in the image. According to the definition of the International Society for Optics and Photonics (ISO), a target occupying less than 0.12% of the total pixels in an image can be classified as a small target. These targets are not only small in size, but also suffer from low signal strength and poor contrast due to atmospheric attenuation, sensor noise, and environmental thermal radiation interference during long-distance transmission, making them easily obscured by complex background clutter. Therefore, infrared ground-to-air target detection is essentially a challenging problem of detecting small targets, and its core challenges can be summarized as follows:

[0004] (i) Lack of target features, making identification difficult. Infrared targets typically appear as unstructured or weakly structured spots in images, lacking the rich color, texture, and edge features found in visible light images. Targets often consist of only a few to tens of pixels, failing to provide effective shape information, making it difficult for traditional target detection algorithms based on handcrafted features to establish reliable target representation models.

[0005] (ii) Complex and variable background with low signal-to-noise ratio. The background in infrared images often contains strong structural information such as clouds, ground features, and sea surfaces. These background areas may exhibit drastic grayscale variations and edge textures. At the same time, the thermal noise, readout noise, and non-uniform noise of the infrared detector itself further degrade image quality. The thermal contrast between the target and the background is significantly affected by changes in ambient temperature and time; the infrared characteristics of the same scene may be drastically different at different times.

[0006] (III) Failure of General Detection Methods and Poor Domain Adaptability. In recent years, deep learning-based target detection technologies (such as the YOLO series and Faster R-CNN) have achieved great success in visible light image detection tasks. However, when these general detection frameworks are directly transferred to infrared small target detection tasks, performance drops drastically. The main reasons are: First, the pooling layers and downsampling operations in general detection networks further compress the feature responses of small targets, causing the targets to completely disappear in deep feature maps; second, the construction of existing large-scale target detection datasets is mainly based on visible light images, while infrared image data is scarce and the annotation cost is high, making it difficult for models to obtain sufficient training; in addition, the anchor box scale preset in general detection frameworks is usually seriously mismatched with the actual size of infrared small targets, resulting in scarce positive samples and low detection recall.

[0007] Overall, current research is focused on improving the accuracy and robustness of infrared weak target detection by fusing contextual information, introducing attention mechanisms, and optimizing network structure in the absence of salient features.

[0008] The closest prior art to this invention is the invention patent with application number CN202211461433.8, entitled "A Method and System for Infrared Small Target Detection Based on Context Information". The technical solution of this patent mainly includes: extracting dynamic context information to establish long-distance dependencies between various information elements of the features; flattening the input deep features into blocks into sequences and introducing positional information, then feeding them into a multi-head attention mechanism for weighted summation, and using residual connections to optimize the results and accelerate convergence; during feature fusion, using a multi-information fusion layer to aggregate channel and spatial information from multiple features, with the multi-information fusion layer performing information fusion operations at each feature scale.

[0009] The aforementioned existing technologies have the following shortcomings:

[0010] First, this scheme relies heavily on a large number of serial and parallel convolutions for feature extraction in the backbone network. Pure convolutional structures are limited by local receptive fields and struggle to capture the global feature dependencies of small infrared targets in different spatial directions. For small infrared targets exhibiting anisotropic speckle morphology (such as targets stretched along the flight direction due to thermal diffusion and atmospheric turbulence), pure convolutional operations cannot adaptively adjust the feature extraction strategy according to the target's directional characteristics, resulting in insufficient target feature representation capabilities.

[0011] Second, this scheme mainly relies on max pooling and average pooling during network downsampling to retain key information in the channel layers. Pooling operations average and discard key details in the feature map, causing the weak signal of small infrared targets to be further attenuated or even completely disappeared in the deep feature map. In addition, max pooling and average pooling can only perform simple statistical aggregation and do not have the ability to purify the spatial structure of the feature map, thus failing to effectively suppress isolated noise and background clutter interference similar in size to the target.

[0012] Therefore, there is an urgent need for a detection method that can adaptively enhance the directional features of small infrared targets while effectively purifying spatial structure and suppressing background noise. Summary of the Invention

[0013] To address the shortcomings of existing technologies, this invention provides an infrared small target detection method and apparatus based on a pyramid morphology network. It aims to solve the problems in existing infrared small target detection methods, such as the inability of pure convolutional structures to adapt to target orientation characteristics, resulting in insufficient feature representation, and the lack of spatial structure purification capability in conventional pooling operations, leading to signal attenuation and background interference of small targets.

[0014] According to an embodiment of the present invention, an infrared small target detection method based on a pyramid morphological network includes:

[0015] Acquire the infrared image of the object to be detected;

[0016] The infrared image is input into a pre-trained feature extraction network to obtain the target detection result;

[0017] The feature extraction network includes a backbone network, a learnable directional interactive attention module, and a spatial pyramid pooling morphology module.

[0018] The backbone network is used to extract an initial feature map from the infrared image;

[0019] The learnable directional interactive attention module is used to perform multi-directional learnable rotation and energy attention weighting on the initial feature map, fuse features from different directional branches, and output an enhanced feature map.

[0020] The spatial pyramid pooling morphology module is used to perform multi-scale spatial pyramid pooling on the enhanced feature map to obtain aggregated features, and then perform morphological operations on the aggregated features to achieve spatial structure purification, outputting a purified feature map, which is used to generate the target detection result.

[0021] Preferably, the learnable direction interactive attention module performs the following operations on the input initial feature map, wherein the initial feature map is: C, H, and W represent the number of channels, height, and width, respectively.

[0022] The initial feature maps are input into the first branch, the second branch, and the third branch, respectively.

[0023] In the first branch, the initial feature map is rotated counterclockwise along the height axis by a first learnable angle to obtain a shape as follows. The first rotated feature map is used to swap the height and width dimensions; energy-driven spatial attention is applied to the first rotated feature map to obtain the first attention weight; the first attention weight is multiplied element-wise with the first rotated feature map, and then the first learnable angle is rotated clockwise along the width axis to obtain the shape restored to... The first enhanced feature map;

[0024] In the second branch, the initial feature map is rotated counterclockwise along the width axis by a second learnable angle to obtain a shape as follows. The second rotated feature map is used to swap the height and width dimensions. Energy-driven spatial attention is applied to the second rotated feature map to obtain a second attention weight. This second attention weight is then multiplied element-wise with the second rotated feature map, and the shape is rotated clockwise along the height axis by the second learnable angle to restore the shape. The second enhanced feature map;

[0025] In the third branch, the energy-driven spatial attention is directly applied to the initial feature map to obtain the third attention weight, which is then multiplied element-wise with the initial feature map to obtain the third enhanced feature map. ;

[0026] The first enhanced feature map, the second enhanced feature map, and the third enhanced feature map are fused to obtain the enhanced feature map. .

[0027] Preferably, the first learnable angle and the second learnable angle are defined by arctangent parameterization: Where θ is the first learnable angle or the second learnable angle. and These are learnable parameters to ensure that the rotation angle is bounded within the range of (-90°, 90°) and is automatically optimized through backpropagation.

[0028] Preferably, energy-driven spatial attention is applied in the following manner:

[0029] Calculate the spatial mean μ of the feature map to be processed:

[0030] Calculate the deviation of each pixel value x in the feature map to be processed from the spatial mean μ, and calculate the spatial variance accordingly. ;

[0031] For each pixel, calculate its normalized energy. ;

[0032] The normalized energy is activated using Sigmoid to obtain the attention weight map;

[0033] The attention weight map is multiplied element-wise with the feature map to be processed.

[0034] Preferably, the multi-scale spatial pyramid pooling of the spatial pyramid pooling morphology module adopts a fast spatial pyramid pooling structure, and performs the following operations sequentially on the input enhanced feature map:

[0035] Channel alignment is achieved through 1×1 convolution;

[0036] The channel-aligned feature map is subjected to multiple max pooling operations, with the output of the previous pooling operation used as the input for each pooling operation.

[0037] All feature maps before pooling are concatenated with the feature maps after each pooling in the channel dimension;

[0038] The spliced ​​feature map is convolved with 1×1 to obtain the aggregated feature.

[0039] Preferably, the morphological operation is implemented through a morphological enhancement unit, which includes four parallel morphological branches to process the aggregated features; the four morphological branches are a dilation branch, an erosion branch, an opening operation branch, and a closing operation branch.

[0040] The dilation branch is used to perform a dilation operation on the aggregated feature;

[0041] The corrosion branch is used to perform corrosion operations on the polymerization feature;

[0042] The opening operation branch is used to perform an erosion operation on the aggregated features first, and then a dilation operation;

[0043] The closing operation branch is used to perform a dilation operation on the aggregated feature first, and then an erosion operation;

[0044] The outputs of the four morphological branches are respectively processed by 1×1 convolution to adjust the channels, then concatenated in the channel dimension, and then fused by 1×1 convolution. The fused features are multiplied by the learnable gain parameter β and added to the aggregated features to obtain the output feature map of the spatial pyramid pooling morphological module.

[0045] Preferably, the expansion operation is implemented through a max pooling layer; the erosion operation includes max pooling the aggregation feature and taking the negative of the pooling result to approximate the minimum value of the sliding window.

[0046] Preferably, the feature extraction network uses YOLOv8n as the base network, the backbone network is the YOLOv8n backbone network, the learnable direction interactive attention module is added to the backbone network, and the spatial pyramid pooling morphology module is used to replace the original spatial pyramid pooling module of YOLOv8n.

[0047] On the other hand, according to embodiments of the present invention, an infrared small target detection device based on a pyramid morphological network is also provided, comprising:

[0048] The feature extraction module is used to extract features from the infrared image to be detected and output the target detection result; the feature extraction module includes a backbone network, a learnable directional interactive attention unit, and a spatial pyramid pooling morphological unit;

[0049] The backbone network is used to extract an initial feature map from the infrared image;

[0050] The learnable directional interactive attention unit is used to perform multi-directional learnable rotation and energy attention weighting on the initial feature map, fuse features from different directional branches, and output an enhanced feature map.

[0051] The spatial pyramid pooling morphological unit includes cascaded multi-scale spatial pyramid pooling sub-units and morphological enhancement sub-units; the multi-scale spatial pyramid pooling sub-units are used to perform multi-scale pooling aggregation on the enhanced feature map and output aggregated features; the morphological enhancement sub-units are used to perform morphological operations on the aggregated features and output purified feature maps.

[0052] The target detection module is used to generate target detection results based on the purified feature map.

[0053] Preferably, the learnable directional interactive attention unit specifically includes:

[0054] The first directional processing subunit includes a cascaded first learnable rotation layer, a first energy attention layer, and a first inverse rotation layer. The first learnable rotation layer is used to rotate the initial feature map counterclockwise by a first learnable angle along the height axis and exchange the height and width dimensions. The first energy attention layer is used to apply energy-driven spatial attention to the rotated feature map. The first inverse rotation layer is used to rotate the weighted feature map clockwise by the first learnable angle along the width axis and restore the original dimensional shape to obtain the first enhanced feature.

[0055] The second directional processing subunit includes a cascaded second learnable rotation layer, a second energy attention layer, and a second inverse rotation layer. The second learnable rotation layer is used to rotate the initial feature map counterclockwise along the width axis by a second learnable angle and swap the height and width dimensions. The second energy attention layer is used to apply energy-driven spatial attention to the rotated feature map. The second inverse rotation layer is used to rotate the weighted feature map clockwise along the height axis by the second learnable angle and restore the original dimensional shape to obtain the second enhanced feature.

[0056] The original retained sub-units are used to directly perform energy attention weighting on the initial feature map to obtain the third enhanced feature;

[0057] The fusion subunit is used to add and fuse the first enhancement feature, the second enhancement feature, and the third enhancement feature to obtain the enhancement feature map.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] This invention addresses the problem in existing technologies where pure convolutional structures are limited by local receptive fields and cannot adaptively adjust feature extraction strategies based on the directional characteristics of anisotropic diffuse spots on small infrared targets by introducing a learnable directional interactive attention module. This module, through the synergistic effect of learnable directional rotation and energy attention mechanisms, enables small infrared targets to achieve significant feature enhancement in different spatial directions; simultaneously, it achieves targeted suppression of non-target backgrounds through a three-branch fusion strategy.

[0060] This invention also addresses the problems of conventional pooling operations lacking spatial structure purification capabilities, easily leading to weak signal attenuation of small targets, and being susceptible to background noise interference by introducing a spatial pyramid pooling morphology module. This module employs a cascaded architecture of SPPF followed by morphological enhancement. SPPF amplifies the receptive field to capture multi-scale context, while the morphological enhancement unit purifies the feature map's spatial structure through parallel branches of four operations: dilation, erosion, opening, and closing, effectively compensating for the loss of detailed information in pure pooling operations.

[0061] This invention uses a learnable directional interactive attention module in conjunction with a spatial pyramid pooling morphology module to form a complementary process of purification followed by enhancement: the spatial pyramid pooling morphology module first removes small noise and connects broken targets through morphological operations, providing clean feature input for the learnable directional interactive attention module; on this basis, the learnable directional interactive attention module aligns the purified target features to the optimal direction through learnable directional rotation, and the energy attention mechanism further amplifies the difference between the target and the residual background. Attached Figure Description

[0062] Figure 1The network structure diagram of PMNet, the pyramid morphological network provided by this invention;

[0063] Figure 2 The operation flowchart of the Learnable Direction Interactive Attention Module (LIDA) provided by this invention;

[0064] Figure 3 The module structure diagram of the Spatial Pyramid Pooling Morphology Module (SPPM) provided by this invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be noted that the following embodiments are only for explaining the present invention and do not constitute a limitation thereof.

[0066] Key term definitions

[0067] LIDA: Learnable Directional Interaction Attention, is a learnable attention mechanism proposed in this invention for enhancing the directional features of small infrared targets.

[0068] SPPM: Spatial Pyramid Pooling Morphology, is a cascaded module proposed in this invention for purifying the spatial structure of small infrared targets.

[0069] SPPF: Spatial Pyramid Pooling Fast, is the original multi-scale pooling module in YOLOv8n.

[0070] PMNet: Pyramid-Morphology Network, is the infrared small target detection network constructed in this invention based on the YOLOv8n benchmark network.

[0071] Example 1

[0072] This embodiment constructs a PMNet network based on the improved YOLOv8n benchmark network. By introducing the Learnable Directional Interactive Attention Module (LIDA) and the Spatial Pyramid Pooling Morphology Module (SPPM) into the feature extraction network, it addresses the problems in existing technologies where pure convolutional structures are limited by local receptive fields and cannot adapt to target orientation characteristics, and conventional pooling operations lack spatial structure purification capabilities.

[0073] like Figure 1As shown, the pyramid morphology network PMNet is divided into three main parts: the backbone, the neck, and the detection head. The network input is infrared images from the infrared small target dataset, and the output is the infrared small target detection result image. This paper's network is based on YOLOv8n with two additional modules. LIDA is added to the backbone network for feature enhancement. Through parameterized orientation transformation matrices and adaptive orientation gating networks, it overcomes the limitations of traditional fixed-orientation rotation attention, achieving end-to-end learning of the optimal feature interaction direction, enhancing the model's ability to extract important features at multiple scales, and suppressing irrelevant background noise. The designed SPPM module replaces the traditional SPPF module, retaining SPPF's fast feature extraction capability while embedding differentiable morphology into a deep network. This is equivalent to introducing explicit regularization based on brightness topological priors for infrared small targets, enabling the network to maintain spatial sensitivity to target structures even under extremely low signal-to-noise ratio conditions.

[0074] The Pyramid Morphology Network PMNet is built upon the YOLOv8n benchmark network. YOLOv8n is a classic lightweight object detection network that strikes a good balance between detection speed and accuracy. It consists of a backbone network, a neck network, and a detection head. In general object detection tasks, YOLOv8n efficiently completes the task through a process of feature extraction via the backbone network, multi-scale feature fusion via the neck network, and detection head outputting the detection results. However, in infrared small target detection scenarios, where targets occupy only a few pixels and lack significant features, YOLOv8n's original pure convolutional structure and pooling operations are insufficient to effectively address this challenge, necessitating the introduction of new modules for targeted improvements.

[0075] The network structure is as follows:

[0076] The backbone network consists of a first 3×3 convolutional layer, a second 3×3 convolutional layer, a first C2f module, a third 3×3 convolutional layer, a LIDA module, a second C2f module, a fourth 3×3 convolutional layer, a third C2f module, a fifth 3×3 convolutional layer, a fourth C2f module, and a SPPM module, all connected sequentially. The LIDA module, positioned before the second C2f module, enhances the initial feature maps extracted from the shallow layers, strengthening the target response early in feature propagation. The SPPM module, located at the end of the backbone network, replaces the original SPPF module in YOLOv8n and performs multi-scale context aggregation and spatial structure refinement, outputting refined feature maps to the neck network. The first, second, third, and fourth C2f modules in the backbone network all have lateral connections, passing their respective output feature maps to the neck network, providing different levels of features for multi-scale feature fusion.

[0077] The neck network receives feature maps output by four C2f modules of the backbone network and performs multi-scale feature fusion through the four C2f modules and four concatenation operations. The specific connection relationships are as follows: The output of the first C2f module of the backbone is processed by the first C2f module of the neck and then enters the first splicing operation. The output of this splicing is divided into two paths: one path is passed to the first detection head module of the detection head, and the other path is convolved by 3×3 and then enters the second splicing operation. The output of the second C2f module of the backbone is processed by the second C2f module of the neck and then enters the second splicing operation. This splicing operation also receives the output of the first splicing after 3×3 convolution. The splicing result is upsampled and then enters the third splicing operation. The output of the third C2f module of the backbone is processed by the third C2f module of the neck and then enters the third splicing operation. This splicing operation also receives the output of the second splicing after upsampling. The splicing result is divided into two paths: one path is passed to the second detection head module of the detection head, and the other path is convolved by 3×3 and then enters the fourth splicing operation. The output of the fourth C2f module of the backbone is processed by the fourth C2f module of the neck and then enters the fourth splicing operation. This splicing operation also receives the output of the third splicing after 3×3 convolution. The splicing result is passed to the third detection head module of the detection head. The neck network, through the aforementioned feature fusion path, allows for full interaction of features from different levels of the backbone network, enabling the high-resolution detail features in the shallow layers and the semantic features in the deep layers to complement each other.

[0078] The detection head consists of three parallel detection head modules, which respectively receive the outputs from the first, third, and fourth stitches of the neck. The three detection head modules are responsible for detecting targets of different scales, and the final results are converged to form an infrared small target detection result image.

[0079] An infrared small target detection method based on a pyramid morphological network proposed according to the present invention includes:

[0080] Acquire the infrared image of the object to be detected;

[0081] The infrared image is input into a pre-trained feature extraction network to obtain the target detection result;

[0082] The feature extraction network includes a backbone network, a learnable directional interactive attention module, and a spatial pyramid pooling morphology module.

[0083] The backbone network is used to extract an initial feature map from the infrared image;

[0084] The learnable directional interactive attention module is used to perform multi-directional learnable rotation and energy attention weighting on the initial feature map, fuse features from different directional branches, and output an enhanced feature map.

[0085] The spatial pyramid pooling morphology module is used to perform multi-scale spatial pyramid pooling on the enhanced feature map to obtain aggregated features, and then perform morphological operations on the aggregated features to achieve spatial structure purification, outputting a purified feature map, which is used to generate the target detection result.

[0086] In this embodiment, the SPPM module adopts a cascaded sequence of fast spatial pyramid pooling followed by morphological enhancement. Specifically, SPPF first performs multi-scale receptive field expansion and context aggregation, and then the morphological enhancement unit refines the aggregated features to improve their spatial structure. This order is not arbitrary but based on a deep analysis of information flow: expanding the receptive field first allows morphological operations to judge structural features within a larger spatial range, thus more accurately distinguishing between real targets and background noise.

[0087] In infrared small target detection, due to thermal diffusion and atmospheric turbulence, the target exhibits anisotropic diffuse speckle morphology after imaging (stretched along the flight direction and relatively compact in the vertical direction). Traditional convolutional neural networks with fixed square kernels (such as 3×3 convolutions) lack specificity for this anisotropic feature; the shape and orientation of the kernel remain unchanged regardless of the target's orientation, resulting in low feature extraction efficiency. Some existing improved methods employ a fixed 90° rotation to enhance orientation awareness, but this fixed rotation can only capture orthogonal directional features and cannot adapt to the diffuse speckle morphology of infrared targets in arbitrary directions.

[0088] To address the aforementioned issues, this embodiment proposes a learnable direction-interaction attention mechanism based on the learnable direction-interaction attention module LIDA. By using a parameterized direction transformation matrix and an adaptive direction-gated network, it overcomes the limitations of traditional fixed-direction rotation attention, achieving end-to-end learning of the optimal feature interaction direction. This embodiment introduces a trainable direction transformation matrix. ,in Let d represent the learning direction and C represent the number of channels. By applying a linear transformation to the rotated feature map, the interaction between channels is modeled. Each direction is learned independently, avoiding parameter coupling, thus achieving adaptive modeling of features in different directions.

[0089] like Figure 2 As shown, let the initial feature map shape of the input learnable direction interactive attention module be... Where C, H, and W represent the number of channels, height, and width, respectively.

[0090] The LIDA module contains three parallel branches, each processing the initial feature map. The use of a three-branch design, rather than a single or multiple branches, is a trade-off based on the energy distribution characteristics of small infrared targets: too few branches cannot effectively cover the orientation space, while too many branches introduce redundant computation and increase the risk of overfitting.

[0091] The design rationale for setting up three branches is as follows: Due to thermal diffusion and atmospheric turbulence, infrared aircraft targets exhibit a diffuse pattern with a bright center and gradually darkening edges after imaging, and their energy distribution is anisotropic. Rotation in a single direction can only enhance features in a specific direction. Rotation along the altitude axis (H-axis) enhances the energy distribution in the vertical direction (e.g., climbing or diving targets), while rotation along the width axis (W-axis) enhances the energy distribution in the horizontal direction (e.g., level-flying targets). The original branch retains the unrotated baseline features to prevent feature distortion caused by over-rotation. If only the rotating branch is set without retaining the original branch, when the target's energy distribution is close to isotropic, the rotation operation may actually destroy the original feature representation and reduce detection accuracy. The three branches capture features from three orthogonal subspaces—vertical, horizontal, and baseline—and achieve complete coverage of the directional space through fusion.

[0092] First branch (height axis rotation branch):

[0093] First, the initial feature map Rotate counterclockwise along the height axis to the first learnable angle The resulting shape is The first rotated feature map simultaneously achieves the exchange of height and width dimensions.

[0094] It is important to note that this embodiment employs learnable rotation instead of fixed-direction or fixed-angle rotation because these two existing methods each have fundamental limitations: fixed-direction convolution kernels (such as standard 3×3 convolutions) have a cross-shaped or square receptive field, limiting their ability to extract features from small infrared targets that are not axially distributed. For example, for a UAV target flying at a 45° angle, neither horizontal nor vertical convolution kernels can effectively capture the complete shape of its diffuse speckle; a fixed 90° rotation can only capture orthogonal features and cannot adapt to the diffuse speckle shape of infrared targets in any direction. Learnable angle parameters By automatically optimizing through backpropagation, the network can adapt to the actual directional distribution of targets in the image, fundamentally solving the problem that fixed-direction operations cannot adapt to diverse target orientations.

[0095] Furthermore, to avoid gradient vanishing, learnable angles Defined via arctangent parameterization:

[0096]

[0097] in, From the first learnable perspective Or the second learnable angle , and For learnable parameters, This ensures that the rotation angle is bounded within the range of (-90°, 90°) and is automatically optimized through backpropagation. In deep learning, directly learning unconstrained angle parameters can lead to optimization difficulties, as the angle values ​​may diverge to arbitrarily large values, causing gradient explosion or rendering the rotation operation meaningless. The arctangent function naturally limits the output to the range of (-90°, 90°), ensuring both the differentiability of the rotation operation and the stability of the training process. In the initial stage of training, the arctangent function can be used to ensure that the rotation angle is bounded within the range of (-90°, 90°) and automatically optimized through backpropagation. Initialize to 1, Initialize it to 0.01, so that the initial value of the rotation angle is close to 0°. That is, in the initial state, the network tends not to rotate. During the training process, the optimal rotation angle is gradually learned according to the data distribution, so that it can remain stable in the early stage of training.

[0098] The spatial transformation of a rotation operation is defined as follows:

[0099]

[0100] in, Represents the spatial coordinates after rotation. The center of rotation is usually taken as the geometric center of the feature map; The rotation angle is learnable, in radians. This transformation acts spatially on the input feature map while preserving its channel dimension and spatial resolution.

[0101] After completing the orientation rotation, this embodiment does not directly output the rotated feature map. Instead, it introduces an energy-driven spatial attention mechanism for further processing. The rotation operation itself can only change the directional distribution of features and cannot distinguish between the target and the background: background clutter (such as cloud edges or sea ripples) may also exhibit a strong response in a specific direction. If only rotation is performed without attention, although the rotated feature map aligns the target's energy distribution to a specific direction, strong response regions in the background are also preserved or even enhanced in that direction, leading to an increased risk of false alarms. If only attention is performed without rotation, since standard attention mechanisms are insensitive to direction, they may simultaneously amplify both the target and background noise of similar brightness. Using both in combination—the rotation operation aligns the anisotropic energy distribution of the target to a specific direction, and then applying energy attention in that direction—maximizes the difference in energy dimension between the target and the background.

[0102] Energy-driven spatial attention differs from common self-attention mechanisms (such as multi-head attention in Transformers) in that it does not rely on similarity calculations between locations within the feature map. Instead, it generates attention weights based on the energy value of each pixel relative to global statistical characteristics. The advantage of this design is its high computational efficiency and its suitability for the physical characteristics of small infrared targets: small targets appear as local high-energy regions in images, with their pixel values ​​deviating significantly from the background mean.

[0103] The computational steps for energy-driven spatial attention are as follows:

[0104] (1) Mean calculation, used to calculate the background response level of the entire spatial plane, that is, to calculate the average brightness or activation value of all pixels on the feature map:

[0105]

[0106] in, This represents the pixel value at position (i, j) in the feature map to be processed, with the mean value being... It reflects the baseline response level of the entire feature map and serves as a reference standard for subsequent judgments on whether each pixel is prominent.

[0107] (2) Centered square, which measures the degree to which each pixel deviates from the average:

[0108]

[0109] The centering operation removes the overall brightness shift of the feature map, allowing the energy metric to focus on the relative salience of pixels rather than their absolute numerical value. This is particularly important for infrared images, as the overall brightness of infrared images can drift with factors such as ambient temperature and detector gain.

[0110] (3) Variance calculation: Calculate the average deviation of the entire spatial plane as the normalization benchmark.

[0111]

[0112] It should be noted that variance It depicts the overall fluctuation level of the feature map. In a scene with a uniform background. Smaller, in complex background scenes The deviation is relatively large. Normalizing by dividing the deviation by the variance can maintain the comparability of attention weights in scenarios with different complexities.

[0113] (4) Energy function: calculate the normalized energy of each pixel. The molecule represents the pixel. The degree of deviation of the fraction itself, with the denominator representing the global fluctuation level:

[0114]

[0115] Where λ is a constant, and can be taken as λ=0.0001 to prevent the denominator from being zero.

[0116] The energy function effectively measures pixel saliency by normalizing the deviation of each pixel from the global mean by dividing by the variance. Small infrared targets appear as locally bright areas in the image, with pixel values ​​deviating significantly from the background mean, resulting in higher energy values. While background areas may exhibit local brightness variations, their normalized energy remains relatively stable.

[0117] (5) Apply Sigmoid activation to the normalized energy to obtain the attention weight map, and map the energy values ​​to the (0, 1) interval:

[0118]

[0119] The smooth, non-linear characteristics of the Sigmoid function enable attention weights to have good discriminative power when the energy value is moderate, and tend to saturate when the energy value is extreme, thus preventing a few extreme pixel values ​​from dominating the entire attention map.

[0120] (6) Multiply the attention weight map A and the feature map F to be processed element by element to obtain the weighted feature map:

[0121]

[0122] This embodiment selectively enhances the feature map: high-energy regions (target regions) receive greater weight, while low-energy regions (background regions) receive less weight. Through this method, target features are enhanced, and background noise is suppressed.

[0123] After multiplying the first attention weight element-wise with the first rotated feature map, rotate the first learnable angle clockwise along the width axis. The feature map shape is restored to C×H×W to obtain the first enhanced feature map. The clockwise rotation angle is the same as the counterclockwise rotation angle to ensure that the spatial dimensions of the recovered feature map are consistent with the input, thus guaranteeing strict spatial alignment of the outputs of each branch during subsequent three-branch fusion. If the recovered rotation angle does not match the forward rotation angle, the feature maps output by the three branches will have spatial misalignment, introducing artifacts during the summation and fusion process.

[0124] Second branch (width axis rotation branch):

[0125] The processing flow for the second branch is similar to that of the first branch. The initial feature map is rotated counterclockwise along the width axis by a second learnable angle. The second rotational feature map with shape H×W×C is obtained. This allows for the exchange of height and width dimensions. Similarly, it is defined using arctangent parameterization: ,in and These are learnable parameters and can be initialized to values ​​that bring the rotation angle close to 0°. Energy-driven spatial attention is applied to the second rotation feature map to obtain second attention weights. These second attention weights are then multiplied element-wise with the second rotation feature map, and the map is rotated clockwise along the height axis by a second learnable angle. The feature map shape is restored to C×H×W to obtain the second enhanced feature map. .

[0126] The purpose of setting up a second branch is to complement the first branch. Setting only the H-axis rotation branch only enhances the vertical features and is insufficient for horizontally flying targets; setting only the W-axis rotation branch has the opposite effect. By rotating in two orthogonal directions, the height and width axes, combined with learnable rotation angles, the network can cover any direction in the two-dimensional plane: according to the rotation theory of two-dimensional space, rotation in any direction can be decomposed into a combination of rotations in two orthogonal directions, thus achieving a complete modeling of the anisotropic energy distribution of the target.

[0127] Third branch (original retained branch):

[0128] The third branch does not perform rotation operations on the initial feature map. Instead, it directly applies energy-driven spatial attention to obtain the third attention weights, and multiplies them element-wise with the initial feature map to obtain the third enhanced feature map. .

[0129] The purpose of setting up a third branch is to preserve the unrotated baseline features. In real-world infrared scenarios, the energy distribution of some targets may be nearly isotropic (such as targets directly facing the detector). In such cases, rotation not only fails to enhance features but may also introduce unnecessary transformation errors, inevitably leading to a certain degree of feature smoothing. The original preservation branch provides the network with an option to avoid directional transformation, ensuring that even if the rotation branch fails to effectively enhance features, the network can still retain its basic detection capabilities through the original branch. Simultaneously, the output of the original branch also serves as a reference for the rotation branch, playing a stabilizing and calibration role during fusion. When the output of the rotation branch differs significantly from that of the original branch, summing and fusing them can achieve smoothing and regularization effects.

[0130] The first, second, and third enhanced feature maps are fused element-wise to obtain the enhanced feature map output by the LIDA module. This element-wise addition method, rather than channel concatenation, achieves feature fusion while maintaining the same number of channels. Channel concatenation leads to a threefold increase in the number of channels, increasing the computational burden on subsequent networks; while addition allows the features from the three branches to be superimposed in the same feature space. The network automatically learns the relative contribution weights of each branch through backpropagation, enhancing the compactness and consistency of the feature representation.

[0131] In this embodiment, the LIDA module aligns the anisotropic energy distribution of the target to a specific direction through rotation operations. Energy attention amplifies the energy difference between the target and the background in this direction, and the fusion strategy achieves three-branch complementarity, suppressing misjudgment in a single direction. The synergistic effect of the two enables infrared small targets to achieve significant feature enhancement in different spatial directions, while simultaneously achieving directional suppression of non-target backgrounds. This effectively overcomes the technical problem of pure convolutional structures being limited by local receptive fields and unable to adapt to target directional characteristics.

[0132] Furthermore, in infrared small target detection, besides the problem of insufficient directional feature representation, there is another equally critical issue: how to preserve the spatial structure information of the target under extremely low signal-to-noise ratio conditions. While pooling operations in existing detection networks (such as max pooling and average pooling) can effectively expand the receptive field and reduce computational cost, the pooling process is essentially a coarse-grained aggregation of information. For example, max pooling only retains the maximum value within the window, while average pooling averages all values ​​within the window. For targets of normal size, this information loss is within acceptable limits; however, for infrared small targets of only a few pixels, pooling operations may completely drown out the target signal in background noise.

[0133] Therefore, this embodiment addresses the aforementioned problems by employing a cascaded architecture of multi-scale expansion and spatial structure refinement to achieve a progressive feature evolution from pixel-level local features to global semantic associations. The Spatial Pyramid Pooling Morphology Module (SPPM) adds a morphological enhancement unit to the Fast Spatial Pyramid Pooling Module (SPPF), refining the spatial structure of features while maintaining multi-scale aggregation capabilities. Morphological operations perform spatial filtering based on brightness topology rather than statistical properties. Through dilation to expand high-response regions, erosion to suppress isolated noise, and opening / closing operations to repair the target structure, it effectively compensates for the loss of detailed information in pure pooling operations.

[0134] In this embodiment, the cascading order of the spatial pyramid pooling morphology modules is: first, fast spatial pyramid pooling SPPF, then morphology.

[0135] The Spatial Pyramid Pooling Morphology Module first performs context aggregation within a parallel multi-scale receptive field using the Fast Spatial Pyramid Pooling (SPPF) branch, capturing target and background information at different granularities. SPPF is used as the first stage because multi-scale receptive field expansion allows the network to observe the target simultaneously at multiple scales, avoiding incomplete information capture due to a single-scale receptive field being too small or too large. If the receptive field is too small, the network can only see local fragments of the target, unable to determine whether the fragment is part of a real target or a bright background spot; if the receptive field is too large, the target signal may be diluted by a large area of ​​background. Multi-scale parallel observation can simultaneously preserve local details and global context. Based on this, morphological refinement is then performed. Morphological operations can determine which are real target structures and which are background noise that needs to be suppressed at a larger receptive field scale, with significantly higher accuracy than operations performed at a single scale.

[0136] like Figure 3 As shown, let the enhanced feature map of the input SPPM module be... ,in, The number of channels in the input feature map.

[0137] Channel alignment of the input feature map is performed using 1×1 convolutions to adjust the number of channels to a uniform dimension. ,get ;

[0138] by Based on this, perform three consecutive max pooling operations, with the output of the previous pooling operation used as input for each pooling operation:

[0139]

[0140]

[0141]

[0142] Next, all feature maps before pooling are... Feature maps after each pooling , , Perform splicing along the channel dimension: .

[0143] It should be noted that the features before pooling Preserving the finest spatial detail is crucial for small infrared targets with only a few pixels. After three consecutive max-pooling operations, the feature map at the original resolution... The sensory field has expanded further, and detailed information about small targets may have become severely blurred. Incorporating concatenation provides a shortcut for the network, allowing fine-grained information to bypass pooling operations and be directly passed to subsequent layers. This effectively alleviates the problem of information loss in small targets. If only pooled features are retained, small targets may completely disappear after multiple pooling operations.

[0144] Finally, a 1×1 convolution is performed on the stitched feature map to eliminate channel redundancy between features of different scales, resulting in aggregated features. : .

[0145] While Fast Spatial Pyramid Pooling (SPPF) can efficiently aggregate multi-scale contextual information, max pooling is sensitive to noise. In infrared images, bright spots caused by cloud edges, sunlight reflection, and equipment thermal noise exhibit localized high response. These bright spots may be retained or even amplified after max pooling, forming false targets that are difficult to distinguish from real targets. This is one of the main reasons for the high false alarm rate of existing detection methods.

[0146] To address this issue, this embodiment introduces a learnable morphological enhancement unit after SPPF. Through dilation and erosion operations, it explicitly enhances target edges, suppresses background noise, and separates adhered regions, providing a clean feature map with a clear topological structure for subsequent detection. The morphological operations perform spatial filtering based on brightness topology rather than statistical properties, enabling the differentiation between real targets and noise at the topological level: real targets appear as bright regions with some connectivity in space, while noise is typically isolated and discontinuous. The dilation operation connects target regions broken due to grayscale depressions, while the erosion operation eliminates isolated bright noise spots. This topology-based filtering method effectively compensates for the shortcomings of SPPF, which relies solely on statistical aggregation.

[0147] The morphological enhancement unit is based on four classic morphological operations: dilation (D), erosion (E), opening (O), and closing (C), constructing a multi-branch parallel processing flow. Dilation involves maximizing the value within a sliding window, while erosion involves minimizing it. These four operations, rather than just one or two, are chosen because they cover the complete requirements for spatial structure refinement: dilation expands the target region, erosion shrinks the target region and suppresses noise, opening removes noise while maintaining the target's position, and closing fills holes while preserving the overall shape of the target. These four operations complement each other, comprehensively addressing various spatial structure degradation problems encountered in infrared small target detection, including target fragmentation, edge blurring, noise attachment, and internal holes.

[0148] Specifically, given the input aggregate features The morphological enhancement unit divides the feature map into four parallel paths along the channel dimension. Each branch operation maintains the feature map spatial size unchanged and applies the following operations respectively:

[0149] (1) Expansion branch: Due to thermal diffusion and the system's point spread function, infrared targets often exhibit a diffuse spot with a bright center and gradually darkening edges after imaging, and may show gray-scale depressions or even breaks within the target. Dilatation branching expands the high-response region by taking local maxima, thus reconnecting the broken target areas caused by gray-scale depressions and restoring the target's complete outline, effectively solving the problem of missed detections caused by target breaks.

[0150] (2) Corrosion branches: In the infrared background, cloud edges, sunlight reflection, and equipment thermal noise can produce bright spots that are similar in size or even shape to the target. These bright spots are the main cause of false alarms. Erosion operations, by taking local minima, have a strong suppressive effect on isolated bright noise: real targets typically appear as continuous and stable local maxima regions with relatively high overall brightness in their neighborhoods; after the erosion operation takes the minimum value, a certain response is still maintained. In contrast, noisy bright spots are randomly occurring isolated points with generally low pixel values ​​in their neighborhoods; after the erosion operation takes the minimum value, the noise response is significantly reduced. Through this mechanism, the erosion operation can suppress noise while effectively preserving the real target.

[0151] (3) Opening operation branches: The opening operation, which combines erosion and dilation, achieves a balance between noise reduction and shape preservation. The target's center position remains almost unchanged, while edge burrs and parasitic bright spots are removed. In strong clutter backgrounds (such as sea surface ripples), the bright spots produced by the ripples are similar in size to the target but have different shapes (ripples are linear or striped, while targets are dotted). The opening operation can effectively distinguish target points from ripple bright spots by utilizing morphological differences, extracting a clean target skeleton. While erosion alone can suppress noise, it can also shrink the target area, causing the target size to decrease or even disappear. Similarly, dilation alone can enlarge the target area but amplify noise.

[0152] (4) Closing operation branch: The process involves expansion followed by erosion. The closing operation primarily focuses on conformal filling, first expanding to connect fractures and fill voids, then eroding to shrink the target back to approximately its original size. When a target breaks into multiple bright patches due to atmospheric disturbance, the closing operation first connects the broken parts into a whole through expansion, then erodes to shrink the whole back to the target's proper size. The closing operation branch has a stabilizing effect on transient fractures of distant targets caused by turbulent scintillation, effectively reducing missed detections due to target fractures.

[0153] After extracting complementary edge and region information through four parallel branches, the outputs of each branch are individually channel-adjusted using 1×1 convolutions, and then concatenated along the channel dimension. The reason for setting independent 1×1 convolutions is that the features extracted by different morphological branches have different distribution characteristics: dilation favors high-response expansion, erosion favors low-response contraction, and opening / closing operations favor mid-frequency structures. Through independent channel adjustment, the network can learn the optimal channel weight allocation for each branch, avoiding conflicts between different distribution features caused by direct concatenation.

[0154] After concatenation, the features are fused using a 1×1 convolution. The fused features are then multiplied by a learnable gain parameter β and then combined with the aggregated features. Adding them together, we obtain the purified feature map Y output by the SPPM module:

[0155]

[0156] Among them, the learnable gain parameter It can be initialized to 0 or a small positive number. Learnable gain parameter. The applicability of morphological operations varies across different scenarios. In clean backgrounds with clearly defined targets, excessive morphological processing can lead to loss of target details; for example, dilation might cause two adjacent small targets to merge into one, and erosion might completely eliminate faint targets. In cluttered scenarios, stronger morphological filtering is needed to suppress numerous false alarms. Learnable gain parameters can be used to address this. The network can adaptively adjust the intensity of morphological enhancement, which can be understood as: when When SPPM approaches 0, its effect is close to the original SPPF, preserving more original information and suitable for clean scenes; when... When the size is large, the morphological enhancement effect is stronger, making it suitable for scenarios with severe noise. Residual connections (plus Y2) ensure that even if there are deviations in morphological processing, the original multi-scale aggregation information is not completely lost, thus guaranteeing the training stability and robustness of the network.

[0157] In this embodiment, the SPPM module adopts a cascaded order of SPPF followed by morphology. SPPF first expands the equivalent range of receptivity and captures multi-scale context. Morphology then refines the receptive field at this larger scale. At this point, the network can observe a more complete spatial structure and accurately determine that a persistent local extremum region over a large area is more likely to be a real target, while a bright spot that only stands out within a 3×3 window is more likely to be noise. This avoids premature loss of weak target details and effectively suppresses clutter.

[0158] During the model training phase, infrared images from the labeled infrared small target detection dataset are used as training samples and input into the PMNet network. The network calculates the prediction results through forward propagation. The loss function can adopt the original loss function of YOLOv8n, including classification loss and localization loss. The error between the prediction result and the true label is calculated through the loss function. The parameters of each module in the network are updated using the backpropagation algorithm, including the learnable rotation angle parameter in the LIDA module and the learnable gain parameter β in the SPPM module, until the network converges.

[0159] The learning mechanism for rotation angle in the LIDA module is as follows: In each training iteration, the network updates the value based on the gradient information of the loss function with respect to the rotation angle, using the chain rule of arctangent parameterization. Thus indirectly adjusting and This end-to-end learning approach allows the rotation angle to automatically converge to the optimal value based on the actual orientation distribution of the target in the training data.

[0160] During the inference phase, the infrared image to be detected is input into the trained PMNet network. The network sequentially extracts the initial feature map through the backbone network, performs directional feature enhancement on the initial feature map through the LIDA module to obtain the enhanced feature map, and performs multi-scale context aggregation and spatial structure purification on the enhanced feature map through the SPPM module to obtain the purified feature map. Finally, the detection head generates the target detection result based on the purified feature map and outputs the target's category and location information.

[0161] To verify the technical effectiveness of this embodiment, ablation experiments were conducted on the IRSTD-1k dataset. The IRSTD-1k dataset contains multiple infrared images taken by real-world infrared cameras, in which various types of small targets, including drones, organisms, ships, and vehicles, are captured at different distances from the thermal imaging device. The backgrounds range from oceans, rivers, fields, mountains, and cities to clouds, introducing a large amount of clutter and noise, which can fully reflect the complexity of actual infrared detection scenarios.

[0162] First, to verify the impact of the cascading order in the SPPM module on the detection of small infrared targets, an SPPM ablation experiment was conducted, and the experimental results are shown in Table 1.

[0163] Table 1: SPPM Ablation Experiment of Spatial Pyramid Pooling Morphological Module

[0164]

[0165] As shown in Table 1, the mAP50 of the cascaded method (SPPF + Morph) is 73.6%, significantly better than the 70.4% achieved by the cascaded method (Morph + SPPF), demonstrating that the cascaded order is crucial for infrared small target detection. SPPF first expands the receptive field, capturing multi-scale context, and then morphology refines the image at a larger scale, avoiding premature loss of weak target details while effectively suppressing clutter. In the reverse order, morphology operates at the original scale; due to the limited receptive field, it may misjudge diffuse pixels at the target edge as noise and filter them out, leading to a decrease in detection performance.

[0166] To verify the contribution of each module to the infrared small target detection task, YOLOv8n was used as the baseline network to evaluate the independent and synergistic effects of the SPPF, SPPM and LIDA modules respectively. The experimental results are shown in Table 2.

[0167] Table 2: Ablation experiments with different modules

[0168]

[0169] Using YOLOv8n as the baseline network (mAP50 of 71.2%), the PMNet network in this embodiment (which includes the LIDA module and the SPPM module, with SPPM using a cascaded order of SPPF followed by morphology) achieves an mAP50 of 74.0% and an accuracy of 82.1%.

[0170] After replacing SPPF with SPPM, the mAP50 improved from 71.2% to 73.6%, the precision (P) significantly improved from 77.2% to 81.9%, while the recall (R) remained basically unchanged (66.8% and 67.8%). These results demonstrate that morphological operations have a significant effect on the spatial structure purification of small infrared targets. SPPM, through parallel branches of dilation-erosion-opening / closing operations, effectively suppresses strong clutter interference such as cloud edges and sea surface glare, reducing the false detection rate and thus significantly improving precision.

[0171] After introducing the LIDA module separately, mAP50 improved from 71.2% to 72.8%, precision (P) increased to 79.2%, and recall (R) remained at 67.9%. This result demonstrates that learnable directional rotational attention can enhance the target's directional adaptability. Infrared aircraft targets exhibit anisotropic dispersion patterns due to thermal diffusion and atmospheric turbulence. LIDA, through learnable rotational transformations along the H / W axes, enhances the target's feature saliency in different spatial directions.

[0172] The cascaded design of SPPM and LIDA forms a complementary process of first refining and then enhancing: SPPM's morphological operations first remove small noise through opening operations and connect broken targets through closing operations, providing LIDA with clean feature input; LIDA then uses learnable orientation rotation to align the refined target features to the optimal orientation, and the energy attention mechanism further amplifies the difference between the target and the residual background. This synergy between spatial structure regularization and directional feature enhancement achieves optimal precision, but the conservative filtering strategy of morphology results in a slight loss in recall. When used together, SPPM first refines features through morphological operations, and LIDA then enhances directional features, achieving optimal precision.

[0173] Example 2

[0174] The difference between this embodiment and Embodiment 1 is that the cascading order of SPPF and morphological enhancement units in the SPPM module is reversed. That is, morphological enhancement is performed on the input feature map first, and then SPPF multi-scale pooling aggregation is performed on the enhanced feature map. The structure and function of the LIDA module in this embodiment are the same as in Embodiment 1, and the other parts of the network (including the backbone network and the detector head) remain unchanged.

[0175] Specifically, in the SPPM module of this embodiment, the input enhanced feature map first enters the morphological enhancement unit, where spatial structure is purified through four parallel branches: dilation, erosion, opening, and closing. Then, the purified feature map is input into the SPPF part, where multi-scale context aggregation is completed through 1×1 convolution channel alignment, triple concatenated max pooling, channel concatenation, and 1×1 convolution fusion.

[0176] The essential difference between this embodiment and Embodiment 1 lies in the order of information processing. These two orders correspond to different design philosophies: Embodiment 1 (SPPF first, then morphology) follows the idea of ​​global observation first, then local refinement, first using a large receptive field to fully understand the scene, and then focusing on the target details; this embodiment (morphology first, then SPPF) follows the idea of ​​local filtering first, then global integration, first cleaning up noise at the original resolution, and then gathering the clean information to multiple scales.

[0177] This embodiment is applicable to the following specific scenarios: when the noise in the original input features or shallow features is extremely strong and the clutter energy is higher than the target energy, performing morphological operations first can suppress large-scale background bright spots and isolated noise points in advance, avoiding the subsequent SPPF MaxPool operation from mistakenly amplifying noise clutter into multi-scale false targets. This order reversal is equivalent to adding a spatial pre-filter before expanding the receptive field, reducing the risk of false alarm coupling in subsequent multi-scale fusion. For example, in a scene with strong glare on the sea surface, a large number of bright spots generated by sunlight reflected from the sea surface are difficult to distinguish from the target within a 3×3 window. Suppressing these isolated bright spots first through morphological erosion operations, and then performing multi-scale pooling aggregation, can effectively reduce false alarms.

[0178] Experimental data (see Table 1) show that the cascaded order of morphology followed by SPPF (Morph+SPPF) has an mAP50 of 70.4%. Although the overall performance is lower than the 73.6% of Example 1, it still has application value in specific high-noise scenarios: its recall rate is relatively high (68.1%), indicating that more candidate targets are retained in extreme noise scenarios. Therefore, this embodiment should be included in the scope of protection of this invention, providing flexible options for different application scenarios.

[0179] Example 3

[0180] The difference between this embodiment and Embodiment 1 is that the number of learnable directional branches in the LIDA module or the number of morphological branches in the SPPM module can be adjusted according to the actual application scenario. Embodiment 1 provides the most complete configuration (three-branch LIDA + four-branch SPPM), which is suitable for general scenarios; in actual deployment, it can be simplified or extended according to the target characteristics, background complexity, and computational resource constraints.

[0181] In a specific variant, the number of directional branches in the LIDA module is not limited to three. When the target orientation distribution in the application scenario is relatively concentrated (e.g., only horizontally flying targets exist, commonly seen in the cruise phase of civil aircraft), only the width axis rotation branch and the original retained branch can be retained. The altitude axis rotation branch contributes very little to this scenario, and omitting it can reduce the computational overhead by about one-third. When the target orientation distribution in the application scenario is more diverse (e.g., drone swarms with targets facing different directions), more directional branches can be added (e.g., 45° direction, 135° direction) to cover a richer directional space and improve the detection capability for targets at extreme angles.

[0182] In another specific variant, the number of morphological branches in the SPPM module can be increased or decreased depending on the complexity of the detection scene. For example, in scenes with relatively simple backgrounds (such as clear sky backgrounds), only the dilation and erosion branches can be retained, omitting the opening and closing branches to reduce computational overhead. In clear sky backgrounds, there are fewer cases of target fracture and noise attachment, so the contribution of opening and closing operations is limited. In scenes with extremely complex backgrounds (such as urban heat island backgrounds), all four branches can be retained to obtain the strongest spatial structure purification capability. In urban backgrounds, building thermal radiation, air conditioner outdoor units, etc., produce a large number of bright spots similar to the target, requiring four operations to work together.

[0183] In another specific variation, besides YOLOv8n, other lightweight object detection networks with similar architectures can also be used as the basic framework. As long as the network includes a backbone network, a neck network, and a detection head, and the neck network employs a spatial pyramid pooling module, the LIDA and SPPM modules of this invention can be applied for improvement. The LIDA and SPPM modules proposed in this invention are general improvements to the feature extraction and fusion processes, independent of the specific implementation details of YOLOv8n, and possess good transferability.

[0184] All of the above variations are simple replacements or parameter adjustments based on the same inventive concept, allowing for flexible adjustments to module configurations in different scenarios. However, the core learnable direction rotation attention mechanism and the cascaded architecture of pooling followed by morphology remain unchanged and should all be understood as being included within the scope of protection of this invention.

[0185] Example 4

[0186] This embodiment provides an infrared small target detection device based on a pyramid morphological network, which is used to perform the infrared small target detection method described in Embodiment 1. This embodiment further illustrates the technical solution of the present invention from the perspective of device structure.

[0187] The apparatus in this embodiment includes a feature extraction module and a target detection module. The feature extraction module includes a backbone network, a learnable directional interactive attention unit, and a spatial pyramid pooling morphological unit. The backbone network is used to extract an initial feature map from the input infrared image. The learnable directional interactive attention unit performs multi-directional learnable rotation and energy attention weighting on the initial feature map, fusing features from different directional branches to output an enhanced feature map. The spatial pyramid pooling morphological unit includes cascaded multi-scale spatial pyramid pooling sub-units and morphological enhancement sub-units; the multi-scale spatial pyramid pooling sub-units perform multi-scale pooling aggregation on the enhanced feature map to output aggregated features; the morphological enhancement sub-units perform morphological operations on the aggregated features to output a refined feature map.

[0188] The target detection module is used to generate target detection results based on the purified feature map.

[0189] The device's workflow is as follows: after the infrared image to be detected is input, it sequentially passes through the backbone network to extract the initial feature map, the learnable directional interactive attention unit to enhance the directional features, the spatial pyramid pooling morphology unit to perform multi-scale context aggregation and spatial structure purification, and finally the target detection module outputs the detection result.

[0190] The above-described device embodiments and method embodiments one to three belong to the same inventive concept. Each unit in the device corresponds to each processing step in the method and has the same technical effect, which will not be repeated here.

[0191] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting small infrared targets based on a pyramid morphological network, characterized in that, include: Acquire the infrared image of the object to be detected; The infrared image is input into a pre-trained feature extraction network to obtain the target detection result; The feature extraction network includes a backbone network, a learnable directional interactive attention module, and a spatial pyramid pooling morphology module. The backbone network is used to extract an initial feature map from the infrared image; The learnable directional interactive attention module is used to perform multi-directional learnable rotation and energy attention weighting on the initial feature map, fuse features from different directional branches, and output an enhanced feature map. The spatial pyramid pooling morphology module is used to perform multi-scale spatial pyramid pooling on the enhanced feature map to obtain aggregated features, and then perform morphological operations on the aggregated features to achieve spatial structure purification, outputting a purified feature map, which is used to generate the target detection result.

2. The infrared small target detection method based on a pyramid morphological network according to claim 1, characterized in that, The learnable directional interactive attention module performs the following operations on the input initial feature map, wherein the initial feature map is: C, H, and W represent the number of channels, height, and width, respectively. The initial feature maps are input into the first branch, the second branch, and the third branch, respectively. In the first branch, the initial feature map is rotated counterclockwise along the height axis by a first learnable angle to obtain a shape as follows. The first rotated feature map is used to swap the height and width dimensions; energy-driven spatial attention is applied to the first rotated feature map to obtain the first attention weight; the first attention weight is multiplied element-wise with the first rotated feature map, and then the first learnable angle is rotated clockwise along the width axis to obtain the shape restored to... The first enhanced feature map; In the second branch, the initial feature map is rotated counterclockwise along the width axis by a second learnable angle to obtain a shape as follows. The second rotated feature map is used to swap the height and width dimensions; Energy-driven spatial attention is applied to the second rotated feature map to obtain a second attention weight. The second attention weight is then multiplied element-wise with the second rotated feature map, and the second learnable angle is rotated clockwise along the height axis to obtain the shape restored as follows. The second enhanced feature map; In the third branch, the energy-driven spatial attention is directly applied to the initial feature map to obtain the third attention weight, which is then multiplied element-wise with the initial feature map to obtain the third enhanced feature map. ; The first enhanced feature map, the second enhanced feature map, and the third enhanced feature map are fused to obtain the enhanced feature map. .

3. The infrared small target detection method based on a pyramid morphological network according to claim 2, characterized in that, The first learnable angle and the second learnable angle are defined by arctangent parameterization: Where θ is the first learnable angle or the second learnable angle. and These are learnable parameters to ensure that the rotation angle is bounded within the range of (-90°, 90°) and is automatically optimized through backpropagation.

4. The infrared small target detection method based on a pyramid morphological network according to claim 2, characterized in that, Apply energy-driven spatial attention in the following manner: Calculate the spatial mean μ of the feature map to be processed: Calculate the deviation of each pixel value x in the feature map to be processed from the spatial mean μ, and calculate the spatial variance accordingly. ; For each pixel, calculate its normalized energy. ; The normalized energy is activated using Sigmoid to obtain the attention weight map; The attention weight map is multiplied element-wise with the feature map to be processed.

5. The infrared small target detection method based on pyramid morphological networks according to claim 1, characterized in that, The multi-scale spatial pyramid pooling morphology module of the spatial pyramid pooling module adopts a fast spatial pyramid pooling structure, and performs the following operations on the input enhanced feature map in sequence: Channel alignment is achieved through 1×1 convolution; The channel-aligned feature map is subjected to multiple max pooling operations, with the output of the previous pooling operation used as the input for each pooling operation. All feature maps before pooling are concatenated with the feature maps after each pooling in the channel dimension; The spliced ​​feature map is convolved with 1×1 to obtain the aggregated feature.

6. The infrared small target detection method based on a pyramid morphological network according to claim 1 or 5, characterized in that, The morphological operations are implemented through a morphological enhancement unit, which contains four parallel morphological branches to process the aggregated features; the four morphological branches are dilation branch, erosion branch, opening operation branch, and closing operation branch. The dilation branch is used to perform a dilation operation on the aggregated feature; The corrosion branch is used to perform corrosion operations on the polymerization feature; The opening operation branch is used to perform an erosion operation on the aggregated features first, and then a dilation operation; The closing operation branch is used to perform a dilation operation on the aggregated feature first, and then an erosion operation; The outputs of the four morphological branches are respectively processed by 1×1 convolution to adjust the channels, then concatenated in the channel dimension, and then fused by 1×1 convolution. The fused features are multiplied by the learnable gain parameter β and added to the aggregated features to obtain the output feature map of the spatial pyramid pooling morphological module.

7. The infrared small target detection method based on a pyramid morphological network according to claim 6, characterized in that, The expansion operation is implemented through a max pooling layer; the erosion operation includes max pooling the aggregation feature and taking the negative of the pooling result to approximate the minimum value of the sliding window.

8. The infrared small target detection method based on pyramid morphological networks according to claim 1, characterized in that, The feature extraction network uses YOLOv8n as the base network, the backbone network is the YOLOv8n backbone network, the learnable direction interactive attention module is added to the backbone network, and the spatial pyramid pooling morphology module is used to replace the original spatial pyramid pooling module of YOLOv8n.

9. An infrared small target detection device based on a pyramid morphological network, characterized in that, include: The feature extraction module is used to extract features from the infrared image to be detected and output the target detection result; the feature extraction module includes a backbone network, a learnable directional interactive attention unit, and a spatial pyramid pooling morphological unit; The backbone network is used to extract an initial feature map from the infrared image; The learnable directional interactive attention unit is used to perform multi-directional learnable rotation and energy attention weighting on the initial feature map, fuse features from different directional branches, and output an enhanced feature map. The spatial pyramid pooling morphological unit includes cascaded multi-scale spatial pyramid pooling sub-units and morphological enhancement sub-units; the multi-scale spatial pyramid pooling sub-units are used to perform multi-scale pooling aggregation on the enhanced feature map and output aggregated features; The morphological enhancement subunit is used to perform morphological operations on the aggregated features and output a purified feature map. The target detection module is used to generate target detection results based on the purified feature map.

10. The infrared small target detection device based on a pyramid morphological network according to claim 9, characterized in that, The learnable directional interactive attention unit specifically includes: The first directional processing subunit includes a cascaded first learnable rotation layer, a first energy attention layer, and a first inverse rotation layer. The first learnable rotation layer is used to rotate the initial feature map counterclockwise by a first learnable angle along the height axis and exchange the height and width dimensions. The first energy attention layer is used to apply energy-driven spatial attention to the rotated feature map. The first inverse rotation layer is used to rotate the weighted feature map clockwise by the first learnable angle along the width axis and restore the original dimensional shape to obtain the first enhanced feature. The second directional processing subunit includes a cascaded second learnable rotation layer, a second energy attention layer, and a second inverse rotation layer. The second learnable rotation layer is used to rotate the initial feature map counterclockwise along the width axis by a second learnable angle and swap the height and width dimensions. The second energy attention layer is used to apply energy-driven spatial attention to the rotated feature map. The second inverse rotation layer is used to rotate the weighted feature map clockwise along the height axis by the second learnable angle and restore the original dimensional shape to obtain the second enhanced feature. The original retained sub-units are used to directly perform energy attention weighting on the initial feature map to obtain the third enhanced feature; The fusion subunit is used to add and fuse the first enhancement feature, the second enhancement feature, and the third enhancement feature to obtain the enhancement feature map.

Images