Infrared dense weak and small target detection method based on background semantics and cross-task feature interaction

Abstract

The invention discloses an infrared dense weak and small target detection method based on background semantics and cross-task feature interaction, and the method comprises the steps: firstly constructing a DenseSIRST infrared image data set with background semantic annotation, introducing a pixel-level sky background segmentation mask, and improving the perception capability of a model for a background context; then, a double-branch network architecture named BAFE-Net is provided, a detection and semantic segmentation collaborative learning mode is adopted, a feature channel with high task discrimination is extracted through a dynamic channel selection mechanism, and a feature adapter is introduced to realize explicit channel replacement between two tasks; therefore, the distinguishing capability of the model on a real target and a false alarm in a complex scene is enhanced. According to the method, excellent detection precision and recall performance are obtained on a DenseSIRST infrared detection reference data set, the false alarm rate can be remarkably reduced, and the method is particularly suitable for application scenes with multi-target dense distribution, complex background and false alarm interference in infrared images, such as the infrared vision fields of border security, airspace monitoring, night search and rescue, unmanned system navigation and the like.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of infrared target detection, and specifically relates to an infrared dense small target detection method based on background semantics and cross-task feature interaction. BACKGROUND

[0002] Weak and small target detection in an infrared image is an important research direction in the field of computer vision and has wide application requirements in the fields of airspace monitoring and search and rescue tasks. However, the existing technology has many limitations in processing dense small target detection, mainly in the following aspects:

[0003] Data set defect problem: The existing infrared small target data sets such as SIRST, IRSTD1K and the like mainly focus on single target or sparse target scenes, and lack special data sets for dense target distribution scenes. This leads to insufficient generalization ability of the existing model when facing the dense target scene in actual application, and the detection precision significantly decreases.

[0004] Deficiency of existing model: The traditional infrared target detection method has significant limitations in model design and feature extraction. First, the background information is not fully utilized, and the existing method often ignores the importance of background semantic information and only focuses on the foreground target features, resulting in poor adaptability of the model to complex background environments. Under different background types such as sky, ground and buildings, the contrast between the target and the background has significant differences, but the existing method lacks an effective background perception mechanism. Second, the interference between dense targets is serious. When multiple small targets are densely distributed in the image, mutual occlusion and feature confusion between targets are easy to occur, and the traditional single target detection framework is difficult to effectively distinguish adjacent targets, and is prone to miss detection or false detection.

[0005] Specifically, the existing technology has the following deficiencies in the application of dense infrared small target detection: the detection precision and recall rate of the existing mainstream detection method on the dense target data set are generally low, far from meeting the actual application requirements. At the same time, the false positive rate of the existing method is high, and it is easy to misidentify background noise as a target in a complex background environment, which seriously affects the practicability of the system. Therefore, an integrated detection solution is needed that can effectively utilize background semantic information, enhance feature discrimination ability and adapt to dense target scenes. SUMMARY

[0006] The purpose of the present application is to provide an infrared dense small target detection method based on background semantics and cross-task feature interaction, aiming to systematically solve a series of problems such as lack of data set, insufficient utilization of background information and serious interference between targets in the existing technology in the detection of dense infrared small targets.

[0007] To achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows:

[0008] An infrared dense small target detection method based on background semantics and cross-task feature interaction, characterized by comprising the following steps:

[0009] Step 1: Extract single target image regions from public infrared small target data sets, and construct an infrared small target image library containing multiple shapes and scales.

[0010] (a) Extract single small target regions from public data sets such as SIRST V1, SIRST V2, IRSTD1K, and SIRSTAUG, ensuring diversity and representativeness of the targets.

[0011] (b) Perform minimum background cropping on each extracted target to ensure that the target region contains complete target information and minimal background interference, and establish a high-quality target instance library.

[0012] Step 2: Use the background-aware Gaussian Copy-Paste (BAG-CP) method to generate simulated images containing dense infrared small target clusters on real infrared background images, construct dataset DenseSIRST, and finally generate a dataset including 1024 infrared images and 13655 dense small targets.

[0013] (a) Semantic-aware region selection: Select the SIRST V2 dataset as the background image, and randomly select 1-3 candidate regions from the sky area of the background image, each with different sizes, as the dense target pasting region.

[0014] (b) Target pasting and fusion: For each dense region, randomly select 6-12 targets from the small target library, scale the targets to 5x5 pixels, and paste them into the dense region with a target spacing of 1-2 pixels to simulate natural cluster distribution.

[0015] (c) Adaptive Gaussian smoothing: Apply an adaptive Gaussian fusion matrix to dynamically rotate and scale the targets, better integrate the targets and background, perform smooth transition, avoid harsh visual boundaries, and enhance the realism of the simulated image.

[0016] Step 3: Perform pixel-by-pixel semantic labeling on the background region of the infrared image, and define the density quantization indicators of the clustering region.

[0017] (a) Background semantic labeling: Perform pixel-by-pixel semantic labeling on the background region of the infrared image, divide the background into "sky" and "non-sky" two categories, and generate the corresponding mask segmentation map, providing semantic prior for subsequent background-aware detection.

[0018] (b) Density quantification: define the density quantification index of the clustering area, define as in a dense area when the Euclidean distance between any two target centroids is less than a threshold, calculate the density of dense targets and sparse targets, and evaluate the density of target distribution.

[0019] Step 4: Construct a cross-task feature interaction network (BAFE-Net) containing a target detection and background semantic segmentation double branch to realize cross-task fusion of target features and background semantics.

[0020] (a) Double branch design: the detection head contains two parallel branches of target detection branch and background segmentation branch, realizes double branch interaction of target detection and background semantic segmentation, and improves the overall performance of the model through cooperative learning.

[0021] (b) Dynamic channel selection mechanism: calculate the channel weights of the target detection branch and the background segmentation branch, respectively select the top K channels with the highest weights of the detection branch and the background segmentation branch as the most discriminative feature channels, effectively filter redundant information, and enhance the pertinence of feature representation

[0022] (c) Cross-task feature hard exchange: construct a binary exchange mask to perform a bidirectional hard exchange operation between the target detection branch and the background segmentation branch, project the channel features selected by each branch into a common space through an MLP to realize cross-modal feature fusion, and make full use of the complementary information between tasks.

[0023] (d) Multi-task joint loss function: the target detection loss adopts Focal Loss to handle the class imbalance problem, combines IoU Loss to improve the positioning accuracy, and is used to optimize the classification and bounding box regression of the target; the background segmentation loss adopts cross-entropy loss, which is used to optimize the background semantic segmentation result, and the loss functions of the two tasks are weighted and fused according to the weight ratio of 1:1.

[0024] (e) Result generation: the target detection branch outputs the final detection result, and the performance of the detection result is evaluated based on the standard evaluation index, which can be quantitatively evaluated by the average precision mean (mAP), representing the overall detection performance of the model on dense small targets, and comprehensively evaluating the detection performance of BAFE-Net method.

[0025] The beneficial effects and core innovations of the present application are:

[0026] Innovative data set construction method: through the background perception Gaussian Copy-Paste technology, the problem of lack of dense infrared small target data set is effectively solved. Compared with the traditional Copy-Paste method, BAG-CP can generate more real and natural dense target distribution, providing high-quality data support for model training.

[0027] Dual-branch architecture design: for the first time, the target detection and background semantic segmentation tasks are combined, through

[0028] The dual-branch design of BAFE-Net realizes cross-task feature interaction. This design not only can make full use of background semantic information to guide target detection, but also can enhance the feature representation ability of the model through inter-task complementary learning. On the DenseSIRST dataset, the mAP 07 of BAFE-Net reaches 27.4%, which is improved by 4.2% compared with the baseline FCOS method.

[0029] Efficient feature interaction mechanism: the proposed dynamic channel selection and cross-task feature hard switch mechanism can adaptively select the most discriminative feature channels and realize explicit feature exchange between the detection branch and the segmentation branch. This design significantly improves the model's ability to distinguish dense targets.

[0030] Excellent detection performance: significant performance improvement is achieved on multiple infrared small target detection benchmark datasets. On the DenseSIRST dataset, the mAP 07 reaches 0.274, the mAP 12 reaches 0.248, on the IRSTD-1k dataset, the mAP 07 reaches 0.410, the mAP 12 reaches 0.370, and the comprehensive performance is better than the current mainstream target detection and infrared segmentation methods. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a flowchart for infrared dense target detection.

[0032] Figure 2 is a small target library target image.

[0033] Figure 3 is a BAG-CP architecture diagram for dataset construction.

[0034] Figure 4 is a DenseSIRST dataset display.

[0035] Figure 5 is a BAFE-Net method overall architecture diagram. DETAILED DESCRIPTION

[0036] The application provides an infrared dense small target detection method based on background semantics and cross-task feature interaction, which is used for solving technical problems such as serious target interference between dense targets, high false detection rate and insufficient use of background information in a dense target scene. The system as a whole comprises an infrared data set construction module, a cross-task feature interaction module of background segmentation task and detection task, a dynamic channel selection module and a cross-task feature hard exchange module. First, a DenseSIRST infrared image data set with background semantic annotation is constructed, a pixel-level sky background segmentation mask is introduced, and the perception ability of the model to the background context is improved. Then a dual-branch network architecture named BAFE-Net is proposed, which adopts a detection and semantic segmentation collaborative learning method, extracts feature channels with strong task discriminability through a dynamic channel selection mechanism, and introduces a feature adapter to realize explicit channel replacement between the two tasks, thereby enhancing the model's ability to distinguish real targets and false alarms in complex scenes.

[0037] The application will be further described below in combination with the drawings and examples in the specification.

[0038] Figure 1 An infrared dense target detection flowchart is given. The specific steps are as follows:

[0039] Step 1: Extract single target image regions from public infrared small target data sets, and construct an infrared small target image library containing multiple shapes and scales. The small target library image is as shown in Figure 2

[0040] (1) Extract single small target regions from SIRST V1, SIRST V2, IRSTD1K and SIRSTAUG public data sets to ensure the diversity and representativeness of the targets.

[0041] (2) Perform minimum background cropping on each extracted target to ensure that the target region contains complete target information and the background interference is minimized, and establish a high-quality target instance library.

[0042] Step 2: Use the background-aware Gaussian Copy-Paste (BAG-CP) method to fuse the synthesized small targets with the infrared background images in the public SIRST V2 data set to construct the data set DenseSIRST. The BAG-CP strategy is based on the principle of mixup, unlike simple copy-paste, BAG-CP will selectively paste the target into the semantic region determined by the background context, thereby enhancing the naturalness and diversity of the synthesized samples. The overall framework is as shown in Figure 3

[0043] ​​(1) Semantic-aware target pasting region selection: From each background image of SIRST V2, one to three candidate regions are randomly defined within the relatively smooth and uniform sky area. The size of each selected region is fixed as 20x20 pixels and serves as a dense pasting region for simulating clustered target distribution.

[0044] (2) Target pasting and blending process: Within each selected region, 6 to 12 targets are randomly sampled from the target library for pasting. The size of each target is adjusted to be no larger than 5x5 pixels to simulate realistic scale variations. The spacing between targets is set to be 1 to 2 pixels to simulate the natural clustered patterns commonly seen in dense small target scenes.

[0045] To seamlessly blend the pasted targets with the background, an adaptive Gaussian blending matrix G is applied to produce smooth transitions at target edges and avoid noticeable visual boundaries. The synthesized target T new is computed as:

[0046] T new = B raw + T raw x λ x G, λ ∈ [0.5, 1]

[0047] where B raw is the original background block, T raw is the original target instance, and λ is a random intensity scaling factor introduced to add diversity. The Gaussian matrix G is defined as:

[0048]

[0049] where w and h represent the width and height of the target, respectively. The parameters p x and p y are the offsets of the Gaussian center relative to the target size, randomly sampled from [0, 0.2]. The standard deviations s x and s y control the spread of the Gaussian kernel, randomly sampled from [0.3, 0.6] to achieve different degrees of edge softness and improve the naturalness of target boundaries.

[0050] (3) Geometric and appearance augmentation: To further increase target diversity, each pasted target undergoes a random rotation transformation with an angle θ randomly sampled from [-90°, 90°], implemented as follows:

[0051]

[0052] where (m x , m y ) is the center of the target block.

[0053] Step 3: Pixel-wise semantic annotation is performed on the background region in the infrared image, and a density quantification index of the clustering region is defined;

[0054] (1) Background semantic annotation: Pixel-wise semantic annotation is performed on the background region in the infrared image, and the background is divided into "sky" and "non-sky" two categories, and the corresponding mask segmentation graph is generated, providing semantic prior for subsequent background perception detection, as shown in Figure 4

[0055] (2) Density quantification: In order to quantitatively capture the degree of target clustering, a density measure customized for clustering regions in the dataset is defined. When the Euclidean distance between any two target centers is less than 10 pixels, it is considered to be in a clustering region. The density of the clustering region is calculated as follows:

[0056]

[0057] where represents the number of targets in the cluster, represents the cumulative pixel area covered by all target masks in the cluster, represents the total pixel area of the clustering region. This index reflects both the number of targets and the density.

[0058] In order to compare the consistency between images, the density value is normalized by minimum-maximum scaling according to the global minimum and maximum density observed in the clustering and sparse target regions. The average density of the clustering region is 0.4092, while the average density of the single target region is 0.1342, emphasizing the effectiveness of this index in distinguishing between dense grouping targets and sparse targets.

[0059] Step 4: Build a cross-task feature interaction network (BAFE-Net) containing target detection and background semantic segmentation double branches, realize cross-task fusion of target features and background semantics, and the overall architecture of BAFE-Net is shown in Figure 5

[0060] (1) Double branch design: Since traditional single-branch detection frameworks usually have difficulty in effectively distinguishing between real targets and background interference, especially in scenes where target information is limited and contextual clues are ambiguous. A multi-task collaborative framework is proposed, which integrates target detection and background semantic segmentation into a unified prediction head, called

[0061] ​​​BAFE-Head. The core motivation behind this design lies in the complementarity of these two tasks: object detection focuses on locating small and sparse targets, while semantic segmentation captures global scene semantics and background structure. By jointly optimizing these two tasks, the framework leverages background semantics to suppress false positives and utilizes target localization cues to refine segmentation boundaries. This joint formulation facilitates more comprehensive and robust feature representations, enhancing the network's discriminative ability in cluttered infrared scenes.

[0062] (2) Dynamic channel selection mechanism: This mechanism adaptively selects and exchanges task-related channel features between the detection and segmentation branches. Channel-level attention vectors are generated through a projection module, and weights are assigned to channels based on their contributions, selecting the top-k most important channels.

[0063] To capture fine-grained local representations, two independent convolution operations are first applied to the input feature map X, each reducing the channel dimension by half. These two branches aim to extract complementary local patterns.

[0064]

[0065] in This represents the concatenation operation. To integrate the global context and emphasize information-rich regions, average pooling and max pooling are applied along the channel dimension to form a compact spatial descriptor, followed by convolutional projection and the Sigmoid activation function.

[0066]

[0067] Where max and avg are the feature descriptors for the max pooling space and the average pooling space, respectively. σ is a convolution operation used to reduce the dimensionality of connected features, where σ is the sigmoid activation function. Feat denotes weighted fusion.

[0068] The attention map is segmented along the channels, and the previously extracted features X1 and X2 are reweighted using element-wise multiplication. The reweighted features are then concatenated to form the output of the projection module.

[0069]

[0070] Where ⊙ represents element-wise multiplication, the operator `<channel>` represents a channel-wise spatial weighted operation, using the corresponding attention map to modulate local features. `<mean>` represents a global average pooling operation, aggregating feature representations across spatial dimensions and reducing spatial information to a channel-wise global descriptor.

[0071] For each sample, select the top K channels with the highest weights, where K is half the total number of channels:

[0072] V topk ,I topk = TopK(m, K)

[0073] where V topk represents the top K largest weight values of each sample. I topk represents the channel index corresponding to these weights. This dynamic selection mechanism can adaptively identify the most informative channels for each sample, thereby enhancing the effectiveness of cross-task feature interaction.

[0074] (3) Cross-task feature hard swapping: A binary swapping mask is constructed to perform a bidirectional hard swapping operation between the target detection branch and the background segmentation branch. The channel features selected by each branch are projected into a common space through an MLP to achieve cross-modal feature fusion, fully utilizing the complementary information between tasks.

[0075] To perform feature swapping, let X1 and X2 be the input tensors from the target detection and background segmentation tasks, respectively. Using the channel index I topk from (2), a binary swapping mask e is constructed:

[0076]

[0077] where i represents the channel index. This mask identifies the feature channels that need to be swapped. The one-dimensional channel mask is broadcast to the spatial dimension:

[0078]

[0079] where represents the broadcast operation. The detection and segmentation branches generate E d1 and E d2 , respectively. Bidirectional hard swapping is performed as follows:

[0080]

[0081] where ⊙ represents element-wise multiplication, Y1 and Y2 represent the output features, E d1 and E d2 represent the dynamic mask. To address the cross-modal nature of features, an adapter module is introduced, which is implemented in the form of a multi-layer perceptron (MLP) to project features into a common space before swapping. This design enables effective cross-modal interaction while preserving the characteristics of specific modalities.

[0082] Table 1 verifies the ablation experiment results of each module, proving that each module can significantly improve detection performance.

[0083] Table 1 Comparison of performance of each module ablation experiment

[0084]

[0085] (4) Multi-task joint loss function: The target detection loss adopts Focal Loss to handle the class imbalance problem, and combines IoU Loss to improve the positioning accuracy, which is used to optimize the classification and bounding box regression of the target; the background segmentation loss adopts cross-entropy loss, which is used to optimize the background semantic segmentation result, and the loss functions of the two tasks are weighted and fused according to the weight ratio of 1:1.

[0086] (5) Result generation: The average accuracy (mAP) and recall rate with IoU threshold of 0.5 (mAP 07 ) and 0.75 (mAP 12 ) are used as evaluation indicators of the experiment. Compared with the current most advanced target detection method and infrared segmentation method, the advantages of BAFE-Net method in accuracy, recall rate, false alarm rate and other aspects are verified. Table 2 and Table 3 compare the most advanced target detection method on DenseSIRST and IRSTD-1k data sets respectively, it can be seen that the single-stage method of directly regressing the bounding box and the class on the feature pyramid has excellent performance on the two data sets, and the BAFE-Net method is better than all other single-stage methods on the two data sets. In order to position BAFE-Net in the field of infrared small target detection (IRSTD) comprehensively, it is compared with the most advanced method specially designed for this task, as shown in Table 4, the comparison results on DenseSIRST and IRSTD-1k data sets. The results show that the performance of BAFE-Net is also better than these special IRSTD methods, which further proves its effectiveness.

[0087] Table 2 Performance comparison of the proposed BAFE-Net and other mainstream target detection methods on DenseSIRST dataset

[0088]

[0089]

[0090] Table 3 Performance comparison of the proposed BAFE-Net and other mainstream target detection methods on IRSTD-1k dataset

[0091]

[0092]

[0093] Table 4 Performance comparison of the proposed BAFE-Net and other mainstream infrared segmentation methods on DenseSIRST and IRSTD-1k datasets

[0094]

[0095]

[0096] In summary, the application discloses an infrared dense small target dataset DenseSIRST, and relates to an infrared dense small target detection method based on background semantics and cross-task feature interaction. The application aims to solve technical problems such as serious target interference, high false detection rate, insufficient use of background information and the like in infrared small target detection, and provides an intelligent detection solution with high precision and low false alarm for infrared vision application fields such as border security, airspace monitoring, night search and rescue, unmanned system navigation and the like.

Claims

1. A method for detecting dense, weak infrared targets based on background semantics and cross-task feature interaction, characterized in that, Includes the following steps: Step 1: Extract single-target image regions from publicly available infrared small target datasets and construct an infrared small target image library containing various shapes and scales; Step 2: Using the background-aware Gaussian Copy-Paste (BAG-CP) method, simulated images containing dense clusters of small infrared targets are generated on the real infrared background image, and the dataset DenseSIRST is constructed. Step 3: Perform pixel-by-pixel semantic annotation on the background region in the infrared image and define the density quantization index of the clustered region; Step 4: Construct a cross-task feature interaction network (BAFE-Net) that includes two branches: target detection and background semantic segmentation. This network enables cross-task fusion of target features and background semantics. The infrared image to be detected is input into the network, and the target detection branch outputs the detection results of weak infrared targets in the image.

2. The infrared dense weak target detection method according to claim 1, characterized in that, The steps for constructing the infrared small target image library in step 1 are as follows: Step 21: Extract individual small target regions from the SIRST V1, SIRST V2, IRSTD1K, and SIRSTAUG public datasets to construct a diverse small target library; Step 22: Perform minimal background cropping on each extracted target to ensure that the target area contains complete target information and has minimal background interference.

3. The infrared dense weak target detection method according to claim 1, characterized in that, The steps in step 2 for generating the simulation image using the background-aware Gaussian Copy-Paste method are as follows: Step 31, semantically aware region selection: Select the SIRST V2 dataset as the background image, and randomly select 1-3 candidate regions from the sky region of the background image. Each region is of different size and is used as the dense target pasting region. Step 32, Target Pasting and Blending: For each dense area, randomly select 6-12 targets from the small target library, scale the targets to within 5×5 pixels, and set the target spacing to 1-2 pixels to simulate a natural clustered distribution; Step 33, Adaptive Gaussian Smoothing: Apply an adaptive Gaussian fusion matrix to achieve a smooth transition between the target and the background, avoiding harsh visual boundaries.

4. The infrared dense weak target detection method according to claim 1, characterized in that, The steps for background semantic annotation in step 3 are as follows: Step 41, Background semantic annotation: Perform pixel-by-pixel semantic annotation on the background region in the infrared image, divide the background into two categories: "sky" and "non-sky", and generate the corresponding mask segmentation map; Step 42, Density Quantization: Define the density quantization index of the clustering region. When the Euclidean distance between any two target centroids is less than the threshold, it is defined as being in a dense region.

5. The infrared dense weak target detection method according to claim 1, characterized in that, The steps for constructing the BAFE-Net dual-branch detection network in step 4 are as follows: Step 51, Dual-branch design: The detection head includes two parallel branches: an object detection branch and a background segmentation branch, to achieve dual-branch interaction between object detection and background semantic segmentation; Step 52, Dynamic Channel Selection Mechanism: Calculate the channel weights of the target detection branch and the background segmentation branch, and select the top K channels with the highest weights as the most discriminative feature channels; Step 53, Cross-task feature hard swap: Construct a binary swap mask, perform a bidirectional hard swap operation between the object detection branch and the background segmentation branch, and use MLP to project features into the common space to achieve cross-modal feature fusion; Step 54, Multi-task joint loss function: Simultaneously optimize the target detection loss and background segmentation loss, and the loss functions of the two tasks are weighted and fused in a 1:1 weight ratio; Step 55, Result Generation: The target detection branch outputs the final detection result, which can be quantitatively evaluated by the mean accuracy, characterizing the model's overall detection performance for dense and weak targets.

Images