Aerial infrared dim small target detection method based on spatial-frequency domain feature fusion

Abstract

The present invention discloses a method for detecting small infrared targets in the air based on spatial-frequency domain feature fusion. The method comprises: acquiring a plurality of infrared small target images under complex backgrounds; dividing the plurality of infrared small target images under complex backgrounds according to a preset ratio to obtain a training set and a test set of the plurality of infrared small target images under complex backgrounds; inputting the training set of infrared small target images under complex backgrounds into a pre-constructed target detection network using spatial-frequency domain feature fusion, training the pre-constructed target detection network using spatial-frequency domain feature fusion, and obtaining a successfully trained target detection network using spatial-frequency domain feature fusion. The present invention solves the technical problems of low detection accuracy and high false alarm and missed detection rates in existing convolutional neural networks for detecting small infrared targets under complex backgrounds.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a method for detecting small infrared targets in the air based on spatial-frequency domain feature fusion. Background Art

[0002] In order to solve the problem of infrared dim target detection, a variety of traditional methods have been proposed, which are mainly divided into filtering methods, local contrast methods, and low-rank methods; however, these methods generally rely on prior knowledge and are sensitive to hyperparameters, which requires them to be closely matched with specific data sets in practical applications, limiting their versatility and flexibility; in addition, traditional mathematical modeling algorithms are complex in design and often cannot effectively cope with diverse practical scenarios, and their generalization ability is weak; these factors make traditional methods perform poorly in the task of infrared dim target detection in complex backgrounds.

[0003] In recent years, with the development of deep learning technology, convolutional neural networks (CNNs), with their superior feature extraction and model fitting capabilities, have emerged and gradually become dominant in the field of infrared small target detection. Trained with large amounts of sample data, CNNs can automatically learn and extract complex hierarchical features from the data, significantly improving target-background distinction and small target detection accuracy. Although deep learning-based detection algorithms offer greater adaptability, infrared small target detection in complex backgrounds still suffers from low accuracy, high false alarm and missed detection rates, and low sensitivity and loss of feature information when extracting features from images with low local signal-to-noise ratios. Therefore, in-depth research on deep learning-based infrared small target detection methods is urgently needed to improve their detection performance. Summary of the Invention

[0004] The embodiment of the present invention provides an aerial infrared dim small target detection method based on space-frequency domain feature fusion, so as to at least solve the technical problems of low detection accuracy, high false alarm and missed detection rates when detecting infrared dim small targets in complex backgrounds using existing convolutional neural networks.

[0005] According to one aspect of an embodiment of the present invention, a method for detecting infrared small targets in the air based on space-frequency domain feature fusion is provided. The method may include: acquiring a number of infrared small target images under complex backgrounds; dividing the infrared small target images under complex backgrounds according to a preset ratio to obtain a training set and a test set of the infrared small target images under complex backgrounds; inputting the training set of infrared small target images under complex backgrounds into a pre-constructed target detection network with space-frequency domain feature fusion, training the pre-constructed target detection network with space-frequency domain feature fusion, and obtaining a successfully trained target detection network with space-frequency domain feature fusion, wherein the pre-constructed target detection network with space-frequency domain feature fusion includes an encoder module, a space-frequency dual-branch feature extraction module, an asymmetric space-frequency feature fusion decoder module, and a segmentation head module; inputting a test set of several infrared images in the air into the successfully trained target detection network with space-frequency domain feature fusion to obtain a segmentation result of the test set image.

[0006] Optionally, the encoder module includes four convolutional layers of different scales, and multi-scale extraction is performed on the training set images in sequence through the four convolutional layers of different scales to obtain feature maps of four different scales, wherein the four feature maps of different scales include a feature map of a first scale, a feature map of a second scale, a feature map of a third scale, and a feature map of a fourth scale.

[0007] Optionally, the space-frequency dual-branch feature extraction module includes a neck block, a spatial domain branch, a frequency domain branch, a connection module and a 1×1 convolution layer, the spatial domain branch includes a convolution module, a CBR module, a codebook module, a fusion module, a full connection and a 1×1 convolution layer, a channel-level multiplication module, and a channel-level addition module; the frequency domain branch includes a first normalization layer, a second normalization layer, a first self-attention mechanism, a second self-attention mechanism, a Fourier transform layer, a channel-level multiplication module, and a channel-level addition module; the feature map of the fourth scale is input into the neck block to obtain the initial feature; the initial feature is input into the spatial domain branch to obtain the spatial domain feature of the feature map of the fourth scale; the initial feature is input into the frequency domain branch to obtain the frequency domain feature of the feature map of the fourth scale; the spatial domain feature and the frequency domain feature are input into the connection module and the 1×1 convolution layer in sequence to obtain the target feature of the feature map of the fourth scale.

[0008] Optionally, inputting the initial features into the spatial domain branch to obtain the spatial domain features of the feature map of the fourth scale includes: inputting the initial features into the convolution module and the CBR module in sequence to obtain the target feature map; inputting the target feature map into the codebook module to obtain information of any codeword corresponding to the target feature map; inputting the information of all codewords into the fusion module to obtain the overall information of the target feature map for all codewords; inputting the overall information of the target feature map for all codewords into the fully connected layer and the 1×1 convolution layer in sequence to obtain the significant features of the target in the feature map of the fourth scale; calculating the significant features of the target in the feature map of the fourth scale and the scale factor coefficient, and inputting them and the target feature map into the channel-level multiplication module to obtain the local area features corresponding to the feature map of the fourth scale; inputting the initial features and the local area features into the channel-level addition module to obtain the spatial domain features of the feature map of the fourth scale.

[0009] Optionally, the decoder module of the asymmetric space-frequency feature fusion includes four asymmetric space-frequency feature fusion modules of different scales and a U-shaped dilated convolution, wherein the four asymmetric space-frequency feature fusion modules of different scales include an asymmetric space-frequency feature fusion module of a first size, an asymmetric space-frequency feature fusion module of a second size, an asymmetric space-frequency feature fusion module of a third size and an asymmetric space-frequency feature fusion module of a fourth size; the target features and feature maps of the feature map of the fourth scale are input into the asymmetric space-frequency feature fusion module of the first scale to obtain a first-scale fused feature map; the first-scale fused feature map is input into the U-shaped dilated convolution module to obtain a target fine feature map; the target fine feature map and the feature map of the third scale are input into the asymmetric space-frequency feature fusion module of the second scale to obtain a second-scale fine fused feature map; the second-scale fine fused feature map and the feature map of the second scale are input into the asymmetric space-frequency feature fusion module of the third scale to obtain a third-scale fine fused feature map; the second-scale fine fused feature map and the feature map of the first scale are input into the asymmetric space-frequency feature fusion module of the fourth scale to obtain a fourth-scale fine fused feature map.

[0010] Optionally, the fourth-scale fine fusion feature map is input into a segmentation head module to obtain a small target in an infrared small target image under a complex background.

[0011] Beneficial effects of the present invention:

[0012] (1) The present invention introduces a U-shaped network structure, which achieves an efficient balance between feature extraction and feature fusion through the symmetrical design of the encoder and decoder. In the feature extraction stage, the encoder gradually extracts the deep semantic information in the image through layer-by-layer convolution and pooling operations; while in the feature fusion stage, the decoder combines the deep semantic information with the shallow texture information through layer-by-layer upsampling and convolution operations to achieve comprehensive information fusion; this symmetrical structure not only helps to maintain the consistency of the spatial information and semantic information of the image, but also significantly improves the accuracy of the network; through the U-shaped network structure, key information in the image can be more effectively extracted and utilized, providing a solid foundation for subsequent image processing tasks.

[0013] (2) The present invention introduces the frequency domain processing method of fast Fourier transform into the convolutional neural network. The image is converted from the spatial domain to the frequency domain through FFT, and the frequency information in the frequency domain is used to effectively filter the image. This method can more accurately capture the detailed information in the image, especially the feature information of small targets. At the same time, the frequency domain processing method can also effectively suppress noise and interference and improve the contrast of the image. Therefore, the combination of spatial domain feature and frequency domain feature processing can significantly improve the network's detection ability for small targets, providing new ideas and methods for applications in the fields of image processing and computer vision.

[0014] (3) The present invention adopts an asymmetric fusion strategy, which achieves information complementarity and enhancement by effectively fusing shallow texture information and deep semantic information. In the feature fusion process, the asymmetric fusion strategy can flexibly adjust the fusion mode and weight according to the characteristics of features at different levels, thereby obtaining a more accurate and comprehensive feature representation. This strategy not only improves the detection performance of the network, but also enhances the robustness and generalization ability of the network. Through the asymmetric fusion strategy, the target object in the image can be more accurately identified, providing strong support for tasks such as image classification and target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0016] Figure 1 is a flow chart of a method for detecting small infrared targets in the air based on spatial-frequency domain feature fusion according to an embodiment of the present invention;

[0017] Figure 2 is a schematic diagram of a target detection network using pre-built spatial-frequency domain feature fusion according to an embodiment of the present invention;

[0018] Figure 3is a schematic diagram of a space-frequency dual-branch feature extraction module according to an embodiment of the present invention;

[0019] Figure 4 is a schematic diagram of an asymmetric space-frequency feature fusion module according to an embodiment of the present invention;

[0020] Figure 5 1 is a schematic diagram of ROC curves of SFNet and other methods on the NUAA dataset according to an embodiment of the present invention;

[0021] Figure 6 3 is a visualization diagram of the detection results according to an embodiment of the present invention and the detection results of a comparison network. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or precedence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or devices.

[0024] Example 1

[0025] According to an embodiment of the present invention, a method for detecting weak and small infrared targets in the air based on spatial-frequency domain feature fusion is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system comprising at least one set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0026] Figure 1 FIG. 1 is a flow chart of a method for detecting small infrared targets in the air based on spatial-frequency domain feature fusion according to an embodiment of the present invention. Figure 1As shown, the method may include the following steps:

[0027] Step S101: Acquire several infrared small target images under complex backgrounds.

[0028] In the technical solution provided in the above step S101 of the present invention, three public datasets, NUDT, NUAA, and IRSTD-1K, are used, and the training set images thereof are resized to 256×256.

[0029] Step S102 : dividing the plurality of infrared small target images under complex backgrounds according to a preset ratio to obtain a training set and a test set of the plurality of infrared small target images under complex backgrounds.

[0030] In the technical solution provided in step S102 of the present invention, the NUDT, NUAA, and IRSTD-1K public data sets are divided according to a preset ratio to obtain training sets and test sets of infrared dim target images under several complex backgrounds, and the images of the training sets and test sets are resized to 256×256.

[0031] Step S103: Input the training set of infrared dim target images under complex background into the pre-constructed target detection network with space-frequency domain feature fusion, train the pre-constructed target detection network with space-frequency domain feature fusion, and obtain a successfully trained target detection network with space-frequency domain feature fusion, wherein the pre-constructed target detection network with space-frequency domain feature fusion includes an encoder module, a space-frequency dual-branch feature extraction module, an asymmetric space-frequency feature fusion decoder module, and a segmentation head module.

[0032] In the technical solution provided in step S103 of the present invention, Figure 2 is a schematic diagram of a target detection network based on pre-built spatial-frequency domain feature fusion according to an embodiment of the present invention, such as Figure 2 As shown in the figure, the training set of infrared dim target images under complex background is input into the pre-constructed target detection network with space-frequency domain feature fusion, and the pre-constructed target detection network with space-frequency domain feature fusion is trained to obtain a successfully trained target detection network with space-frequency domain feature fusion, wherein the pre-constructed target detection network with space-frequency domain feature fusion includes an encoder module, a space-frequency dual-branch feature extraction module, an asymmetric space-frequency feature fusion decoder module and a segmentation head module.

[0033] In step S104, a test set of several aerial infrared images is input into the successfully trained target detection network with spatial-frequency domain feature fusion to obtain the segmentation results of the test set images.

[0034] In the technical solution provided in the above step S104 of the present invention, a test set of several aerial infrared images is input into the successfully trained target detection network with spatial-frequency domain feature fusion to obtain the segmentation results of the test set images.

[0035] The above method of this embodiment is further introduced below.

[0036] As an optional embodiment, in step S103, the encoder module includes four convolutional layers of different scales, and multi-scale extraction is performed on the training set image in sequence through the four convolutional layers of different scales to obtain feature maps of four different scales, wherein the feature maps of the four different scales include a feature map of a first scale, a feature map of a second scale, a feature map of a third scale, and a feature map of a fourth scale.

[0037] In this embodiment, if Figure 2 The four convolutional layers of different scales in the encoder extract the training set images in sequence to obtain the feature map of the first scale, the feature map of the second scale, the feature map of the third scale, and the feature map of the fourth scale.

[0038] As an optional embodiment, step S103, the space-frequency dual-branch feature extraction module includes a neck block and a spatial domain branch and a frequency domain branch, a connection module and a 1×1 convolution layer, the spatial domain branch includes a convolution module, a CBR module, a codebook module, a fusion module, a full connection and a 1×1 convolution layer, a channel-level multiplication module, and a channel-level addition module; the frequency domain branch includes a first normalization layer, a second normalization layer, a first self-attention mechanism, a second self-attention mechanism, a Fourier transform layer, a channel-level multiplication module, and a channel-level addition module; the feature map of the fourth scale is input to the neck block to obtain the initial feature; the initial feature is input to the spatial domain branch to obtain the spatial domain feature of the feature map of the fourth scale; the initial feature is input to the frequency domain branch to obtain the frequency domain feature of the feature map of the fourth scale; the spatial domain feature and the frequency domain feature are sequentially input to the connection module and the 1×1 convolution layer to obtain the target feature of the feature map of the fourth scale.

[0039] In this embodiment, Figure 3 Schematic diagram of a space-frequency dual-branch feature extraction module according to an embodiment of the present invention. Figure 3 As shown in , the space-frequency dual-branch feature extraction module includes a neck block, a spatial domain branch, a frequency domain branch, a connection module, and a 1×1 convolution layer. The spatial domain branch includes a convolution module, a CBR module, a codebook module, a fusion module, a full connection and a 1×1 convolution layer, a channel-level multiplication module, and a channel-level addition module. The frequency domain branch includes a first normalization layer, a second normalization layer, a first self-attention mechanism, a second self-attention mechanism, a Fourier transform layer, a channel-level multiplication module, and a channel-level addition module. Figure 2 and Figure 3As shown, the feature map of the fourth scale is input into the neck block to obtain the initial feature; the initial feature is input into the spatial domain branch to obtain the spatial domain feature of the feature map of the fourth scale; the initial feature is input into the frequency domain branch to obtain the frequency domain feature of the feature map of the fourth scale, and the spatial domain feature and the frequency domain feature are sequentially input into the connection module and the 1×1 convolution layer to obtain the target feature of the feature map of the fourth scale. Figure 3 The intrinsic codebook encoder in is the spatial domain branch, and the space-frequency query attention is the frequency domain branch.

[0040] As an optional embodiment, the initial feature is input into the spatial domain branch to obtain the spatial domain features of the feature map of the fourth scale, including: inputting the initial feature into the convolution module and the CBR module in sequence to obtain the target feature map; inputting the target feature map into the codebook module to obtain the information of any codeword corresponding to the target feature map; inputting the information of all codewords into the fusion module to obtain the overall information of the target feature map for all codewords; inputting the overall information of the target feature map for all codewords into the fully connected layer and the 1×1 convolution layer in sequence to obtain the significant features of the target in the feature map of the fourth scale; after calculating the significant features of the target in the feature map of the fourth scale and the scale factor coefficient, the target feature map and the target feature map are input into the channel-level multiplication module to obtain the local area features corresponding to the feature map of the fourth scale; the initial features and the local area features are input into the channel-level addition module to obtain the spatial domain features of the feature map of the fourth scale.

[0041] In this embodiment, if Figure 3 As shown in Figure 2, the spatial domain branches mainly include:

[0042] (1) Intrinsic code book : A set of learnable visual codewords, represented by , where K represents the total number of visual centers;

[0043] (2) Scale factor set :With codebook The corresponding set of factors is expressed as , used to adjust the mapping relationship between input features and codebook.

[0044] Target feature map After encoding, features are obtained , the image information corresponding to the kth codeword is calculated by the following formula:

[0045]

[0046] in, is the i-th pixel in the target feature map; is the kth learnable visual codeword; is the kth scaling factor; Relative to the codeword The position information of each pixel point will be used to convert all code words The information fusion of the target feature map is expressed as follows:

[0047]

[0048] in, Contains BN layer, ReLU layer and average layer.

[0049] After obtaining the output of the codebook, Input to a fully connected layer and a 1×1 convolution layer, the salient features of the target in the feature map of the fourth scale. Then, the target feature map and the scale factor coefficient The above process can be expressed as:

[0050]

[0051] in, Represents 1×1 convolution; is the sigmoid activation function; represents channel-level multiplication, It is the local area feature corresponding to the feature map of the fourth scale.

[0052] Finally, the expression for the channel-level addition operation of the initial features and the local area features Z corresponding to the feature maps of the four scales is:

[0053]

[0054] in, is channel-level addition, It is the spatial domain feature of the feature map of the fourth scale.

[0055] As an optional embodiment, the initial feature is input into the frequency domain branch to obtain the frequency domain feature of the feature map of the fourth scale, including: inputting the initial feature into the first normalization layer to obtain the normalized initial feature, inputting the normalized initial feature into the first self-attention mechanism to obtain the first target feature; inputting the normalized initial feature into the frequency domain feature processor to obtain the frequency domain feature of the normalized initial feature; inputting the frequency domain feature of the normalized initial feature and the first target feature into the second self-attention mechanism to obtain the second target feature; after inputting the second target feature and the initial feature into the channel-level addition module, they are sequentially input into the second normalization layer and the Fourier transform layer to obtain the frequency domain map of the feature map of the fourth scale; after inputting the second target feature and the initial feature into the channel-level addition module, they are input into the channel-level addition module together with the frequency domain map of the feature map of the fourth scale to obtain the frequency domain feature of the feature map of the fourth scale.

[0056] In this embodiment, if Figure 3 As shown in Figure 2, Spatial-Frequency QueryAttention (SFQA) consists of two layers of self-attention mechanisms.

[0057]

[0058] Among them, X and Y represent the frequency domain features of the input initial features and the output fourth-scale feature map of the SFQA module, respectively; is the first target feature, Depthwise convolution operator representing channel space context, with kernel size m×m; Represents the convolution operator with a kernel size of m×m, which is used to capture cross-channel pixel-level contextual features. m can take values ​​from the set {1,2,3}. Before applying the Softmax function, the dot product is modulated by adjusting the learning parameter α. This scaling factor can control the size of the dot product, thereby affecting the intensity of attention.

[0059] As an optional embodiment, step S103, the decoder module of the asymmetric space-frequency feature fusion includes four asymmetric space-frequency feature fusion modules of different scales and a U-shaped dilated convolution, wherein the four asymmetric space-frequency feature fusion modules of different scales include an asymmetric space-frequency feature fusion module of a first size, an asymmetric space-frequency feature fusion module of a second size, an asymmetric space-frequency feature fusion module of a third size, and an asymmetric space-frequency feature fusion module of a fourth size; the target features and feature maps of the feature map of the fourth scale are input into the asymmetric space-frequency feature fusion module of the first scale to obtain a first-scale fused feature map; the first-scale fused feature map is input into the U-shaped dilated convolution module to obtain a target fine feature map; the target fine feature map and the feature map of the third scale are input into the asymmetric space-frequency feature fusion module of the second scale to obtain a second-scale fine fused feature map; the second-scale fine fused feature map and the feature map of the second scale are input into the asymmetric space-frequency feature fusion module of the third scale to obtain a third-scale fine fused feature map; the second-scale fine fused feature map and the feature map of the first scale are input into the asymmetric space-frequency feature fusion module of the fourth scale to obtain a fourth-scale fine fused feature map.

[0060] In this embodiment, the decoder module of the asymmetric space-frequency feature fusion is as follows Figure 2 As shown in the decoder, Figure 4 is a schematic diagram of an asymmetric space-frequency feature fusion module according to an embodiment of the present invention. Figure 4 As shown in the figure, the processing process of the asymmetric space-frequency feature fusion module is as follows: shallow features are processed by the space-frequency pixel attention mechanism to retain the texture and position information of the target; this mechanism uses the pixel-level attention weights in the spatial and frequency domains to weight shallow features, highlight the target area, and suppress background noise; deep features are processed by the channel attention mechanism to enhance semantic information; the channel attention mechanism weights deep features according to the feature importance of each channel, highlights channels with significant semantic information, and improves the semantic understanding ability of the network; the two are fused in an asymmetric way to ensure that more important information of weak targets is retained during the fusion process; the asymmetric fusion strategy flexibly adjusts the fusion method and weight according to the different characteristics of shallow features and deep features to achieve information complementarity and enhancement; the structure of the U-shaped dilated convolution module is shown in the figure. Figure 2 shown.

[0061] As an optional embodiment, the fourth-scale fine fusion feature map is input into the segmentation head module to obtain a weak target in the infrared weak target image under a complex background.

[0062] In this embodiment, the input segmentation head calculates the segmentation loss function SoftIoULoss to guide the network to perform optimization training. The specific formula is as follows:

[0063]

[0064]

[0065]

[0066] in, is the intersection area between the predicted result and the true value; is the union area between the predicted result and the true value, is the first in the input vector corresponding to the fourth scale fine fusion feature map i elements, K is the total number of elements in the input vector, that is, the total number of categories.

[0067] Experimental part:

[0068] The training sets of the embodiments of the present invention include the training sets of three public datasets, NUDT, NUAA, and IRSTD-1K, which respectively include 1,000 synthetic images, 223 real images, and 800 real images. The test sets of the embodiments of the present invention are the test sets of three public datasets, NUDT, NUAA, and IRSTD-1K, which respectively include 327 synthetic images, 224 real images, and 200 real images.

[0069] The embodiment of the present invention takes a synthetic infrared image, a real infrared image and its corresponding transmission map as input, and uses a reference image to calculate the loss function. All images are resized to 256×256. The embodiment of the present invention uses SoftIoULoss as the loss function, adopts the AdamW optimizer, and uses a multi-learning rate decay strategy and a deep supervision training strategy. The experimental batch size is set to 8, the initial learning rate is 0.0003, and the input dimension is set to 64 by default. The embodiment of the present invention uses an NVIDIA GeForce RTX 3090 graphics card to train the entire network, and during the training process, the Adam optimizer is used for 1500 training iterations.

[0070] Based on the above description, Table 1 shows the test results of the method of the embodiment of the present invention and some related methods on the NUDT, NUAA, and IRSTD-1K datasets. The accuracy of the test results is described using the target detection evaluation indicators intersection over union (IoU), detection probability (Pd), false alarm rate (Fa), and ROC curve (Receiver Operating Characteristic). It can be found that the method of the embodiment of the present invention basically achieves the best results in all indicators.

[0071] 1) Intersection over Union (IoU): A pixel-level evaluation metric used to evaluate the segmentation performance of an algorithm. IoU is defined as the ratio of the intersection area of ​​the predicted result to the true label to the union area of ​​the true label. The formula is as follows:

[0072]

[0073] in, is the intersection area; is the area of ​​the union.

[0074] 2) Detection Probability (Pd): An object-level evaluation metric used to evaluate the accuracy of object detection. Pd is defined as the number of correctly predicted objects. and the total number of targets The ratio is as follows:

[0075]

[0076] When the centroid deviation of the predicted target is less than a predefined threshold (set to 3 pixels in the embodiment of the present invention), the prediction is considered to be a correct detection.

[0077] 3) False Alarm Rate (Fa): Another target-level evaluation metric used to evaluate incorrect predictions. Fa is defined as the number of pixels that are incorrectly predicted. Total number of pixels The ratio is as follows:

[0078]

[0079] If the centroid deviation of the predicted target is greater than a predefined threshold (set to 3 pixels in the embodiment of the present invention), the prediction is considered to be a false positive.

[0080] 4) Receiver Operating Characteristic (ROC) Curve: This curve plots the relationship between the True Positive Ratio (TRR) and the False Positive Ratio (FPR) to evaluate the network's detection performance at different thresholds. The closer the ROC curve is to the upper left corner, the higher the detection accuracy. The area under the ROC curve (AUC) also reflects detection performance.

[0081] Table 1 Comparison of the method of the embodiment of the present invention with other related methods on the NUDT, NUAA, and IRSTD-1K datasets

[0082]

[0083] besides, Figure 5 : is a schematic diagram of ROC curves of SFNet and other methods on the NUAA dataset according to an embodiment of the present invention, as shown in FIG. Figure 5 As shown, the area enclosed by the curve of the embodiment of the present invention and the x-axis is the largest, and the inflection point of the curve is closest to the upper left corner, indicating that the embodiment of the present invention can achieve a high detection rate while maintaining a low false alarm rate and has the best performance.

[0084] Figure 6 is a visualization diagram of the detection results according to an embodiment of the present invention and the detection results of the comparison network, Figure 6 The detection results of the comparison networks DNANet, ABCNet and the embodiment of the present invention are visualized. The red boxes indicate correctly detected targets, and the yellow boxes indicate false alarms. It can be seen that although the three networks all have the ability to achieve target segmentation, the number of false alarms in the embodiment of the present invention is significantly reduced compared with DNANet and ABCNet, and the description of the target edge contour is more accurate.

[0085] In an embodiment of the present invention, a plurality of infrared small target images under complex backgrounds are obtained; the plurality of infrared small target images under complex backgrounds are divided according to a preset ratio to obtain a training set and a test set of the infrared small target images under complex backgrounds; the training set of the infrared small target images under complex backgrounds is input into a pre-constructed target detection network with spatial-frequency domain feature fusion, and the pre-constructed target detection network with spatial-frequency domain feature fusion is trained to obtain a successfully trained target detection network with spatial-frequency domain feature fusion, wherein the pre-constructed target detection network with spatial-frequency domain feature fusion includes an encoder module, a spatial-frequency dual-branch feature extraction module, an asymmetric spatial-frequency feature fusion decoder module, and a segmentation head module; the test set of the plurality of aerial infrared images is input into the successfully trained target detection network with spatial-frequency domain feature fusion to obtain a segmentation result of the test set image, thereby solving the technical problem of low detection accuracy and high false alarm and missed detection rates when detecting infrared small targets under complex backgrounds using existing convolutional neural networks, and achieving the technical effect of improving detection accuracy and reducing false alarm and missed detection rates by using a designed network framework to detect infrared small targets under complex backgrounds.

[0086] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0087] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0088] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0089] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0090] In addition, the functional units in various embodiments of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0091] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting small infrared targets in the air based on spatial-frequency domain feature fusion, characterized by: include: Acquire several infrared dim target images under complex backgrounds; Dividing a plurality of infrared dim target images under complex backgrounds according to a preset ratio to obtain a training set and a test set of the plurality of infrared dim target images under complex backgrounds; Inputting a training set of infrared dim target images under complex background into a pre-built target detection network with space-frequency domain feature fusion, training the pre-built target detection network with space-frequency domain feature fusion, and obtaining a successfully trained target detection network with space-frequency domain feature fusion, wherein the pre-built target detection network with space-frequency domain feature fusion includes an encoder module, a space-frequency dual-branch feature extraction module, an asymmetric space-frequency feature fusion decoder module, and a segmentation head module; The space-frequency dual-branch feature extraction module includes a neck block, a spatial domain branch, a frequency domain branch, a connection module, and a 1×1 convolution layer. The spatial domain branch includes a convolution module, a CBR module, a codebook module, a fusion module, a full connection and a 1×1 convolution layer, a channel-level multiplication module, and a channel-level addition module. The frequency domain branch includes a first normalization layer, a second normalization layer, a first self-attention mechanism, a second self-attention mechanism, a Fourier transform layer, a channel-level multiplication module, and a channel-level addition module. The asymmetric space-frequency feature fusion decoder module includes four asymmetric space-frequency feature fusion modules of different scales and a U-shaped dilated convolution, wherein the four asymmetric space-frequency feature fusion modules of different scales include an asymmetric space-frequency feature fusion module of a first size, an asymmetric space-frequency feature fusion module of a second size, an asymmetric space-frequency feature fusion module of a third size, and an asymmetric space-frequency feature fusion module of a fourth size; A test set of several aerial infrared images is input into the successfully trained target detection network with spatial-frequency domain feature fusion to obtain the segmentation results of the test set images.

2. The method according to claim 1, characterized in that The encoder module includes four convolutional layers of different scales, and multi-scale extraction is performed on the training set images in sequence through the four convolutional layers of different scales to obtain feature maps of four different scales, wherein the four feature maps of different scales include a feature map of a first scale, a feature map of a second scale, a feature map of a third scale, and a feature map of a fourth scale.

3. The method according to claim 2, characterized in that Input the feature map of the fourth scale into the neck block to obtain the initial features; Input the initial features into the spatial domain branch to obtain the spatial domain features of the feature map of the fourth scale; Input the initial features into the frequency domain branch to obtain the frequency domain features of the feature map of the fourth scale; The spatial domain features and frequency domain features are sequentially input into the connection module and the 1×1 convolution layer to obtain the target features of the feature map of the fourth scale.

4. The method according to claim 3, characterized in that The step of inputting the initial features into the spatial domain branch to obtain the spatial domain features of the feature map at the fourth scale includes: Input the initial features into the convolution module and the CBR module in sequence to obtain the target feature map; Input the target feature map into the codebook module to obtain the information of any codeword corresponding to the target feature map; Input the information of all codewords into the fusion module to obtain the overall information of the target feature map for all codewords; The overall information of the target feature map for all codewords is sequentially input into the fully connected layer and the 1×1 convolutional layer to obtain the salient features of the target in the feature map of the fourth scale; After calculating the salient features and scale factor coefficients of the target in the feature map of the fourth scale, the salient features and the target feature map are input into the channel-level multiplication module to obtain the local area features corresponding to the feature map of the fourth scale; The initial features and local area features are input into the channel-level addition module to obtain the spatial domain features of the feature map of the fourth scale.

5. The method according to claim 3, characterized in that Inputting the initial features into the frequency domain branch to obtain the frequency domain features of the feature map of the fourth scale includes: Input the initial features into the first normalization layer to obtain the normalized initial features, and input the normalized initial features into the first self-attention mechanism to obtain the first target features; Inputting the normalized initial features into the frequency domain feature processor to obtain the frequency domain features of the normalized initial features; The normalized frequency domain features of the initial features and the first target features are input into the second self-attention mechanism to obtain the second target features; After the second target feature and the initial feature are input into the channel-level addition module, they are sequentially input into the second normalization layer and the Fourier transform layer to obtain the frequency domain map of the feature map of the fourth scale; After the second target feature and the initial feature are input into the channel-level addition module, the frequency domain map of the feature map of the fourth scale is input into the channel-level addition module to obtain the frequency domain features of the feature map of the fourth scale.

6. The method according to claim 3, characterized in that Input the target features and feature map of the feature map of the fourth scale into the asymmetric space-frequency feature fusion module of the first scale to obtain the fused feature map of the first scale; The first-scale fusion feature map is input into the U-shaped dilated convolution module to obtain the target fine feature map; Input the target fine feature map and the feature map of the third scale into the asymmetric space-frequency feature fusion module of the second scale to obtain the fine fused feature map of the second scale; Inputting the second-scale fine fusion feature map and the second-scale feature map into the third-scale asymmetric space-frequency feature fusion module to obtain the third-scale fine fusion feature map; The second-scale fine fusion feature map and the first-scale feature map are input into the fourth-scale asymmetric space-frequency feature fusion module to obtain the fourth-scale fine fusion feature map.

7. The method according to claim 6, characterized in that The fourth-scale fine fusion feature map is input into the segmentation head module to obtain a weak target of the infrared weak target image under a complex background.

8. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

9. A computer-readable storage medium, characterized in that Computer-executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 1.

10. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the method of claim 1 when the instructions are executed.

Images