Infrared small target detection method based on wavelet high and low frequency component difference learning

Abstract

The invention discloses an infrared small target detection method based on wavelet high and low frequency component difference learning, an infrared small target detection model is constructed based on a double-encoder feature extraction network, and the double-encoder feature extraction network comprises a low-frequency encoder and a high-frequency encoder with input data coupled; performing wavelet transform decomposition to obtain low-frequency component infrared image features and high-frequency component infrared image features; learning faint infrared small target features by taking the low-frequency component image features as input; high-frequency component infrared image features serve as input, and target-background clutter difference features are learned to achieve effective distinguishing of small targets and background clutters; original infrared image features are obtained through reconstruction in a decoder, infrared small target detection is achieved through the trained infrared small target detection model, and a final prediction result is output.

Description

Technical Field

[0001] The present invention relates to the field of infrared imaging detection and recognition technology, and in particular to a small target detection method based on wavelet high and low frequency component difference learning. Background Art

[0002] Infrared small target detection, which involves discerning faint targets against complex backgrounds, has important applications in scenarios such as maritime rescue and coastal defense target monitoring. Due to the widespread availability of relevant datasets for infrared small target detection, deep learning methods have rapidly developed in the past few years, significantly outperforming traditional methods. Some of these methods have achieved promising results, but unlike general-purpose target detection tasks in the visible light band, long-range small target detection still faces challenges due to the unique characteristics of infrared imaging equipment and small targets. 1) Network Downsampling Mechanism: Current models for infrared small target detection mostly use the UNet as a baseline. To obtain deep semantic features of the target, they employ a pooling module for downsampling and an interpolation algorithm for upsampling. This process results in further sparse information in the already sparse target region. The pixel information lost due to pooling cannot be restored by interpolation. This results in irreversible information loss in the entire scene, leading to a sharp drop in target response. 2) Targets are dim and weak: Due to atmospheric scattering and absorption at long distances, as well as the target's own radiation capacity, small targets appear dim and weak, lacking saliency. This results in a low number of learnable target features, making it more difficult for the network to capture target information. 3) Background clutter and easily confused target interference: Due to the influence of imaging equipment and environmental factors, infrared images contain background clutter and inherent noise, potentially confusing targets. Since small targets often lack geometric shapes and textures, these interferences appear as similarities to real small targets.

[0003] Finding appropriate solutions to these problems is key to improving the performance of infrared small target detection tasks. 1) A relatively lossless network downsampling mechanism: Infrared small target detection tasks require, more than general-purpose target detection, a method for learning target and scene features while ensuring that the already sparse features of small targets are not irreversibly affected by downsampling. Wavelet transforms provide a nearly lossless downsampling method and are widely used in multiple vision tasks. 2) Improving target information capture: Infrared images show dim target regions, limiting the amount of extractable features, making direct local feature learning on the original image inefficient. Compared to the original image, the low-frequency component image after wavelet processing shows an overall increase in information content, making target features more significant. 3) Adaptive long-range relationship capture: Establishing correlations between target and background regions is crucial for improving the network's ability to distinguish true from false targets. Most current methods choose to directly establish long-range dependencies in the input data. However, gradient changes and pixel fluctuations in irrelevant background regions dilute and interfere with the weights of correlation signals between core regions when the model establishes long-range dependencies. The high-frequency component image after wavelet processing shows more differentiated inter-regional features as a whole. This type of data is more suitable for setting up a long-distance dependency capture module to establish the relationship between the target, background clutter, and confused targets, so as to solve the problem of false alarm targets being difficult to distinguish. Summary of the Invention

[0004] In response to the defects existing in the above-mentioned prior art, the present invention aims to propose an infrared small target detection method based on the differential learning of wavelet high and low frequency components, construct a learning data processing network designed specifically for two types of components, low frequency components and high frequency components, and use the learned differential features to realize the training of infrared small target detection model, and realize infrared small target detection through the trained infrared small target detection model.

[0005] The present invention is achieved by utilizing the following technical solutions:

[0006] The present invention proposes an infrared small target detection method based on wavelet high- and low-frequency component difference learning, which includes the following steps:

[0007] S1, obtaining a pair of low-frequency component infrared image features and high-frequency component infrared image features according to the original infrared image;

[0008] S2, constructing an infrared small target detection model based on a dual-encoder feature extraction network, wherein the dual-encoder feature extraction network includes a low-frequency encoder and a high-frequency encoder coupled with input data;

[0009] S3, downsampling the low-frequency component infrared image features and the high-frequency component infrared image features obtained in S1 using a wavelet transform mechanism through a dual-encoder feature extraction network;

[0010] S4, using the low-frequency component image features as input to learn the features of dim small infrared targets through the low-frequency encoder; using the high-frequency component infrared image features as input to learn the difference features between the target and background clutter, so as to achieve effective distinction between the small target and background clutter;

[0011] In step S5, the characteristics of the dim infrared small target and the difference characteristics of the target-background clutter learned in step S4 are reconstructed and fused to train the infrared small target detection model. The infrared small target detection is realized through the trained infrared small target detection model, and the final prediction result is output.

[0012] Wherein S1 further includes that the low-frequency component infrared image feature is a significant local area target feature, the high-frequency component infrared image feature is a target-background clutter information with difference, and the high-frequency component infrared image feature is composed of three high-frequency components H, V, and D spliced ​​along the image channel dimension.

[0013] S3 further includes downsampling to obtain the downsampled low-frequency component infrared image features and low-frequency component infrared image features with only half the size of the original input image. After the feature learning is completed, the low-frequency component and the high-frequency component are reconstructed using inverse wavelet to obtain the original input image, realizing lossless downsampling and upsampling.

[0014] The low-frequency encoder in S2 is a dense feature pyramid structure, including a basic wavelet feature extraction unit and a dense wavelet feature extraction unit. The vertex of each layer of the dense feature pyramid structure is a basic wavelet feature extraction unit, and each layer of the pyramid is a series of dense wavelet feature extraction units; wherein the basic wavelet feature extraction unit extracts features from the original input image to obtain input data of different scales, and the input data is fused with low-scale basic wavelet feature features and high-scale dense wavelet features through multiple layers of dense wavelet feature extraction units to perform feature learning and obtain different receptive field features.

[0015] The basic wavelet feature extraction unit is a residual block structure. The input data first enters the basic wavelet feature extraction unit. In each basic wavelet feature extraction unit, the low-frequency component infrared image feature A and the high-frequency component infrared image feature U are both input and output feature learning through the residual block with a convolution layer, and the basic wavelet feature is output. It further includes the low-frequency component infrared image feature after residual processing. and high-frequency component infrared image features After the current node is processed, the low-frequency component will be further down-sampled through wavelet transform to obtain low-frequency and high-frequency components after further dimensionality reduction. These two components will serve as the input of the next node.

[0016] The dense wavelet feature extraction unit is a residual block structure, and its input includes the high / low frequency component image features decomposed from the previous node and the high / low frequency component image features decomposed from the adjacent node, and the input is fused. The input A of the dense wavelet feature extraction unit is ij The expression is as follows:

[0017]

[0018]

[0019] Among them, i and j represent the horizontal and vertical coordinates of the node position, that is, the node at the i-th row and j-th column position, Ψ represents the cascade fusion of multiple features at the same layer, DWT represents wavelet transform, that is, downsampling the input data using wavelet transform, IDWT represents inverse wavelet transform, that is, upsampling the component data through inverse wavelet transform to obtain the low-frequency component data before decomposition, μ represents the conventional upsampling operation, [1] represents the component with index 1 generated by wavelet transform, that is, the low-frequency component, [2] represents the component with index 2 generated by wavelet transform, that is, the high-frequency component, k represents the vertical coordinate 1~j-1, Represents the low-frequency component infrared image feature after residual processing of the node at position k in row i, Represents the low-frequency component infrared image feature after residual processing of the node at the i-1th row and jth column position, Represents the low-frequency component infrared image feature after residual processing of the node at the i+1th row and j-1th column position, Represents the high-frequency component infrared image feature of the node at the i+1th row and j-1th column after residual processing, Represents the high-frequency component infrared image features of the node at position k in row i after residual processing.

[0020] The high-frequency encoder includes an adaptive wavelet attention mechanism and a multi-layer difference relationship capture module.

[0021] In the adaptive wavelet attention mechanism, the input high-frequency component infrared image data U is decomposed into four sub-components - a low-frequency component UA and three high-frequency components UH, UV, and UD. The low-frequency component UA is added element-by-element with the three high-frequency components UH, UV, and UD. Three different depth-wise separable convolutions are applied to the added result to enhance the local spatial information and map it to three different feature spaces, as shown in the following formula:

[0022]

[0023] in, Represents depthwise separable convolution, (UA+[UH,UV,UD]) refers to the addition of UA to the three components, X AH ,XAV ,X AD Represents variables in three high-frequency directional feature spaces;

[0024] Furthermore, in three different feature spaces, global average pooling and global maximum pooling are applied respectively to compress data from different feature spaces along the channel direction. Subsequently, the compressed feature vectors are concatenated to obtain unique guiding feature maps in each high-frequency direction feature space. An adaptive fusion strategy is introduced to allow the model to dynamically adjust the weights of each feature map according to the characteristics of the data to perform feature fusion optimization. The adaptive fusion strategy process is shown in the following formula:

[0025]

[0026]

[0027] Among them, P avg (·) and P max (·) represents GAP and GMP, Represents the guiding feature map in each feature space k, f 1×1 represents the convolution with a kernel of 1 used to change the number of channels, α, β, χ represent updateable weight parameters, Sigmoid represents the normalization function, and M represents the final attention image;

[0028] The output of the adaptive wavelet attention mechanism is shown as follows:

[0029] W O =UA⊙M+UA.

[0030] The multi-layer difference relationship capture module is a Transformer network model, which further includes a self-attention module and a cross-attention module. The self-attention module is used to establish semantic difference associations between different pixel areas within a single-resolution feature map; the cross-attention module is used to establish global difference relationships, build long-distance dependencies between all feature layers in the entire model, enhance the perception of high-frequency components, and reconstruct the difference relationship between the target and background clutter;

[0031] Given the adaptive wavelet attention mechanism output result W of layer i o ∈R ci,hi,wi , where ci, hi, wi represent the number of channels, height, and width of the feature map of layer i respectively; using 1×1 convolution, we get I i ∈R 128,hi,wi , and then flatten it to become a sequence of data T i ∈R 128,hi*wi ,The overall processing process of the multi-layer difference relationship capture module is shown in the following formula:

[0032]

[0033] Among them, S stands for self-attention layer, LN stands for linear normalization layer, and MLP stands for multi-layer perceptron. represents the result of the self-attention module, represents the high-frequency component infrared image features of the cascade corresponding to the layers other than the current layer, C represents the cross-attention layer, Represents the output of the cross-attention module, T i O Represents the final output of the multi-layer difference relationship capture module;

[0034] T i cas The composition is shown below:

[0035]

[0036] When the sub-layer number is greater than i, the sub-layer is the output of the multi-layer difference relationship capture module, providing the difference features between the target and the background; when the sub-layer number is less than i, the sub-layer is the input of the multi-layer difference relationship capture module.

[0037] The query, key, and value in the cross-attention module are as follows:

[0038]

[0039]

[0040]

[0041] In the above formula, W q , W k , W v Represents the linear fully connected layer matrix, using a four-attention-head structure to learn the attention matrix. represents the query, key, and value corresponding to the λth head of the i-th layer;

[0042] Then the correlation calculation is performed to obtain the output of the cross-attention layer, as shown in the following formula:

[0043]

[0044] The final output result is scaled to obtain the output of the encoder, and the original infrared image features are reconstructed in the decoder.

[0045] Compared with the prior art, the present invention can achieve the following beneficial technical effects:

[0046] 1) This method enhances local saliency and global contextual differences by decomposing the input data, and combines differential feature extraction that matches the data distribution characteristics to achieve complementary feature extraction; the input data is decomposed into low-frequency and high-frequency components using wavelet transform;

[0047] 2) For the low-frequency part that carries key target information, a low-frequency encoder is designed, using a densely connected convolutional neural network and a DWT-based downsampling technique to preserve the integrity of the target features and learn richer target features;

[0048] 3) For the high-frequency components rich in contrast information between the target and the background, an adaptive wavelet attention mechanism is introduced to optimize the fusion of multiple components and reduce the negative noise interference of high-frequency components. A multi-layer difference relationship capture module is designed to enhance target differentiation ability. The multi-component fusion is optimized through adaptive weighting. At the same time, the multi-scale feature maps are connected through the multi-layer difference relationship capture module to enhance target differentiation ability, realizing a high-precision infrared small target detection method.

[0049] 4) Combining the characteristic advantages of high- and low-frequency domain data, richer and more discriminative features are extracted to improve detection accuracy while reducing the false alarm rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flow chart of an infrared small target detection method based on wavelet high and low frequency component difference learning of the present invention;

[0051] Figure 2 This is a technical roadmap for infrared small target detection based on differential learning of wavelet high and low frequency components of the present invention;

[0052] Figure 3 1 is a diagram showing a comparison between a wavelet downsampling method according to an embodiment of the present invention and a conventional downsampling method;

[0053] Figure 4 Schematic diagram of a low-frequency encoder according to an embodiment of the present invention;

[0054] Figure 5 Schematic diagram of an adaptive wavelet attention mechanism based on wavelet according to an embodiment of the present invention;

[0055] Figure 6 Schematic diagram of a multi-layer difference relationship capture module according to an embodiment of the present invention;

[0056] Figure 7 The following is a comparison chart of the detection effects of the present invention and other methods on infrared images of different scenes. DETAILED DESCRIPTION

[0057] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0058] like Figure 1 As shown, the present invention proposes an infrared small target detection method based on wavelet high and low frequency component difference learning, comprising the following steps:

[0059] Step 1: After capturing the original infrared image I, perform wavelet transform (DWT) on it using formula (1), decompose the input low-frequency component infrared image features and the high-frequency component infrared image features, divide them into different frequency sub-bands, and obtain paired low-frequency component infrared images and high-frequency component infrared images. Then, obtain the low-frequency component infrared feature A of the low-frequency component infrared data, obtain the three high-frequency component infrared features H, V, and D of the high-frequency component infrared data, and divide the data into a training set, a validation set, and a test set.

[0060] (A, H, V, D) = DWT (I) (1) This step is further described as follows:

[0061] Step 101: concatenate the three high-frequency component infrared features H, V, and D along the image channel dimension to form a new high-frequency component infrared image feature U, expressed as U = Concat(H, V, D), where Concat represents a concatenation operation along the channel dimension.

[0062] Step 102: The low-frequency infrared feature A contains more significant local area target features, and the high-frequency infrared feature U contains more differentiated target-background clutter information, which is used in subsequent network flow.

[0063] Step 103: divide the data set into two parts. Specifically, 70% of the data set is used as a training set to train the constructed infrared small target detection model, 20% of the data set is used as a validation set to verify the model performance during the training phase, and 10% of the data set is used as a test set to test the performance of the optimal model.

[0064] Step 2: Construct an infrared small target detection model, which takes low-frequency component infrared image features and high-frequency component infrared image features as input and takes the detection results of the infrared small target detection model as output; the model is constructed based on a dual-encoder feature extraction network, and the structure of the network includes a low-frequency encoder and a high-frequency encoder coupled with input data, wherein the low-frequency encoder is designed as a densely connected convolutional neural network, which takes the low-frequency component generated by the wavelet transform as input and learns the characteristics of dim infrared small targets to obtain better detection targets. The high-frequency encoder is designed as a global difference relationship capture network, which takes the high-frequency component generated by the wavelet transform as input and learns the difference characteristics of the target and background clutter to better distinguish the target from the background clutter; therefore, the dual-encoder feature extraction network realizes the detection of the infrared small target detection model through wavelet high / low frequency component difference learning;

[0065] Step 3: Down-sample the low-frequency component infrared image features and the high-frequency component infrared image features obtained in S1 using a wavelet transform mechanism through a dual-encoder feature extraction network, thereby obtaining a larger receptive field while ensuring that rare features of small targets are not lost in conventional downsampling;

[0066] Step 301: In the downsampling process, instead of discarding the pixel information of the feature map directly, the input feature map is decomposed using wavelet transform to divide it into different frequency sub-bands, including low-frequency components and high-frequency components. At this time, the low-frequency component infrared data and the high-frequency component have only half the size of the original input image, thus achieving downsampling. Figure 3 As shown in Figure 1, (a) shows the traditional downsampling method. After the traditional downsampling and upsampling to restore the original size, the target features are significantly lost in the original infrared image and texture image. (b) shows a before-and-after comparison of wavelet downsampling and inverse wavelet upsampling to restore the original size. The original infrared image and texture image are almost intact. Therefore, after feature learning is completed, the low-frequency and high-frequency components are reconstructed using inverse wavelet to obtain the original input image, achieving lossless downsampling and upsampling.

[0067] Step 4: The low-frequency encoder uses a densely connected convolutional neural network as input to extract features of the low-frequency component infrared image using the densely connected convolutional neural network of the low-frequency encoder. That is, the local feature extraction capability of the convolutional neural network is used to extract features of the low-frequency component infrared image with more significant local area features, thereby completing feature learning of dim targets.

[0068] Step 5: Utilize the dim infrared small target features and target-background clutter difference features learned in step S4 to reconstruct and fuse them in the multi-layer difference relationship capture module as the Transformer network model decoder, train the infrared small target detection model, implement infrared small target detection using the trained infrared small target detection model, and output the final prediction result.

[0069] Step 6: Set the training hyperparameters, train the built model, load the network weight file, collect real infrared images, use the wavelet transform module mentioned in the present invention to obtain low-frequency component images that can better reflect the significant characteristics of the local area of ​​the target and high-frequency component images that can better reflect the difference characteristics of the target-background clutter, input the component images into the trained infrared small target detection model, and output the final prediction results.

[0070] like Figure 4 As shown, the low-frequency encoder of the present invention adopts a dense feature pyramid structure, including a dense feature pyramid structure composed of a basic wavelet feature extraction unit and a dense wavelet feature extraction unit. Each basic wavelet feature extraction unit and the dense wavelet feature extraction unit are respectively used as a node. A plurality of such nodes are provided, and the nodes are interconnected to form a dense connection type structure. The vertex of each layer of the dense feature pyramid structure is a basic wavelet feature extraction unit, and each layer of the pyramid is a series of dense wavelet feature extraction units. The basic wavelet feature extraction unit is used to extract features from the original input image to obtain input data of different sizes. The input data is fused with low-scale basic wavelet feature features and high-scale dense wavelet features through multiple layers of dense wavelet feature extraction units to perform feature learning and obtain different receptive field features as effective image features.

[0071] The basic wavelet feature extraction unit is constructed using residual blocks. Input data first enters the basic wavelet feature extraction unit. In each basic wavelet feature extraction unit, the low-frequency infrared image feature A and the high-frequency infrared image feature U are input. Both input features and output features are learned through a residual block (ResBlock) with a convolutional layer. The output basic wavelet features further include the low-frequency infrared image feature A' and the high-frequency infrared image feature U' after residual processing. The residual block of each basic wavelet feature extraction unit consists of two convolutional layers and several skip connections. The skip connection between the two convolutional layers directly transmits the input data to the output. The basic wavelet feature extraction unit serves as the convolutional layer of the residual block.

[0072] After the current node (i.e., the basic wavelet feature extraction unit) is processed, the residual low-frequency component infrared image feature A' will be further downsampled through the wavelet transform to obtain further reduced low-frequency and high-frequency components. These two components will serve as the input of the next node. This is a lossless downsampling mechanism because the reduced-size component data can be reconstructed back to its original size through an inverse wavelet transform.

[0073] The structure of the dense wavelet feature extraction unit is similar to that of the basic wavelet feature extraction unit. The dense wavelet feature extraction unit is constructed using residual blocks. The key difference is that the input of the dense wavelet feature extraction unit includes not only the high / low frequency component image features decomposed by the extraction unit of the previous node (i.e., the dense feature extraction unit or the basic wavelet feature extraction unit), but also the high / low frequency component image features decomposed by the adjacent nodes, including large-scale feature maps representing more refined features, small-scale feature maps representing more semantic features, and multiple feature maps of the same layer (i.e., the same scale), all of which are fused into input.

[0074] The difference between the dense wavelet feature extraction unit and the basic wavelet feature extraction unit is that the dense wavelet feature extraction unit fuses feature maps of different layers (different scales) at the input end, and then is processed by the residual block.

[0075] Due to the characteristics of wavelet transform, the large-scale feature maps of low-frequency component image features and high-frequency component image features are fused in different ways, that is, the large-scale feature map of low-frequency component image features is obtained by inverse wavelet transform, while the large-scale feature map of high-frequency component image features is obtained by direct upsampling transform. ij Expressed using formulas (2) and (3):

[0076]

[0077]

[0078] Among them, i and j represent the horizontal and vertical coordinates of the position of the node (i.e., dense wavelet feature extraction unit or basic wavelet feature extraction unit), that is, the feature extraction unit (i.e., dense feature extraction unit or basic wavelet feature extraction unit) at the i-th row and j-th column position, Ψ represents the cascade fusion of multiple features at the same layer, DWT represents wavelet transform, that is, downsampling the input data using wavelet transform, IDWT represents inverse wavelet transform, that is, upsampling the component data through inverse wavelet transform to obtain the low-frequency component data before decomposition, μ represents the conventional upsampling operation, [1] represents the component with index 1 generated by wavelet transform, that is, the low-frequency component, [2] represents the component with index 2 generated by wavelet transform, that is, the high-frequency component, k is a formal parameter, representing the vertical coordinate 1~j-1, which is used to broadly represent many node positions, Represents the low-frequency component infrared image feature after residual processing of the node at position k in row i, Represents the low-frequency component infrared image feature after residual processing of the node at the i-1th row and jth column position, Represents the low-frequency component infrared image feature after residual processing of the node at the i+1th row and j-1th column position, Represents the high-frequency component infrared image feature of the node at the i+1th row and j-1th column after residual processing, Represents the high-frequency component infrared image features of the node at position k in row i after residual processing.

[0079] Since the wavelet transform decomposes the input features into four components A, H, V, and D, the high-frequency encoder is implemented using a global difference relationship capture network, adaptively fusing the three wavelet spatial component features H, V, and D representing high frequencies. The high-frequency encoder includes an adaptive wavelet attention mechanism and a multi-layer difference relationship capture module, and uses a transformer architecture to establish the difference characteristics of the target and background clutter in the input high-frequency component infrared data on a multi-scale feature map, thereby effectively distinguishing small targets from easily confused background clutter. It is further described as follows:

[0080] High-frequency infrared data contains key information that can distinguish targets from background clutter, but it will inevitably bring noise interference. Therefore, in high-frequency encoders, such as Figure 5 As shown in the figure, in the adaptive wavelet attention mechanism, the input high-frequency component infrared image feature U is first decomposed into four sub-components, including the low-frequency component infrared image feature UA and three high-frequency component infrared image features UH, UV, and UD. The three high-frequency component infrared image features are used to guide the low-frequency component infrared image features to enhance the difference characteristics between the true and false target areas and reduce the noise interference caused by directly using the high-frequency components. Specifically, the low-frequency component infrared image feature UA is added element-by-element to the three high-frequency component infrared image features UH, UV, and UD to integrate information at different frequency levels. Subsequently, three different depth-wise separable convolutions are applied to process the added result to enhance the local spatial information and map it to three different feature spaces. The purpose of this step is to use the decomposed high-frequency signal to generate guiding information, so as to more effectively capture the difference relationship between local areas. The above process is shown in formula (4):

[0081]

[0082] in, Represents depthwise separable convolution, (UA+[UH,UV,UD]) refers to the addition of UA to the three components, X AH ,X AV,X AD Represents the variables in the three high-frequency directional feature spaces. In the three feature spaces, global average pooling (GAP) and global maximum pooling (GMP) are applied respectively to compress the data from different feature spaces along the channel direction. These two pooling operations can extract global statistical features and compress the data into low-dimensional feature vectors. Subsequently, these compressed feature vectors are spliced ​​to obtain the unique guiding feature maps in each high-frequency directional feature space. This step can aggregate the information in each feature space. Next, in order to fuse the data from the three different frequency feature spaces, it is considered that different feature spaces may provide guiding feature maps with different gains. Therefore, instead of simply adding and fusion the data in the three feature spaces, updateable parameters are introduced so that the data in each feature space can be adaptively fused. This adaptive fusion strategy allows the model to dynamically adjust the weights of each feature map according to the characteristics of the data and optimize the effect of feature fusion. The above process can be expressed by formulas (5) and (6):

[0083]

[0084]

[0085] In the above formula, P avg (·) and P max (·) represents GAP and GMP, Represents the guiding feature in each feature space k

[0086] Feature graph, where k∈{H,V,D}, f 1×1 Represents a convolution with a kernel of 1 used to change the number of channels. α, β, and χ represent updateable weight parameters used to dynamically adjust the weight ratio of the guiding feature maps output by each feature space to ensure that the fusion of the feature spaces provides optimal target-clutter background differentiation information. M represents the final attention image. Multiplying M by the decomposed low-frequency component UA strengthens the more distinctive signal weight in UA and weakens the influence of the noise component. Ultimately, the output of this module can be expressed as follows:

[0087] W O =UA⊙M+UA (7)

[0088] like Figure 6 As shown, the multi-layer difference relationship capture module adopts the transformer architecture. Given the adaptive wavelet attention mechanism of the i-th layer, the output result is W o ∈R ci,hi,wi , where ci, hi, wi represent the number of channels, height, and width of the feature map of layer i respectively. Using 1×1 convolution, we get I i ∈R 128,hi,wi, and then flatten it to become a sequence of data T i ∈R 128,hi*wi , the overall processing process of the multi-layer difference relationship capture module can be described as formula (8):

[0089]

[0090] Among them, S stands for self-attention layer, LN stands for linear normalization layer, and MLP stands for multi-layer perceptron. is the result of the self-attention module, which completes the global establishment of the target-background relationship within each layer. In this module, the cascade features of the corresponding features of the other three layers except the current layer are used to provide high-frequency feature details of different fine-grained levels. C represents the cross-attention layer, which performs matrix calculations on the local pixel area in the current scale feature map and the local pixel areas in the other three different scale feature maps, thereby establishing the relationship between them. i cros It is the output of the cross-attention module, which establishes the relationship between the local area in the current layer and the local areas in the other three different fine-grained feature layers, which helps the network learn the different fine-grained target and background features.

[0091] T i O It is the final output result of the multi-layer difference relationship capture module. After scaling, the output of the entire high-frequency encoder can be obtained.

[0092] The self-attention module aims to establish semantically distinct relationships between pixel regions within a single-resolution feature map. Based on the input of the current layer, the query (Q), key (K), and value (V) are calculated. Since four attention heads are used, attention relationships are solved independently in the space corresponding to each attention head. Finally, these results are concatenated along the channel dimension to obtain the differential feature relationships between regions within the single-resolution feature map.

[0093] When the original input data flows into this stage, it has already undergone a lot of computational processing. Unlike general targets, the pixel information of small targets and their background clutter is relatively sparse, and the pixel area occupied by a single unit is small, which is easy to be lost in the continuous calculation layer. If the target feature or background clutter feature is lost in a certain feature layer, performing internal attention only in that layer will result in invalid calculation. Therefore, the cross-attention module is introduced to establish a more complete level of global difference relationship, build long-distance dependencies between all feature layers in the entire model, enhance the perception of high-frequency components, and reconstruct the difference relationship between the target and background clutter. Specifically, the output of the internal attention module of the i-th layer is used as Q, and the cascade features of the remaining three layers are used as K and V. It should be noted that the composition of these sub-layers will change according to i. As shown in formula (9), when the sub-layer number is greater than i, the sub-layer is the output of the multi-layer difference relationship capture module, providing the difference features between the target and the background; when the sub-layer number is less than i, it is the input of the multi-layer difference relationship capture module, which can provide more original and valuable information:

[0094]

[0095] The mathematical expressions of the above Q, K, V processes are as follows:

[0096]

[0097]

[0098]

[0099] In the above formula, W q , W k , W v is the linear fully connected layer matrix. Since the attention matrix is ​​learned using a four-attention-head structure, This represents the query, key, and value corresponding to the λth head of the i-th layer. Then, correlation calculation is performed to obtain the output of the cross-attention layer. The calculation formula is shown in (13). Using the Softmax activation function, the difference relationship between different layers is established, and the model has a better ability to perceive the high-frequency components of the target-background clutter:

[0100]

[0101] like Figure 7As shown in Figure 1, the infrared small target detection effect comparison diagram of the method of the present invention and other methods for infrared scenes, in which the red circle is the real target, the box is the magnified detail display of the real target area, the blue circle is the false detection target, and the yellow circle is the missed detection target. Compared with the deep learning method, the traditional method PSTNN has more false alarm targets and insufficient reconstruction accuracy, and the detection quality is relatively poor. In the comparison of deep learning methods, whether it is for point targets or targets with specific shape features, the method of the present invention shows a better refined reconstruction capability than other methods. Specifically, as shown in image-(1), when processing "dark" feature targets, DNA-Net has false target detection, other learning-based methods have missed target detection, and only the method proposed in the present invention achieves effective detection. This is mainly due to the enhancement of scene intensity and target saliency by the low-frequency encoder, which effectively improves the "dark" nature of dark targets. The gain brought by the improvement of target saliency is also reflected in targets with specific shapes. As shown in image-(2,3,4), the target contour reconstructed by the method of the present invention is closer to GT. In particular, in image-(3), faced with dual targets that are extremely close in spatial distance, this method is the only deep learning method that can accurately segment them into two independent targets, fully demonstrating its superior refined segmentation capabilities. As shown in image-(5,6), in scenarios with false alarm target interference, the comparative method often suffers from the problems of missed detection of dim targets and misdetection of false alarm targets. However, the method proposed in this invention can effectively suppress background false alarm targets while accurately highlighting the real targets. This excellent performance is mainly attributed to the long-range dependency established by the high-frequency encoder, which significantly improves the network's global target relationship processing capabilities.

[0102] In addition to intuitive qualitative comparison, the present invention also uses two pixel-level image indicators: Intersection over Union (IoU) and Normalized Intersection over Union (nIoU), and two target-level indicators: Probability of Detection (Pd) and False-Alarm Rate (Fa) to objectively analyze the quantitative effect of the method in the present invention from a quantitative evaluation perspective. IoU is widely used in segmentation-based tasks. It is calculated by dividing the intersection of the actual pixel value and the predicted pixel value by their union. Its calculation unit is the specific target in each test image. It mainly represents the model's prediction level of the target contour fineness. nIoU is an indicator to better balance model-driven and deep learning methods. Its calculation unit is a specific image. Even if there are multiple targets in the image, they are normalized to one target. Pd represents the ratio of the number of correctly predicted targets to the total number of targets, which represents the prediction accuracy of the target position. Fa represents the ratio of the number of incorrectly predicted pixels to the total number of pixels in the image, which represents the model's prediction ability for falsely detected targets. The smaller this indicator is, the better. Table 1 shows the quantitative results. In general, thanks to the powerful learning ability of deep neural networks for features, the deep learning method is significantly higher than the model-driven method in all four indicators. This is because the infrared dataset contains challenging images with different signal-to-noise ratios, different clutter backgrounds, and different types of targets. The deep learning method has strong robustness to scene changes. In contrast, the model-driven method relies on manually designed features and will have a large number of false detections and missed detections when faced with diverse scenes. In addition, it can be observed that the improvement of pixel-level performance by the deep learning method is significantly higher than the improvement of target-level performance compared to the model-driven method. This is because traditional methods mainly focus on the overall positioning of the target rather than precise shape matching, so they perform poorly in pixel-level indicators. The method proposed in the present invention achieves the best performance in all indicators, significantly outperforming all compared methods. This fully demonstrates that the proposed method has outstanding advantages in terms of refined reconstruction of target contours and pixel-level target-background feature differentiation. As shown in Table 1, the image quality evaluation results.

[0103] Table 1

[0104]

[0105]

[0106] In summary, this paper implements a novel infrared small target detection and recognition technology that couples the data's unique characteristics with the method used. The model is designed using a deep learning model based on high- and low-frequency components. This model combines two encoders: a low-frequency encoder and a high-frequency encoder. This model leverages the complementary properties of the high- and low-frequency components generated by the wavelet transform, designing feature extraction modules tailored to their distinct data characteristics. Low-frequency components are characterized by strong target saliency, and a densely connected convolutional neural network architecture is used to process them to extract local features. Given the sparsity of small targets in images, traditional downsampling can lead to the loss of critical target information. To address this, wavelet and inverse wavelet transforms are introduced to replace traditional downsampling and upsampling operations, preserving fine-grained feature representations of the target region. For high-frequency components containing target-background contrast information, two modules are designed: an adaptive wavelet attention mechanism utilizes the high-frequency components to generate attention-guiding information, enhancing the ability to discern local features; and a multi-layer difference relationship capture module establishes long-range contextual associations on multi-scale feature maps, reconstructing the correlation between the target and background clutter for ultimate, accurate detection.

[0107] To verify the effectiveness of the present invention in detecting infrared small targets, this embodiment selects infrared images from different scenes as a test set and compares them with model-driven methods such as Top-Hat, Max-Median, WSLCM, TTLCM, IPI, PSTNN, and MSLSTIPT, as well as deep learning-based methods such as ALC-Net, ACM-Net, ISNet, UIU-Net, RDIAN, MTU-Net, AGPCNet, DNA-Net, and SCTransNet. GT is the labeled image required for training the model. Four infrared small target detection evaluation metrics are used for objective comparison: two pixel-level metrics: Intersection over Union (IoU) and Normalized Intersection over Union (nIoU), and two object-level metrics: Probability of Detection (Pd) and False-Alarm Rate (Fa).

[0108] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0109] The present invention is not limited to the embodiments described above. The above description of the specific embodiments is intended to describe and illustrate the technical solutions of the present invention. The above specific embodiments are merely illustrative and not restrictive. Without departing from the scope of the present invention and the scope of protection of the claims, those skilled in the art may make various specific modifications based on the teachings of the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. A method for detecting small infrared targets based on differential learning of high and low frequency components of wavelet, characterized in that: The method comprises the following steps: S1, obtaining a pair of low-frequency component infrared image features and high-frequency component infrared image features according to the original infrared image; S2, constructing an infrared small target detection model based on a dual-encoder feature extraction network, wherein the dual-encoder feature extraction network includes a low-frequency encoder and a high-frequency encoder coupled with input data; S3, downsampling the low-frequency component infrared image features and the high-frequency component infrared image features obtained in S1 using a wavelet transform mechanism through a dual-encoder feature extraction network; S4, using the low-frequency component image features as input to learn the features of dim small infrared targets through the low-frequency encoder; using the high-frequency component infrared image features as input to learn the difference features between the target and background clutter, so as to achieve effective distinction between the small target and background clutter; In step S5, the characteristics of the dim infrared small target and the difference characteristics of the target-background clutter learned in step S4 are reconstructed and fused to train the infrared small target detection model. The infrared small target detection is realized through the trained infrared small target detection model, and the final prediction result is output.

2. The infrared small target detection method based on wavelet high and low frequency component difference learning according to claim 1 is characterized in that: Wherein S1 further includes that the low-frequency component infrared image feature is a significant local area target feature, the high-frequency component infrared image feature is a target-background clutter information with difference, and the high-frequency component infrared image feature is composed of three high-frequency components H, V, and D spliced ​​along the image channel dimension.

3. The infrared small target detection method based on wavelet high and low frequency component difference learning according to claim 1 is characterized in that: S3 further includes downsampling to obtain the downsampled low-frequency component infrared image features and low-frequency component infrared image features with only half the size of the original input image. After the feature learning is completed, the low-frequency component and the high-frequency component are reconstructed using inverse wavelet to obtain the original input image, realizing lossless downsampling and upsampling.

4. The infrared small target detection method based on wavelet high and low frequency component difference learning according to claim 1 is characterized in that: The low-frequency encoder in S2 is a dense feature pyramid structure, including a basic wavelet feature extraction unit and a dense wavelet feature extraction unit. The vertex of each layer of the dense feature pyramid structure is a basic wavelet feature extraction unit, and each layer of the pyramid is a series of dense wavelet feature extraction units; wherein the basic wavelet feature extraction unit extracts features from the original input image to obtain input data of different scales, and the input data is fused with low-scale basic wavelet feature features and high-scale dense wavelet features through multiple layers of dense wavelet feature extraction units to perform feature learning and obtain different receptive field features.

5. The infrared small target detection method based on wavelet high and low frequency component difference learning according to claim 4 is characterized in that: The basic wavelet feature extraction unit is a residual block structure. The input data first enters the basic wavelet feature extraction unit. In each basic wavelet feature extraction unit, the low-frequency component infrared image feature A and the high-frequency component infrared image feature U are both input and output feature learning through the residual block with a convolution layer, and the basic wavelet feature is output. It further includes the low-frequency component infrared image feature after residual processing. and high-frequency component infrared image features After the current node is processed, the low-frequency component will be further down-sampled through wavelet transform to obtain low-frequency and high-frequency components after further dimensionality reduction. These two components will serve as the input of the next node.

6. The infrared small target detection method based on wavelet high and low frequency component difference learning according to claim 4 is characterized in that: The dense wavelet feature extraction unit is a residual block structure, and its input includes the high / low frequency component image features decomposed from the previous node and the high / low frequency component image features decomposed from the adjacent node, and the input is fused. The input A of the dense wavelet feature extraction unit is ij The expression is as follows: Among them, i and j represent the horizontal and vertical coordinates of the node position, that is, the node at the i-th row and j-th column position, Ψ represents the cascade fusion of multiple features at the same layer, DWT represents wavelet transform, that is, downsampling the input data using wavelet transform, IDWT represents inverse wavelet transform, that is, upsampling the component data through inverse wavelet transform to obtain the low-frequency component data before decomposition, μ represents the conventional upsampling operation, [1] represents the component with index 1 generated by wavelet transform, that is, the low-frequency component, [2] represents the component with index 2 generated by wavelet transform, that is, the high-frequency component, k represents the vertical coordinate 1~j-1, Represents the low-frequency component infrared image feature after residual processing of the node at position k in row i, Represents the low-frequency component infrared image feature after residual processing of the node at the i-1th row and jth column position, Represents the low-frequency component infrared image feature after residual processing of the node at the i+1th row and j-1th column position, Represents the high-frequency component infrared image feature of the node at the i+1th row and j-1th column after residual processing, Represents the high-frequency component infrared image features of the node at position k in row i after residual processing.

7. The infrared small target detection method based on wavelet high and low frequency component difference learning according to claim 1 is characterized in that: The high-frequency encoder includes an adaptive wavelet attention mechanism and a multi-layer difference relationship capture module.

8. The infrared small target detection method based on wavelet high and low frequency component difference learning according to claim 7 is characterized in that: In the adaptive wavelet attention mechanism, the input high-frequency component infrared image data U is decomposed into four sub-components - a low-frequency component UA and three high-frequency components UH, UV, and UD. The low-frequency component UA is added element-by-element with the three high-frequency components UH, UV, and UD. Three different depth-wise separable convolutions are applied to the added result to enhance the local spatial information and map it to three different feature spaces, as shown in the following formula: in, Represents depthwise separable convolution, (UA+[UH,UV,UD]) refers to the addition of UA to the three components, X AH ,X AV ,X AD Represents variables in three high-frequency directional feature spaces; Furthermore, in three different feature spaces, global average pooling and global maximum pooling are applied respectively to compress data from different feature spaces along the channel direction. Subsequently, the compressed feature vectors are concatenated to obtain unique guiding feature maps in each high-frequency direction feature space. An adaptive fusion strategy is introduced to allow the model to dynamically adjust the weights of each feature map according to the characteristics of the data to perform feature fusion optimization. The adaptive fusion strategy process is shown in the following formula: Among them, P avg (·) and P max (·) represents GAP and GMP, Represents the guiding feature map in each feature space k, f 1×1 represents the convolution with a kernel of 1 used to change the number of channels, α, β, χ represent updateable weight parameters, Sigmoid represents the normalization function, and M represents the final attention image; The output of the adaptive wavelet attention mechanism is shown as follows: W O =UA⊙M+UA。 9. The infrared small target detection method based on wavelet high and low frequency component difference learning according to claim 7 is characterized in that: The multi-layer difference relationship capture module is a Transformer network model, which further includes a self-attention module and a cross-attention module. The self-attention module is used to establish semantic difference associations between different pixel areas within a single-resolution feature map; the cross-attention module is used to establish global difference relationships, build long-distance dependencies between all feature layers in the entire model, enhance the perception of high-frequency components, and reconstruct the difference relationship between the target and background clutter; Given the adaptive wavelet attention mechanism output result W of layer i o ∈R ci,hi,wi , where ci, hi, wi represent the number of channels, height, and width of the feature map of layer i respectively; using 1×1 convolution, we get I i ∈R 128,hi,wi , and then flatten it to become a sequence of data T i ∈R 128,hi*wi ,The overall processing process of the multi-layer difference relationship capture module is shown in the following formula: T i self =LN(S(T i )+T i ) T i cros =LN(C(T i self ,T i cas )+T i self ) T i O =LN(MLP(T i cros )+T i cros ) Among them, S stands for self-attention layer, LN stands for linear normalization layer, MLP stands for multi-layer perceptron, T i self Represents the result of the self-attention module, T i cas represents the high-frequency component infrared image features of the cascade corresponding to the layers other than the current layer, C represents the cross-attention layer, T i cros Represents the output of the cross-attention module, T i O Represents the final output of the multi-layer difference relationship capture module; T i cas The composition is shown below: When the sub-layer number is greater than i, the sub-layer is the output of the multi-layer difference relationship capture module, providing the difference features between the target and the background; when the sub-layer number is less than i, the sub-layer is the input of the multi-layer difference relationship capture module.

10. The infrared small target detection method based on wavelet high and low frequency component difference learning according to claim 9 is characterized in that: The query, key, and value in the cross-attention module are as follows: In the above formula, W q , W k , W v Represents the linear fully connected layer matrix, using a four-attention-head structure to learn the attention matrix. represents the query, key, and value corresponding to the λth head of the i-th layer; Then the correlation calculation is performed to obtain the output of the cross-attention layer, as shown in the following formula: The final output result is scaled to obtain the output of the encoder, and the original infrared image features are reconstructed in the decoder.

Images