Infrared small target detection method based on wavelet decomposition and manifold learning

Abstract

The invention discloses an infrared small target detection method based on wavelet transform and manifold learning, and the method comprises the steps: decomposing an input infrared image into a low-frequency component and a high-frequency component through Haar wavelet transform (HDWT), and designing a special processing module for different components: carrying out the cross-sub-band feature fusion and U-shaped convolutional network learning of the low-frequency component through a low-frequency encoder, and carrying out the feature fusion of the high-frequency component; background structure modeling and target-background distinguishing are enhanced; for a high-frequency component, through non-local feature extraction and parameterized manifold learning, noise interference is suppressed while a target detail topological structure is kept; meanwhile, an attention mechanism is designed, the coding layer space saliency and the decoding layer channel saliency are fused, and the attention on a small target area is improved. According to the method, the frequency domain characteristics are combined, rich and highly distinguished characteristics are jointly extracted, the detection precision is remarkably improved, and the false alarm rate is reduced.

Description

Technical Field

[0001] This invention relates to the field of infrared imaging detection and recognition technology, and in particular to an infrared small target detection method based on wavelet decomposition and manifold learning. Background Technology

[0002] Infrared small target detection, as a task for identifying faint targets in complex backgrounds, has important applications in scenarios such as maritime rescue and border / coastal defense target monitoring. With the development of deep learning technology, infrared small target detection methods have gradually evolved from traditional model-driven modeling methods to deep learning methods. The detection performance of deep learning methods is generally superior to that of traditional methods. Building a detection model based on deep learning methods is a technical problem that this invention urgently needs to solve. The infrared small target detection task differs from general target detection tasks in the visible light band. Due to the unique characteristics of infrared imaging equipment and the small targets themselves, there are still some problems to be solved in long-distance small target monitoring. For example, 1) Target-background confusion: Most models currently used in infrared small target detection tasks use U-Net as the baseline method. The traditional U-Net method extracts features entirely in the spatial domain, ignoring the frequency differences between low-frequency background structures, high-frequency target details, high-frequency noise, and other image components. It performs feature mining entirely in a spatial domain with mixed high and low frequency components, making it difficult to achieve effective separation of the target and the background. 2) Targets exhibit dim characteristics: Due to atmospheric scattering and absorption at long distances, as well as the target's own radiation capabilities, small targets exhibit dim characteristics and lack salience. The amount of learnable detailed features of the target is low, increasing the difficulty for the network to capture target information. 3) Background clutter and noise interference: Due to the influence of imaging equipment and environmental factors, background clutter and inherent noise will be generated in infrared images. Since most small targets lack geometric shapes and textures, these interferences generally present a similarity to real small targets.

[0003] Exploring suitable methods to solve the above problems is key to improving the performance of infrared small target detection tasks. 1) Low-frequency structure coding mechanism: Infrared small target detection is actually a target-background segmentation task, which requires clarifying the differences between target details and background structural features. After wavelet transform, the background structural information is often low-frequency and directionally complementary. Fusion processing of these complementary low-frequency subbands can obtain more complete structural information that is not contaminated by high-frequency interference, thereby achieving accurate background modeling and enhancing the accuracy of segmentation; 2) Improving target detail captureability: The high-frequency subbands present a large amount of noise mixing, and the target area is dark, limiting the amount of extractable detailed features, making direct local feature learning of high-frequency feature maps inefficient. Non-local cross-channel information extraction of high-frequency features can obtain high-dimensional features containing more target detail information, laying the foundation for improving the saliency of target features in the future; 3) Preserving the inherent topological relationship of detail information: A large amount of noise in the high-frequency subbands is mixed with rich target details and is difficult to distinguish. Manifold learning-based methods can suppress noise interference while preserving the inherent topological structure of target details, thereby obtaining enhanced target detail representations and accurately refining the segmentation of small targets. Summary of the Invention

[0004] To address the shortcomings of the existing technologies, this invention proposes an infrared small target detection method based on wavelet transform and manifold learning, which more completely characterizes the structural features of the low-frequency part of infrared images, accurately extracts target details in the high-frequency part while suppressing noise, and achieves high-precision infrared small target detection.

[0005] To achieve the above-mentioned objectives, the present invention proposes the following technical solution:

[0006] This invention proposes an infrared small target detection method based on wavelet transform and manifold learning, comprising the following steps:

[0007] Receive raw infrared images;

[0008] A detection model is constructed using a feature extraction network based on wavelet transform and manifold learning. The original infrared image is input into the detection model. This network includes an encoding layer, a bottleneck layer, and a decoding layer. The encoding layer performs multi-level, downsampled feature extraction on the input image. The bottleneck layer, located between the encoding and decoding layers, fuses and compresses the features output from the encoding layer after high- and low-frequency differentiation processing. The decoding layer upsamples and restores the spatial resolution of the features output from the bottleneck layer, gradually reconstructing the precise location and shape of the target.

[0009] The original infrared image is decomposed by wavelet decomposition through the coding layer to obtain low-frequency sub-bands and high-frequency sub-bands, and features are extracted by low-frequency encoder and high-frequency encoder respectively. The low-frequency encoder extracts features from the low-frequency sub-band to obtain frequency structure features that distinguish the target from the background, and the high-frequency encoder extracts non-local features from the high-frequency sub-band to obtain high-dimensional detail features.

[0010] The bottleneck layer fuses and compresses the processed low-frequency structural features and high-frequency detail features to generate a feature representation rich in multi-scale semantic information.

[0011] The high-dimensional detailed features are input into the manifold learning module, and through parameterized manifold learning, they are mapped into a low-dimensional representation that preserves the inherent topological structure of the data, so as to suppress background noise interference and enhance the target edge contour information.

[0012] An attention mechanism is constructed, which utilizes the spatial saliency of the encoding layer and the channel semantic feature saliency of the decoding layer to enhance the network's attention to small target regions;

[0013] The target features are reconstructed and segmentation maps are generated using the decoding layer. The spatial resolution of the feature map is gradually restored through upsampling operations, and the coding layer features modulated by the attention mechanism are fused to output the final infrared small target segmentation probability map.

[0014] In some implementations, the coding layer adopts an architecture that integrates Haar discrete wavelet transform and convolutional neural network, and the specific processing flow is as follows:

[0015] Each coding layer begins with the input image. The image is first decomposed into low-frequency and high-frequency sub-bands by Haar discrete wavelet transform. Then, the high-frequency and low-frequency sub-bands are processed separately, and the results are reconstructed by inverse Haar discrete wavelet transform. The reconstructed features are then refined by a CBR convolution operation. Finally, the convolution result is added element-wise to the original input image to form the output of the coding layer.

[0016] In some implementations, the decoding layer adopts an architecture that integrates Haar discrete wavelet transform and convolutional neural network, and the specific processing flow is as follows:

[0017] Each decoding layer begins with input features, which are first decomposed into low-frequency and high-frequency components by Haar discrete wavelet transform. Low-frequency subband injection is optimized through low-frequency processing, while the processing of high-frequency components is replaced by a parallel CBR convolution operation. The processed low-frequency components and the high-frequency components obtained by convolution are reconstructed by inverse Haar discrete wavelet transform, and then fused by another CBR convolution operation. The fused result is added to the input features of the decoding layer to complete the decoding layer operation.

[0018] Furthermore, the lower decoding layer receives the output from the upper decoding layer, which is upsampled to restore spatial resolution. Simultaneously, the features are combined with the weights provided by the attention mechanism and used as input to the lower decoding layer. After multiple decoding layers, the network's final output is finely adjusted sequentially through CBR convolution, 5×5 convolution, and 1×1 convolution, and the final segmentation result is generated by the Sigmoid activation function.

[0019] In some implementations, the output of the bottleneck layer is concatenated with the output of a certain decoding layer between the encoder and the decoder, thereby achieving cross-path fusion of deep semantic information and shallow detail information, providing rich features for the final accurate segmentation.

[0020] In some implementations, the wavelet decomposition is achieved using the Haar discrete wavelet transform, specifically including:

[0021] Original infrared image Decomposed into three low-frequency subbands of infrared image , , and a high-frequency subband The transformation relationship is defined by the following formula:

[0022] ;

[0023] And , , The three low-frequency sub-bands are stitched together along the image channel dimension to form a comprehensive low-frequency feature. .

[0024] In some embodiments, the feature extraction of the low-frequency subband by the low-frequency encoder includes:

[0025] Comprehensive low-frequency characteristics Perform 1×1 convolution to fuse cross-channel information and obtain the fused low-frequency features. As shown below:

[0026] ;

[0027] Low frequency characteristics The input is fed into a shallow U-Net for deep semantic extraction, and the output is low-frequency features of deep semantics. As shown below:

[0028] ;

[0029] Then, the deep semantic low-frequency features After 1×1 convolution and channel splitting, the optimized low-frequency subband is obtained. , , As shown below:

[0030] ;

[0031] in, Indicates a channel expansion operation. This indicates that the shallow U-shaped convolutional neural network U-Net extracts multi-level features from infrared images and performs accurate target segmentation. This represents a 1×1 convolution operation.

[0032] In some implementations, the nonlocal feature extraction and manifold learning of the high-frequency encoder include:

[0033] Input high-frequency subband First, downsampling is performed to obtain high-frequency features. ,right The process of performing cross-channel nonlocal feature extraction on high-dimensional datasets is as follows:

[0034] ;

[0035] in, , , , , and These represent the batch size of the original input images. Indicates for the first For any given pixel, high-dimensional data is constructed by taking all the pixels from all channels within a 3×3 spatial range around it. Represents a high-dimensional data set. This represents the total number of high-dimensional data points that need to be processed.

[0036] In some implementations, the manifold learning module for high-frequency features further includes:

[0037] For high-dimensional data Perform k-nearest neighbor (kNN) search to construct a neighbor set and calculate the minimum Euclidean distance between high-dimensional data. and conditional probability Construct a sparse similarity matrix between high-dimensional data. And symmetricize; define the set of positive neighbor pairs. and negative sample pairs set High-dimensional data is projected onto low-dimensional data using a multilayer perceptron (MLP) to obtain the dimension set of the low-dimensional data. Calculate low-dimensional similarity Calculate manifold embedding loss based on positive and negative sample pairs And integrate it into the overall loss function, as shown below:

[0038] ;

[0039] After manifold embedding, the low-dimensional data is reshaped and processed to obtain the optimized high-frequency subband. .

[0040] In some implementations, the attention mechanism further includes:

[0041] Constructing Spatial Attention: From the Output of the Encoding Layer The spatial attention weights are obtained by performing average pooling and max pooling along the channel dimension, concatenating the segments, and then applying a 7×7 convolution followed by sigmoid normalization. Modulation coding layer features;

[0042] Constructing Channel Attention: From the Decoding Layer Output The channel attention weights are obtained by taking global average pooling and max pooling, processing them through MLP, summing them, and then normalizing. Modulation decoding layer features;

[0043] Finally, the modulated features from the coding and decoding layers are fused together and used as the input for the next decoding layer.

[0044] In some implementations, the overall loss function of the method is a weighted average of the segmentation loss and the manifold embedding loss, as shown below:

[0045] ;

[0046] in, Indicates network segmentation loss. Represents the manifold embedding loss, This represents the adjustment factor for the weights between the balancing segmentation task and feature manifold learning.

[0047] Compared with the prior art, the beneficial effects of the technical solution of the present invention are as follows:

[0048] 1. The input data is decomposed into low-frequency components that better highlight background structure information and high-frequency components that reveal target details. Pre-separation of the signal is achieved through a wavelet decomposition module, followed by differentiated processing by two dedicated pipelines: a low-frequency feature processing module and a high-frequency feature processing module. Finally, information integration and refinement are achieved through a feature fusion and decoding module and an attention enhancement module. Each module performs its specific function while working collaboratively, addressing the current problems in infrared small target detection tasks such as "easily confused target and background clutter" and "weak target."

[0049] 2. Improvements brought about by the structural design of the low-frequency encoder include: leveraging the feature fusion advantages of convolutional neural networks and the semantic feature extraction capabilities of U-Net to learn more significant background structural features in the low-frequency subbands. By fusing, extracting, and then decomposing the three low-frequency subbands, more complete low-frequency features with enhanced background semantic features are obtained. Subsequently, inverse wavelet transform is used to obtain a feature map with accurate and clear background structural information, achieving more accurate target-background separation.

[0050] 3. Non-local feature extraction across channels in high-frequency subbands can yield richer target detail features. High-frequency subbands are mixed with noise and clutter. Non-local feature extraction can merge the detailed features scattered in each channel, thus providing a good foundation for subsequent target detail enhancement.

[0051] 4. Improvements brought about by the structural design of the high-frequency encoder include: innovatively combining the advantages of wavelet transform frequency domain analysis with the feature purification capability of manifold learning, significantly improving the performance of detecting weak and small targets in complex noise backgrounds through a unique high-frequency optimization path; using the manifold learning method, the inherent topological relationship of high-dimensional detail features can be naturally preserved to obtain low-dimensional manifold embedding, so that the target details are preserved while reducing the interference caused by high-frequency noise, which helps to accurately segment the target shape details.

[0052] 5. The attention mechanism captures spatially salient regions of the encoding layer and semantically salient channels of the decoding layer, guiding the focus on the target region during subsequent decoding. This allows attention to be more accurately focused on the target location and details, making the network's feature mining of small targets more accurate and reliable. Attached Figure Description

[0053] Figure 1 This is a flowchart of an infrared small target detection method based on wavelet transform and manifold learning according to the present invention;

[0054] Figure 2 This is a schematic diagram illustrating the difference between high and low frequency subbands in wavelet transform according to an embodiment of the present invention.

[0055] Figure 3 This is a schematic diagram of the infrared small target detection model structure according to an embodiment of the present invention;

[0056] Figure 4 This is a schematic diagram of the structure of a low-frequency encoder according to an embodiment of the present invention;

[0057] Figure 5 This is a schematic diagram of the structure of a high-frequency encoder according to an embodiment of the present invention;

[0058] Figure 6 This is a schematic diagram of the attention mechanism in an embodiment of the present invention;

[0059] Figure 7 This is a comparison chart showing the detection effects of the present invention and other methods on infrared images in different scenarios. Detailed Implementation

[0060] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0061] To verify the effectiveness of this invention in infrared small target detection, this embodiment selects infrared images from different scenes as the test set and compares them with model-driven methods such as Max-Median, Top-Hat, IPI, PSTNN, TTLCM, MSLSTIPT, and WSLCM, as well as deep learning-based methods such as ACM, ALCNet, DNANet, UIU-Net, RDIAN, MTU-Net, SeRankDet, SCTransNet, and GCLNet. GT is the labeled image required for training the model. Five infrared small target detection evaluation metrics are used, including three pixel-level metrics: Intersection over Union (IoU), normalized Intersection over Union (nIoU), and F-measure; and two target-level metrics: Probability of Detection (Pd) and False-Alarm Rate (Fa), for objective comparison.

[0062] Example: Figure 1 The diagram shows a flowchart of an infrared small target detection method based on wavelet transform and manifold learning in this embodiment. The flowchart specifically includes the following steps:

[0063] Step S1: Receive the raw infrared image by taking a real-time infrared camera picture of the area to be tested or by reading the raw infrared image dataset.

[0064] Step S2: The original infrared image is input into the detection network model. The model is implemented using a feature extraction network based on wavelet transform and manifold learning. This network includes an encoding layer, a bottleneck layer, and a decoding layer. The encoding layer is responsible for multi-level, downsampled feature extraction of the input image. The bottleneck layer, located between the encoding and decoding layers, is responsible for fusing and compressing the features output by the encoding layer after high- and low-frequency differentiation processing. It aggregates global contextual information through convolution operations to generate feature representations rich in multi-scale semantic information, providing the highest information density features for subsequent decoding and reconstruction. The decoding layer is responsible for upsampling and restoring spatial resolution of the features output by the bottleneck layer, gradually reconstructing the precise location and shape of the target. To compensate for the loss of spatial details caused by downsampling during encoding, the decoding layer receives multi-scale features modulated by an attention mechanism from the corresponding encoding layer through skip connections, achieving accurate pixel-level localization and finally outputting a segmentation map of the target.

[0065] Step S3: Perform wavelet decomposition on the original infrared image through the coding layer to obtain low-frequency component infrared image and high-frequency component infrared image;

[0066] In different spatial domain levels of the coding layer, the original infrared image is subjected to wavelet transform processing to obtain a low-frequency component infrared image containing significant structural features, and a high-frequency component infrared image containing significant background noise interference and target detail contours; such as Figure 2 The diagram shown illustrates the difference between high and low frequency subbands in wavelet transform according to an embodiment of the present invention; specifically, this step further includes:

[0067] Capture raw infrared images Then, the two-dimensional Haar discrete wavelet transform formula (1) is used for processing to obtain the low-frequency component infrared image as the low-frequency subband. , , and high-frequency component infrared images as high-frequency subbands The data is divided into a training set, a validation set, and a test set.

[0068] (1);

[0069] Step 201: Perform low-frequency feature fusion: three low-frequency molecular bands , , By stitching along the image channel dimension, a new low-frequency component infrared image is formed as a combined feature. ,in This represents a splicing operation along the channel dimension;

[0070] Step 202: In the subsequent network flow, the low-frequency component combination features are used... This indicates that it contains more significant low-frequency structural information, while the high-frequency component infrared data is used... This indicates that it contains information such as target details and high-frequency interference.

[0071] Step 203: The experimental data is divided into three parts: 70% of the dataset is used as the training set to train the infrared small target detection model; 20% of the dataset is used as the validation set to verify the model's performance during the training phase; and 10% of the dataset is used as the test set to subjectively visualize the model's detection performance.

[0072] Step 204: Construct an infrared small target detection network based on wavelet transform and manifold learning, such as... Figure 3 As shown, the network comprises an encoding layer, a bottleneck layer, and a decoding layer. The encoding layer adopts an architecture that integrates Haar discrete wavelet transform and convolutional neural network. The specific processing flow is as follows:

[0073] Each coding layer begins with the input image. The image is first decomposed into low-frequency and high-frequency sub-bands by Haar discrete wavelet transform. Then, the high-frequency and low-frequency sub-bands are processed separately, and the results are reconstructed by inverse Haar discrete wavelet transform. The reconstructed features are then refined by a CBR convolution operation. Finally, the convolution result is added element-wise to the original input image to form the output of the coding layer.

[0074] The decoding layer adopts an architecture that integrates Haar discrete wavelet transform and convolutional neural network. The specific processing flow is as follows:

[0075] Each decoding layer begins with input features, which are first decomposed into low-frequency and high-frequency components by Haar discrete wavelet transform. Low-frequency subband injection is optimized through low-frequency processing, while the processing of high-frequency components is replaced by a parallel CBR convolution operation. The processed low-frequency components and the high-frequency components obtained by convolution are reconstructed by inverse Haar discrete wavelet transform, and then fused by another CBR convolution operation. The fused result is added to the input features of the decoding layer to complete the decoding layer operation.

[0076] Furthermore, the lower decoding layer receives the output from the upper decoding layer, which is upsampled to restore spatial resolution. Simultaneously, the features are combined with the weights provided by the attention mechanism and used as input to the lower decoding layer. After multiple decoding layers, the network's final output is finely adjusted sequentially through CBR convolution, 5×5 convolution, and 1×1 convolution, and the final segmentation result is generated by the Sigmoid activation function.

[0077] Between the encoder and decoder, the output of the bottleneck layer is concatenated with the output of a certain decoding layer, thereby achieving cross-path fusion of deep semantic information and shallow detail information, providing rich features for the final accurate segmentation.

[0078] Specifically, a low-frequency structural information detection model for infrared small target images is constructed in the coding layer. This model includes a low-frequency encoder and a high-frequency encoder. The low-frequency encoder uses three semantically complementary low-frequency sub-bands obtained after wavelet transform as input, and extracts features from these three semantically complementary low-frequency sub-bands through its internally designed convolutional neural network to learn low-frequency structural features, thereby better distinguishing the target from the background. The high-frequency encoder uses the high-frequency sub-bands as input and performs non-local feature extraction on them. By fusing data across multiple channels, a high-dimensional feature data set is formed to obtain a high-dimensional representation containing more details of the infrared small target, serving as high-dimensional detail features. This model is based on a cascaded processing module of wavelet transform and U-Net (SubUNet), employing an end-to-end encoder-decoder neural network architecture. Its core innovation lies in introducing wavelet transform to decompose the input image into high-frequency and low-frequency components in the early stages, and designing two differentiated parallel processing paths based on their characteristics. Finally, an attention mechanism is used to fuse these components, achieving accurate segmentation of infrared small targets.

[0079] The detailed process is broken down step by step as follows:

[0080] 1. Input and Wavelet Decomposition

[0081] Input: An infrared image with a size of 256×256.

[0082] Preprocessing: The input image first passes through a 3×3 convolutional layer for preliminary feature extraction and channel adjustment.

[0083] Core decomposition: The preprocessed features are fed into the Haar Discrete Wavelet Transform (HDWT) module. This module decomposes the image into four sub-bands:

[0084] Low-frequency subband ( , , ): Contains the main structure and background information of the image.

[0085] High-frequency subband ( ): Contains detailed information such as the edges, texture, and noise of the image.

[0086] Component Recombination: Three Low-Frequency Subbands ( , , The high-frequency subbands are concatted along the channel dimension to form a composite low-frequency characteristic. ) are then processed separately.

[0087] 2. Differentiated processing paths for high / low frequencies, with two paths processing in parallel:

[0088] Low-frequency processing path:

[0089] The spliced ​​low-frequency features are first fused across channels using a 1×1 convolution.

[0090] Subsequently, the data is fed into a lightweight U-Net network (SubUNet) for in-depth background structure modeling and target semantic information extraction.

[0091] The processed features are then 1×1 convolutioned and split (Chuncked) back into three optimized low-frequency subbands.

[0092] High-frequency processing path:

[0093] High frequency components ( First, it goes through a convolutional and downsampling layer.

[0094] The core manifold learning module then takes place. This module first constructs a high-dimensional feature set through nonlocal operations, and then uses parametric manifold learning (including steps such as k-nearest neighbor graph construction, positive and negative sample pair definition, and multilayer perceptron (MLP) projection) to map it to a low-dimensional space, thereby preserving and enhancing the detailed contours of the target while suppressing noise.

[0095] The low-dimensional representation obtained from manifold learning is upsampled and convolutionally processed to obtain the optimized high-frequency subband.

[0096] 3. Wavelet Reconstruction and Encoder-Decoder Backbone:

[0097] Reconstruction: The optimized high and low frequency subbands are reconstructed using the inverse Haar discrete wavelet transform (IDWT) to obtain a feature map that integrates enhanced structural and detail information.

[0098] Encoder: The reconstructed features are used as input to a standard encoder (backbone network). The encoder gradually increases the receptive field by downsampling multiple times (labeled "downsampling" and size changes, such as 256→128→64→32) to extract multi-level features.

[0099] Bottleneck layer: The bottleneck layer further expands the receptive field of feature extraction by introducing dilated convolution in the low-frequency encoder, thereby obtaining more abstract semantic information.

[0100] Decoder: The decoder gradually restores the spatial resolution by upsampling multiple times (marked with "upsampling" and size changes, such as 32→64→128→256), and finally outputs a segmentation map with the same size as the input.

[0101] 4. Fusion of attention mechanisms:

[0102] To accurately locate small targets, the network introduces an attention mechanism that connects corresponding layers of the encoder and decoder.

[0103] Specific operation: Features extracted from a certain layer of the encoder are modulated by the spatial attention module and the channel attention module, respectively.

[0104] The modulated features are then added element-wise to the features of the previous layer of the decoder, thereby guiding the decoder to better focus on the regions and features related to the target during the upsampling process.

[0105] 5. Output and Loss Function:

[0106] Output: The final output of the decoder is passed through a Sigmoid activation function to generate a probability map, where the value of each pixel represents the confidence that the location belongs to the target.

[0107] Loss function: The overall loss of the network consists of two parts:

[0108] Segmentation loss The loss metric measures the difference between the output segmentation map and the true label, and is the main loss of the segmentation task.

[0109] Manifold loss It is specifically designed to constrain and optimize the manifold learning process in high-frequency paths, ensuring that it can effectively learn useful detailed representations.

[0110] Therefore, this network model achieves signal separation in the spatial and frequency domains through wavelet transform, processes background structure (low frequency) and target details (high frequency) through differentiated paths, refines detailed features in low-dimensional space through manifold learning, and achieves accurate fusion of multi-scale features through attention mechanism. Finally, through their collaboration, they achieve accurate and robust detection of weak and small infrared targets in complex backgrounds.

[0111] Step 3: Construct a low-frequency structural information detection model for infrared small target images. This model includes a low-frequency encoder and a high-frequency encoder. The low-frequency encoder takes three semantically complementary low-frequency sub-bands obtained after wavelet transform as input and extracts features from the three semantically complementary low-frequency sub-bands through its internally designed convolutional neural network to learn low-frequency structural features, thereby better distinguishing the target from the background. The network uses the low-frequency encoder to fuse and process the three complementary low-frequency sub-bands, obtaining more complete low-frequency structural information while using SubUNet to obtain more semantically characteristic background structure modeling.

[0112] The high-frequency encoder takes the high-frequency sub-band as input, performs non-local feature extraction on the high-frequency sub-band, and forms a high-dimensional feature data set by fusing data across multiple channels to obtain a high-dimensional representation containing more details of infrared small targets, as a high-dimensional detail feature.

[0113] Step 301, as follows Figure 4 The image shown is a low-frequency encoder according to an embodiment of the present invention, and is described in detail below:

[0114] Low-frequency feature fusion: three low-frequency sub-bands , , Concatenate the data along the channel dimension to form a combined feature. Subsequently, a 1×1 convolutional layer is applied. Perform cross-channel information fusion, reduce dimensionality, and output features. .

[0115] Deep feature extraction: fused features The data is fed into a shallow U-Net network, SubUNet, which, through its encoding and decoding structure (including 3×3 convolutions, normalization, ReLU activation, etc.) and skip connections (element-wise addition), extracts and optimizes deep semantic information from low-frequency features, outputting enhanced features. .

[0116] Subband recombination and reconstruction: Enhanced features After undergoing another 1×1 convolution for channel adjustment, it is then split (Chuncked) into three independent, optimized low-frequency subbands. , , Finally, these three optimized subbands were compared with the unprocessed high-frequency subband. The two components are reconstructed together using the Inverse Haar Discrete Wavelet Transform (IHDWT) to obtain the final output Y.

[0117] Low frequency characteristics After being input into the low-frequency encoder, the signal undergoes a 1×1 convolution layer to fuse cross-channel information. This reduces the number of channels to one-third of the original number, allowing complementary low-frequency information from different sub-bands to interact and obtain the fused low-frequency features. Low-frequency characteristics after fusion It can be expressed using formula (2);

[0118] (2);

[0119] Step 302: Combine the low-frequency features with rich low-frequency information. The input is fed into a shallow U-Net shallow U-shaped convolutional neural network, utilizing its... The components obtain features with deeper semantic information. This makes the acquired background structure information richer and more accurate. The dilated convolution introduced in the aforementioned bottleneck layer is applied to the low-frequency feature extraction module of this layer. In the middle. Features with deep semantic information It can be expressed using formula (3);

[0120] (3);

[0121] Step 303, Features The number of channels is expanded by 1×1 convolution, so that the number of output channels is proportional to the characteristics of the input low-frequency encoder. To maintain consistency, the features are further divided into three optimized low-frequency sub-bands along the channel direction. , , This process can be expressed by formula (4);

[0122] (4);

[0123] in, Indicates a channel expansion operation. This indicates that the shallow U-shaped convolutional neural network U-Net extracts multi-level features from infrared images and performs accurate target segmentation. This represents a 1×1 convolution operation.

[0124] Step 304: The high-frequency encoder performs non-local feature extraction on the high-frequency subband to obtain high-dimensional detail features;

[0125] High-frequency encoders for high-frequency sub-bands Non-local feature extraction is performed, and data spanning multiple channels is fused to form a high-dimensional representation containing more details of infrared small targets.

[0126] Step 3041, as follows Figure 5 The high-frequency encoder designed in this invention consists of nonlocal feature extraction, manifold learning, and convolutional layers.

[0127] Step 3042: The input data first enters the nonlocal feature extraction module, which is used to form high-dimensional features containing more detailed information, laying the foundation for subsequent manifold embedding.

[0128] Step 3043: Specifically, to improve the efficiency of the high-frequency feature extraction module, input... First, downsample to , , , and These represent the batch size, number of channels, and image dimensions of the original input image, respectively. After wavelet transform, the dimensions of the high-frequency subbands are reduced to half of their original values.

[0129] Step 3044: Input the high-frequency components of the high-frequency sub-band infrared image. First, downsampling is performed to obtain high-frequency features. ,right The process of performing cross-channel nonlocal feature extraction to obtain all these high-dimensional datasets can be represented by formula (5):

[0130] (5);

[0131] in, , , Indicates for the first For any given pixel, high-dimensional data is constructed by taking all the pixels from all channels within a 3×3 spatial range around it. Represents a high-dimensional data set. This represents the total number of high-dimensional data points that need to be processed.

[0132] Step 4: The processed low-frequency structural features and high-frequency detail features are fused and compressed through the bottleneck layer to generate a feature representation rich in multi-scale semantic information.

[0133] Step 5: Input the high-dimensional detailed features into the manifold learning module, and map them into a low-dimensional representation that preserves the inherent topological structure of the data through parameterized manifold learning, so as to suppress background noise interference and enhance the target edge contour information; including:

[0134] Manifold learning methods are used to process high-dimensional data containing a large amount of detailed target information. While preserving the inherent topological relationships of the target details, it suppresses high-frequency noise interference, achieving a fine and accurate characterization of small targets:

[0135] Step 501: The manifold learning method based on parameterized Uniform Manifold Approximation and Projection (UMAP) is appropriately optimized and adjusted to make it more suitable for convolutional neural network applications that process high-frequency subband target detail information.

[0136] Step 502: Locate each high-dimensional data point. We find the k-Nearest Neighbor (kNN) and construct the neighbor set accordingly. Here, the number of neighbors searched is defined as... The constructed neighbor set can be represented by formula (6);

[0137] (6);

[0138] Step 503: Calculate the minimum Euclidean distance between high-dimensional data. The calculation steps can be expressed by formula (7):

[0139] (7);

[0140] Based on this, the conditional probability between high-dimensional data is calculated. The conditional probability can be expressed by formula (8), where For trainable scaling parameters, used to scale conditional probabilities across high-dimensional data:

[0141] (8);

[0142] Step 504: Construct a sparse similarity matrix between high-dimensional data. The construction steps can be represented by equation (9):

[0143] (9);

[0144] The obtained matrix Symmetry transformation ensures that two high-dimensional data points are either neighbors or non-neighbors. The symmetry transformation operation can be represented by equation (10):

[0145] (10);

[0146] Step 505: Define the set of positive neighbor pairs. And obtain a set of negative sample pairs. The positive neighbor pairs are the matrix obtained in step 404. The two high-dimensional data pairs corresponding to values ​​greater than 0 in the middle. It can be expressed by equation (11):

[0147] (11);

[0148] The negative sample set is selected using a method of first random sampling and then filtering. First, samples equal to twice the number of target negative sample pairs are selected (here, the number of target negative samples is equal to the number of positive neighbor pairs). Consistent) as candidate sample pairs Then, a Boolean mask is used to remove data pairs that are already positive neighbor pairs and data pairs themselves (data pairs are definitely not negative samples of themselves) from the candidate pairs. Finally, the target number of data pairs are selected from the filtered candidate pairs as negative sample pairs. The construction method of the negative sample pair set can be represented by equation (12):

[0149] (12);

[0150] Step 506: Project the high-dimensional data onto low-dimensional data using a multilayer perceptron (MLP), where the dimension of each low-dimensional data point is... ,in The projection process can be represented by equation (13).

[0151] (13);

[0152] Step 507: Calculate the low-dimensional similarity based on the obtained low-dimensional data, and then calculate the manifold projection loss. The calculation of low-dimensional similarity can be expressed by equation (14), where the parameter controlling the shape of the similarity curve is... , .

[0153] (14);

[0154] To constrain the low-dimensional manifold embedding with positive neighbor pairs and negative sample pairs and preserve better topological structure, loss functions are constructed separately for positive neighbor pairs. and loss function for negative sample pairs . The method for constraining the local connectivity of positive neighbor pairs in low-dimensional embeddings and high-dimensional data to remain consistent can be expressed by equation (15):

[0155] (15)

[0156] and This is used to suppress the similarity between negative sample pairs in low-dimensional embeddings. The calculation method can be expressed by equation (16):

[0157] (16);

[0158] Finally and The sum is used as the loss for manifold embedding. and will Joint optimization is performed on the loss function integrated into the overall detection network. This achieves higher target segmentation accuracy while preserving the inherent topological relationships of the data during manifold embedding, thereby enabling effective extraction of high-frequency detail information and suppression of high-frequency noise interference. Manifold embedding loss It can be expressed by equation (17).

[0159] (17);

[0160] Step 508: After manifold embedding, transform the size of the low-dimensional embedding data to... Then on By performing two convolutional layers and one upsampling process, the optimized high-frequency subband is obtained. , Dimensions and Maintain consistency. This process can be represented by equation (18), where Representing convolution, normalization, and activation processing:

[0161] (18);

[0162] Finally, the inverse Haar discrete wavelet transform was used. The output of the low-frequency feature extraction module , and and the output of the high-frequency feature extraction module Reconstruct the layer to obtain its final output. :

[0163] ;

[0164] in, The expression is:

[0165] ;

[0166] Finally, the overall loss function of the network can be expressed by equation (19), which is a weighted sum of the network segmentation loss and the manifold embedding loss:

[0167] (19);

[0168] Step 6: Construct an attention mechanism that leverages the spatial saliency of the encoding layer and the channel semantic feature saliency of the decoding layer to enhance the network's attention to small target regions; for example... Figure 6 As shown, the attention mechanism is divided into spatial attention for the encoding layer and channel attention for the decoding layer. By utilizing the spatial saliency of the encoding layer and the channel semantic feature saliency of the decoding layer, the network's attention to small target regions is enhanced, thereby achieving accurate target segmentation.

[0169] Step 601: Construct spatial attention, such as... Figure 6 As shown, attention weights for the spatial domain are obtained from the output of the network coding layer, which enhance the decoding layer's focus on salient regions of the target. The outputs of the decoding layer are then analyzed. Average of channel dimensions and maximum value , respectively obtained and This process can be represented by equation (20):

[0170] (20);

[0171] The feature maps obtained are then concatenated along the channel dimension, as shown in Equation (21), to obtain... :

[0172] (twenty one);

[0173] Perform convolution operation on the concatenated feature maps And normalized to the interval of 0-1, as shown in equation (22), the attention weights for the space are obtained. :

[0174] (twenty two);

[0175] The final coding layer information is weighted. Modulate using the operation as shown in equation (23) :

[0176] (twenty three);

[0177] Step 602: Construct attention for channels with significant semantic information. Obtain attention weights for each channel from the decoding layer output to strengthen the decoding layer's focus on channels with significant semantic information. As shown in Equation (24), perform global average pooling on each channel of the decoding layer output. and global max pooling , respectively obtained and :

[0178] (twenty four);

[0179] To each and The MLP operation is performed as shown in equation (25). The MLP consists of two fully connected layers. and And one Activation layer composition:

[0180] (25);

[0181] Subsequently, the obtained feature values ​​are summed and normalized, as shown in equation (26), to obtain the channel attention weights. , :

[0182] (26);

[0183] The final decoding layer information is weighted. Modulate using the operation as shown in equation (27) :

[0184] (27);

[0185] Step 603: Finally, the modulated coding layer features and decoding layer features As shown in equation (28), the output of the attention mechanism is obtained through fusion and used as the input of the next decoding layer. :

[0186] (28);

[0187] Step 7: Reconstruct the target features and generate a segmentation map using the decoding layer. Gradually restore the spatial resolution of the feature map through upsampling operations, and fuse the coding layer features modulated by the attention mechanism to output the final infrared small target segmentation probability map.

[0188] The above process further includes:

[0189] Step 8: Set the training hyperparameters, train the built model, load the network weight file, and train on a real infrared image dataset. In each layer of the main network coding layer, use the low-frequency module and high-frequency module mentioned in this invention to obtain low-frequency components that better reflect the target-background structure features and high-frequency components that better reflect the target details. Use the attention mechanism mentioned in this invention between the network coding layer and the decoding layer to fuse the outputs of each decoding layer to obtain the final prediction result.

[0190] This method obtains features in different frequency domains by decomposing the input data using wavelet decomposition, and processes the high and low frequency features separately. For the low-frequency part, which carries rich structural information, a low-frequency feature fusion module is applied to obtain a more complete and comprehensive structural representation. For the high-frequency part, which is rich in target details and noise interference, a cross-channel nonlocal information extraction module and a detail information preservation module based on manifold learning are introduced to accurately characterize target details while suppressing noise. The attention module guides the attention allocation of the decoding layer based on the salient regions of the target in the output features of the coding and decoding layers, thereby achieving high-precision infrared small target detection.

[0191] like Figure 7The image shows a comparison of the infrared small target detection performance of this embodiment with other methods in infrared scenes. The blue circle indicates the real target, the blue box shows magnified details of the real target area, the red circle indicates falsely detected targets, and the yellow circle indicates missed targets. The comparison methods include one traditional method, IPI, and four deep learning methods (ACM, UIU-Net, SCTransNet, and GCLNet). Whether for point targets or targets with specific shape features, the method of this invention demonstrates superior fine-grained reconstruction capabilities compared to other methods. Specifically, as shown in (1), (5), and (6), the method of this invention achieves accurate detection results in multiple scenarios, while other methods exhibit missed detections or false alarms. This is due to the low-frequency feature processing module utilizing complementary information between subbands to capture the complete structure, effectively distinguishing the target from the background. Meanwhile, the high-frequency feature processing module combines manifold learning to maintain the topological structure of target details, suppress noise, and reduce false alarms. As shown in Figures (2) and (4), under complex cloud layers and flat backgrounds, this method can maintain clear target edges and complete contours, while the comparison methods generally show blurred or missing edges, indicating that they are insufficient in capturing high-frequency details. This method can retain key target edge features while suppressing noise. As shown in Figure (3), in multi-target scenarios with close spatial distances, this method can accurately distinguish multiple targets, while other methods, due to their reliance on spatial features and neglect of video domain features, are prone to missed detections or confusion.

[0192] Beyond intuitive qualitative comparisons, this invention also employs a quantitative evaluation approach, utilizing three pixel-level image metrics: Intersection over Union (IoU), normalized Intersection over Union (nIoU), and F-measure; and two target-level metrics: Probability of Detection (Pd) and False-Alarm Rate (Fa) to objectively analyze the quantitative effectiveness of the method. IoU, widely used in segmentation-based tasks, is calculated by dividing the intersection of actual pixel values ​​and predicted pixel values ​​by their union. Its calculation unit is a specific target in each test image, primarily representing the model's prediction level of target contour refinement. nIoU is an metric designed to better balance model-driven and deep learning methods; its calculation unit is a specific image, normalizing multiple targets in an image to a single target. F-measure is a comprehensive performance metric considering both precision and recall. Pd represents the ratio of correctly predicted targets to the total number of targets, indicating the accuracy of target location prediction. Fa represents the ratio of the number of incorrectly predicted pixels to the total number of pixels in the image, indicating the model's ability to predict falsely detected targets; a smaller value is better. Table 1 presents the quantitative results. Overall, thanks to the powerful feature learning ability of deep neural networks, deep learning methods significantly outperform model-driven methods in all five metrics. This is because infrared datasets contain challenging images with different signal-to-noise ratios, different clutter backgrounds, and different types of targets. Deep learning methods exhibit strong robustness to scene changes, while model-driven methods rely on manually designed features, leading to a large number of false positives and false negatives when facing diverse scenes. Furthermore, it can be observed that deep learning methods significantly improve pixel-level performance compared to model-driven methods, but more significantly improve target-level performance. This is because traditional methods mainly focus on the overall localization of the target rather than precise shape matching, resulting in poor performance at the pixel level. The method proposed in this invention achieves optimal performance in all metrics, significantly outperforming all compared methods. This fully demonstrates the outstanding advantages of the proposed method in refined target contour reconstruction and pixel-level target-background feature differentiation. Table 1 shows the image quality evaluation results.

[0193] Table 1

[0194]

[0195] In summary, this invention realizes a novel infrared small target detection and recognition technology. It couples the advantages of data features with the method used, employing a deep learning model based on high and low frequency components. This model combines two encoders: a low-frequency encoder and a high-frequency encoder. It fully utilizes the characteristics of the high-frequency and low-frequency components generated by wavelet transform, designing feature extraction modules tailored to their different data features. The low-frequency components are characterized by directional complementarity and strong structural features; a convolutional neural network structure is used to fuse these components to extract structural features. For the high-frequency components, which contain target detail information and high-frequency interference, a non-local information extraction module and a manifold learning module are designed to enhance the ability to distinguish target detail features. An attention module strengthens the network's focus on the target region based on the regions with significant responses from the encoding and decoding layers, achieving ultimately accurate detection.

[0196] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0197] This invention is not limited to the embodiments described above. The above description of specific embodiments is intended to illustrate and explain the technical solutions of this invention. The specific embodiments described above are merely illustrative and not restrictive. Without departing from the spirit and scope of the claims, those skilled in the art can make many specific modifications based on the teachings of this invention, and these modifications all fall within the scope of protection of this invention.

Claims

1. A method for detecting small infrared targets based on wavelet transform and manifold learning, characterized in that, Includes the following steps: Receive raw infrared images; A detection model is constructed using a feature extraction network based on wavelet transform and manifold learning. The original infrared image is input into the detection model. The network includes an encoding layer, a bottleneck layer, and a decoding layer. The encoding layer performs multi-level, downsampled feature extraction on the input image. The bottleneck layer, located between the encoding and decoding layers, fuses and compresses the features output from the encoding layer after high- and low-frequency differentiation processing. The decoding layer upsamples and restores the spatial resolution of the features output from the bottleneck layer, gradually reconstructing the precise location and shape of the target. The original infrared image is decomposed by wavelet decomposition through the coding layer to obtain low-frequency sub-bands and high-frequency sub-bands, and features are extracted by low-frequency encoder and high-frequency encoder respectively. The low-frequency encoder extracts features from the low-frequency sub-bands to obtain low-frequency structural features that distinguish the target from the background, and the high-frequency encoder extracts non-local features from the high-frequency sub-bands to obtain high-dimensional detail features. The bottleneck layer fuses and compresses the processed low-frequency structural features and high-frequency detail features to generate a feature representation rich in multi-scale semantic information. The high-dimensional detailed features are input into the manifold learning module, and through parameterized manifold learning, they are mapped into a low-dimensional representation that preserves the inherent topological structure of the data, so as to suppress background noise interference and enhance the target edge contour information. An attention mechanism is constructed, which utilizes the spatial saliency of the encoding layer and the channel semantic feature saliency of the decoding layer to enhance the network's attention to small target regions; The target features are reconstructed and segmentation maps are generated using the decoding layer. The spatial resolution of the feature map is gradually restored through upsampling operations, and the coding layer features modulated by the attention mechanism are fused to output the final infrared small target segmentation probability map.

2. The infrared small target detection method based on wavelet transform and manifold learning as described in claim 1, characterized in that, The encoding layer adopts an architecture that integrates Haar discrete wavelet transform and convolutional neural network. The specific processing flow is as follows: Each coding layer begins with the input image, which is first decomposed into low-frequency and high-frequency sub-bands by Haar discrete wavelet transform. Then, the high-frequency and low-frequency sub-bands are processed separately, and the results are reconstructed by inverse Haar discrete wavelet transform. The reconstructed features are then refined by a CBR convolution operation. Finally, the convolution result is added element-wise to the original input image to form the output of the coding layer.

3. The infrared small target detection method based on wavelet transform and manifold learning as described in claim 1, characterized in that, The decoding layer adopts an architecture that integrates Haar discrete wavelet transform and convolutional neural network. The specific processing flow is as follows: Each decoding layer begins with input features, which are first decomposed into low-frequency and high-frequency components by Haar discrete wavelet transform. Low-frequency subband injection is optimized through low-frequency processing, while the processing of high-frequency components is replaced by a parallel CBR convolution operation. The processed low-frequency components and the high-frequency components obtained by convolution are reconstructed by inverse Haar discrete wavelet transform, and then fused by another CBR convolution operation. The fused result is added to the input features of the decoding layer to complete the decoding layer operation. Furthermore, the lower decoding layer receives the output from the upper decoding layer, which is upsampled to restore spatial resolution. Simultaneously, the features are combined with the weights provided by the attention mechanism and used as input to the lower decoding layer. After multiple decoding layers, the network's final output is finely adjusted sequentially through CBR convolution, 5×5 convolution, and 1×1 convolution, and the final segmentation result is generated by the Sigmoid activation function.

4. The infrared small target detection method based on wavelet transform and manifold learning as described in claim 1, characterized in that, Between the encoder and decoder, the output of the bottleneck layer is concatenated with the output of a certain decoding layer, thereby achieving cross-path fusion of deep semantic information and shallow detail information, providing rich features for the final accurate segmentation.

5. The infrared small target detection method based on wavelet transform and manifold learning as described in claim 1, characterized in that, The wavelet decomposition mentioned above is implemented using the Haar discrete wavelet transform, specifically including: Original infrared image Decomposed into three low-frequency subbands of infrared image , , and a high-frequency subband The transformation relationship is defined by the following formula: ； And , , The three low-frequency sub-bands are stitched together along the image channel dimension to form a comprehensive low-frequency feature. .

6. The infrared small target detection method based on wavelet transform and manifold learning as described in claim 1, characterized in that, The feature extraction of the low-frequency subband by the low-frequency encoder includes: Comprehensive low-frequency characteristics Perform 1×1 convolution to fuse cross-channel information and obtain the fused low-frequency features. As shown below: ； Low frequency characteristics The input is fed into a shallow U-Net for deep semantic extraction, and the output is low-frequency features of deep semantics. As shown below: ； Then, the deep semantic low-frequency features After 1×1 convolution and channel splitting, the optimized low-frequency subband is obtained. , , As shown below: ； in, This represents the channel expansion operation, indicating that the shallow U-shaped convolutional neural network U-Net extracts multi-level features from infrared images and performs accurate target segmentation. This represents a 1×1 convolution operation.

7. The infrared small target detection method based on wavelet transform and manifold learning as described in claim 1, characterized in that, The nonlocal feature extraction and manifold learning of the high-frequency encoder include: Input high-frequency subband First, downsampling is performed to obtain high-frequency features. ,right The process of performing cross-channel nonlocal feature extraction on high-dimensional datasets is as follows: ； in, , , , , and These represent the batch size of the original input images. Indicates for the first For any given pixel, high-dimensional data is constructed by taking all the pixels from all channels within a 3×3 spatial range around it. Represents a high-dimensional data set. This represents the total number of high-dimensional data points that need to be processed.

8. The infrared small target detection method based on wavelet transform and manifold learning as described in claim 1, characterized in that, The manifold learning module for high-frequency features further includes: For high-dimensional data Perform k-nearest neighbor search to construct a neighbor set and calculate the minimum Euclidean distance between the high-dimensional data. and conditional probability Construct a sparse similarity matrix between high-dimensional data. And symmetricize; define the set of positive neighbor pairs. and negative sample pairs set High-dimensional data is projected onto low-dimensional data using a multilayer perceptron to obtain the dimension set of the low-dimensional data. Calculate low-dimensional similarity Calculate manifold embedding loss based on positive and negative sample pairs And integrate it into the overall loss function, as shown below: ； After manifold embedding, the low-dimensional data is reshaped and processed to obtain the optimized high-frequency subband. .

9. The infrared small target detection method based on wavelet transform and manifold learning as described in claim 1, characterized in that, The attention mechanism further includes: Constructing Spatial Attention: From the Output of the Encoding Layer The spatial attention weights are obtained by performing average pooling and max pooling along the channel dimension, concatenating the segments, and then applying a 7×7 convolution followed by sigmoid normalization. Modulation coding layer features; Constructing Channel Attention: From the Decoding Layer Output The channel attention weights are obtained by taking global average pooling and max pooling, processing them through MLP, summing them, and then normalizing. Modulation decoding layer features; Finally, the modulated features from the coding and decoding layers are fused together and used as the input for the next decoding layer.

10. The infrared small target detection method based on wavelet transform and manifold learning as described in claim 1, characterized in that, The overall loss function of the method is a weighted sum of the segmentation loss and the manifold embedding loss, as shown below: ； in, Indicates network segmentation loss. Represents the manifold embedding loss, This represents the adjustment factor for the weights between the balancing segmentation task and feature manifold learning.

Images