Auxiliary metadata-driven wide-area infrared small target detection method

Abstract

The invention discloses an auxiliary metadata-driven wide-area infrared small target detection method, which comprises the following steps of: preprocessing an input infrared image, performing dissimilatory precoding and multi-layer perceptron mapping on three types of metadata, generating an initial auxiliary feature, extracting a multi-layer image feature through a backbone network, and generating a final auxiliary feature. Detail features lacked by element high-level semantics are bridged; and performing iterative dynamic feature fusion on the shallow features and the final auxiliary features by using a multi-modal dynamic modulation algorithm to generate dynamic modulation parameters, calculating final modulation features based on the dynamic modulation parameters and the multi-level image features, obtaining edge-optimized modulation features based on the final modulation features, and performing edge optimization on the edge-optimized modulation features. And based on the feature pyramid and the detection head processing input deep features and edge optimized modulation features, a wide-area infrared small target detection result is obtained, and the cross-domain adaptation / generalization ability of the detection method in a complex reality application scene is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and deep learning technologies, and more specifically to a metadata-assisted wide-area infrared small target detection method and device. Background Art

[0002] Infrared imaging technology has significant advantages in capturing target information in adverse weather conditions and low-visibility environments. However, due to the limitations of long-range imaging, the targets to be detected often exhibit the following typical characteristics: (1) extremely limited spatial resolution, resulting in fuzzy target representation; (2) a general lack of texture and structural features in the target area; and (3) complex backgrounds, which easily lead to imaging features with extremely low signal-to-noise ratios under the interference of environmental clutter. These characteristics make infrared small target detection (IRSTD) a long-standing and challenging task in the field of computer vision.

[0003] Existing IRSTD methods primarily focus on image feature extraction to distinguish small targets from complex backgrounds. With the rise of deep learning, many researchers have addressed the IRSTD problem by emphasizing robust feature extraction and refining neural architectures with attention mechanisms, achieving some progress. However, existing methods still suffer from two key drawbacks when scaling to real-world applications: they are optimized for relatively limited application scenarios (such as land-based platforms operating in the longwave infrared band or space-based platforms operating in the shortwave infrared band), failing to adapt to the wide-area generalization requirements across the sensor-platform-band continuum. Domain-specific techniques are employed to improve detection accuracy (e.g., maritime target detection methods that predict the sea surface area through background decomposition). While these methods reduce the complexity of specific scenarios, they inherently limit cross-domain adaptability. In other words, a pure visual modeling paradigm performs poorly when dealing with the complex background clutter and inherently scarce target features of small infrared targets, making them unsuitable for wide-area detection and the attendant cross-domain generalization challenges. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, in a first aspect, the present invention proposes a wide-area infrared small target detection method driven by auxiliary metadata, comprising: preprocessing an input infrared image, performing specific precoding and multi-layer perceptron mapping on three types of metadata corresponding to the infrared image: imaging platform, imaging band, and image resolution, to generate initial auxiliary features; extracting multi-level image features through a backbone network, downsampling shallow features in the multi-level image features and performing point-by-point subtraction with deep features in the multi-level image features to generate detail compensation features, and performing secondary fusion of the detail compensation features with the initial auxiliary features to generate final auxiliary features; iteratively performing dynamic feature fusion on the reduced multi-level image features and the final auxiliary features using a multimodal dynamic modulation algorithm to generate dynamic modulation parameters, performing channel-space dual modulation on the multi-level image features based on the dynamic modulation parameters to obtain channel modulation features and spatial modulation features, and calculating final modulation features based on the channel modulation features, spatial modulation features, and shallow features; and edge-optimizing the final modulation features using a decomposition edge convolution operator chain to obtain edge-optimized modulation features.

[0005] Optionally, the three types of metadata corresponding to the infrared image, namely the imaging platform, imaging band, and image resolution, are specifically pre-encoded and mapped by a multi-layer perceptron to generate initial auxiliary features, including: performing one-hot encoding on the Air-based / Land-based / Space-based in the imaging platform to obtain pre-processed platform metadata; performing one-hot encoding on the LWIR / SWIR / NIR in the imaging band to obtain pre-processed band metadata; performing normalization processing, aspect ratio processing, and periodic function embedding processing on the image resolution in sequence to obtain pre-processed resolution metadata; inputting the pre-processed platform metadata, pre-processed band metadata, and pre-processed resolution metadata into a learnable linear mapping layer to generate platform features, band features, and resolution features, respectively; and using a multi-layer perceptron to map the platform features, band features, and resolution features into initial auxiliary features.

[0006] Optionally, the mapping of the platform features, band features, and resolution features into initial auxiliary features using a multilayer perceptron includes: channel-splicing the platform features, band features, and resolution features to obtain a first spliced ​​feature; and processing the first spliced ​​feature using an MLP layer based on a residual connection to obtain an initial auxiliary feature.

[0007] Optionally, the method generates detail compensation features by downsampling shallow features in the multi-level image features and subtracting them point by point with deep features in the multi-level image features, including: performing a learnable linear channel conversion on the deep features to obtain deep features after channel conversion; performing a spatially aligned downsampling operation on the shallow features to obtain downsampled shallow features; and calculating the inter-layer feature differences between the converted deep features and the downsampled shallow features by element-by-element subtraction to obtain detail compensation features.

[0008] Optionally, the secondary fusion of the detail compensation feature and the initial auxiliary feature to generate the final auxiliary feature includes: splicing the detail compensation feature and the initial auxiliary feature to obtain a second splicing feature; processing the second splicing feature based on a three-layer residual connection MLP layer to obtain the final auxiliary feature.

[0009] Optionally, the multimodal dynamic modulation algorithm is used to iteratively perform dynamic feature fusion on the multi-level image features after dimensionality reduction and the final auxiliary features to generate dynamic modulation parameters, including: performing average pooling and linear transformation processing on the multi-level image features to obtain the multi-level image features after dimensionality reduction; performing two dynamic MLP iterative processing and dimensionality conversion processing on the multi-level image features after dimensionality reduction and the final auxiliary features to generate dynamic modulation parameters.

[0010] Optionally, the method performs channel-space dual modulation on the multi-level image features based on the dynamic modulation parameters to obtain channel modulation features and spatial modulation features, including: deconstructing the dynamic modulation parameters to obtain sample-by-sample channel modulation parameters and sample-by-sample spatial modulation parameters; multiplying the results obtained by processing the sample-by-sample channel modulation parameters with the multi-level image features channel by channel to obtain channel modulation features; reconstructing the sample-by-sample spatial modulation parameters into two dynamic convolution kernels with zero angle constraints; processing the multi-level image features through maximum pooling and average pooling to obtain a multi-scale context attention map; constructing grouped convolution based on the dynamic convolution kernel, and performing spatial modulation on the multi-scale context attention map based on the grouped convolution to obtain spatial modulation features.

[0011] Optionally, the calculating the final modulation feature based on the channel modulation feature, the spatial modulation feature and the shallow feature includes: summing the channel modulation feature, the spatial modulation feature and the multi-level image feature to obtain the final modulation feature.

[0012] Optionally, the edge optimization of the final modulation feature using a decomposition edge convolution operator chain to obtain an edge-optimized modulation feature includes: using the vertical edge convolution and horizontal edge convolution in the decomposition edge convolution operator chain, as well as the learnable gating parameters, to process the final modulation feature to obtain a vertical edge optimization feature and a horizontal edge optimization feature; adding the result of lightweight instance normalization processing of the vertical edge optimization feature and the horizontal edge optimization feature to the final modulation feature to obtain an edge-optimized modulation feature; wherein the initialization parameters of the learnable operators of the vertical edge convolution and the horizontal edge convolution are approximated based on the two-dimensional Gaussian-Laplacian operator.

[0013] In order to achieve the above-mentioned purpose, the second aspect of the present application also provides a wide-area infrared small target detection device driven by auxiliary metadata, including: an initial auxiliary feature generation module, which is used to preprocess the input infrared image, and at the same time perform specialized precoding and multi-layer perceptron mapping on the three types of metadata corresponding to the infrared image, namely, the imaging platform, the imaging band, and the image resolution, to generate initial auxiliary features; a final auxiliary feature generation module, which is used to extract multi-level image features through a backbone network, downsample the shallow features in the multi-level image features, and perform point-by-point difference with the deep features in the multi-level image features to generate detail compensation features, and perform secondary fusion of the detail compensation features and the initial auxiliary features to generate final auxiliary features; a modulation feature generation module, which is used to utilize multi-mode The state dynamic modulation algorithm performs iterative dynamic feature fusion on the multi-level image features after dimensionality reduction and the final auxiliary features to generate dynamic modulation parameters, and performs channel-space dual modulation on the multi-level image features based on the dynamic modulation parameters to obtain channel modulation features and spatial modulation features, and calculates the final modulation features based on the channel modulation features, spatial modulation features and shallow features; the detection module is used to use a decomposition-type edge convolution operator chain to perform edge optimization on the final modulation features to obtain edge-optimized modulation features, and process the input deep features and edge-optimized modulation features based on the feature pyramid to obtain multi-scale fused image features, and use the detection head to process the multi-scale fused image features to obtain the auxiliary metadata-driven wide-area infrared small target detection results.

[0014] The embodiment of the present invention provides a wide-area infrared small target detection method and device driven by auxiliary metadata. Compared with the existing technology, its beneficial effects are as follows: the input infrared image is preprocessed, and the three types of metadata corresponding to the infrared image, namely the imaging platform, imaging band, and image resolution, are specifically pre-coded and multi-layer perceptron mapped to generate initial auxiliary features to solve the problem of formal differences when the metadata is introduced; multi-level image features are extracted through a backbone network, and the shallow features in the multi-level image features are down-sampled and point-by-point subtracted from the deep features in the multi-level image features to generate detail compensation features, and the detail compensation features are secondary fused with the initial auxiliary features to generate final auxiliary features to bridge the detail features lacking in the high-level semantics of the metadata; a multimodal dynamic modulation algorithm is used to iteratively perform dynamic feature fusion on the shallow features and the final auxiliary features to generate dynamic modulation parameters, and the multi-level image features are channel-space dual modulated based on the dynamic modulation parameters to obtain channel modulation features and spatial modulation features, based on the channel modulation The final modulation features are calculated by combining the deep features, spatial modulation features, and shallow features. Reasonable parameters are generated for the modulation through the full interaction of auxiliary features and image features. A decomposition-based edge convolution operator chain is used to edge-optimize the final modulation features to obtain edge-optimized modulation features. The input deep features and edge-optimized modulation features are processed based on a feature pyramid to obtain multi-scale fused image features. The multi-scale fused image features are processed using a detection head to obtain auxiliary metadata-driven wide-area infrared small target detection results. This application introduces image metadata as auxiliary information for the first time in the field of infrared small target detection, establishing a new paradigm for metadata-driven multimodal infrared small target detection. Domain heterogeneity is then effectively addressed by customizing the representation based on the unique features of each input sample. Target details are refined in the form of a one-dimensional vertical-horizontal convolution chain and a priori initialization method that simulates the two-dimensional Gaussian-Laplacian operator. This improves detection accuracy in complex backgrounds with a very low number of parameters and demonstrates strong generalization capabilities in full-scene environments with significant domain differences. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] To more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0016] Figure 1 A flowchart of a wide-area infrared small target detection method driven by auxiliary metadata provided by an embodiment of the present invention;

[0017] Figure 2 A schematic diagram of the complete information flow of an auxiliary metadata-driven wide-area infrared small target detection method provided by an embodiment of the present invention;

[0018] Figure 3 A network structure diagram of an auxiliary metadata-driven wide-area infrared small target detection method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0020] This specification provides method operation steps as described in the embodiments or flowcharts, but more or fewer operation steps may be included based on routine or non-creative work. When implemented in an actual system or server product, the methods shown in the embodiments or figures may be executed sequentially or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0021] To meet the growing demand for large-scale, multi-platform, and practically deployable detection systems, this application proposes an auxiliary metadata-driven wide-area infrared small target detection method, which can be used for wide-area infrared small target detection (Omni-IRSTD). Currently, the paradigm of infrared small target detection in heterogeneous perception domains covers the following four core dimensions: multi-platform observation (such as land-based, airborne, and space-based); multiple target types (point targets, spot targets, and extended targets); multiple spectral bands (such as long-wave infrared LWIR, near-infrared NIR, medium-wave infrared SWIR); and multi-resolution imaging (from 256×256 to 6000×6000).

[0022] Compared to traditional domain-specific infrared small target detection tasks, Omni-IRSTD faces significant challenges due to environmental context, differences in background characteristics, and drastic variations in target appearance distribution. This variability significantly impacts the model's generalization capabilities, leading to catastrophic performance degradation in several current state-of-the-art Omni-IRSTD methods on wide-area benchmarks. This performance degradation highlights the diverse and complex challenges facing Omni-IRSTD, stemming from a lack of paradigm shift.

[0023] The fundamental limitation of existing Omni-IRSTD methods stems from forcing a unified feature representation across imaging domains with significantly different physical properties. This leads to adversarial conflicts in the parameter space between gradient directions across different domains when aggregating multi-domain data, causing the network to fall into a suboptimal equilibrium. In other words, since ideal features differ fundamentally across domains, a unified strategy is both impractical and compromises representational effectiveness. A more appropriate solution would be to make the model domain-aware, dynamically adjusting the representation strategy based on the imaging conditions of the input data. However, classic unimodal vision architectures suffer from a fundamental flaw: they struggle to effectively infer the physical conditions of a specific imaging domain from raw pixel data alone. However, in practical infrared sensing systems, each captured image is naturally accompanied by a set of auxiliary metadata that captures the physical conditions of image acquisition, such as the sensor platform, spectral band, spatial resolution, and observation angle. This often overlooked metadata actually encodes key factors driving the distribution and appearance patterns of domain-specific features, serving as a compact and semantically rich representation and providing a low-dimensional yet highly information-rich abstraction of the imaging domain. By explicitly incorporating these metadata into the learning process, this application can transform the detection paradigm from a passive modeling approach that relies solely on vision to an active framework based on contextual awareness, enabling the model to dynamically adjust its representation strategy based on the domain characteristics of each input sample.

[0024] The technical solution of the present invention is further described below with reference to the accompanying drawings. It should be noted that some parameter settings of the technical solution of the present invention are designed based on the characteristics of the current dataset, such as the metadata presentation format and selection range, differential processing such as one-hot encoding based on the metadata input format, and alignment of image features and auxiliary features to a specified dimension. These are merely embodiments of the concept of the present invention and are not limited to a narrow range of specific values.

[0025] See also Figure 1 and Figure 2 , Figure 1 A flowchart of a metadata-assisted wide-area infrared small target detection method provided by an embodiment of the present invention. Figure 2 A complete information flow diagram of a metadata-assisted wide-area infrared small target detection method provided by an embodiment of the present invention. This application provides a metadata-assisted wide-area infrared small target detection method, which can be executed by a processor of a server or terminal. The method may include:

[0026] S10. Preprocess the input infrared image, and perform specialized precoding and multi-layer perceptron mapping on the three types of metadata corresponding to the infrared image, namely, the imaging platform, imaging band, and image resolution, to generate initial auxiliary features.

[0027] Specifically, the processor can read the infrared image and the corresponding metadata (imaging platform, imaging band, image resolution), and perform corresponding preprocessing respectively. The former performs size transformation, random flipping, normalization and other processing, and the latter performs specialized precoding according to its form and uses a multi-layer perceptron (MLP) for initial mapping, converting the text / digital metadata into a form suitable for the detection network processing method, and then the metadata features are fused to obtain the initial auxiliary features. The purpose of this step is to eliminate the problem of specific form differences when introducing metadata and align it with the image processing network. This application introduces image metadata as auxiliary information in the field of infrared small target detection for the first time, and establishes a new paradigm for metadata-driven multimodal infrared small target detection.

[0028] In an embodiment of the present application, step S10 may include the following execution process:

[0029] S101. Perform one-hot encoding on the Air-based, Land-based, and Space-based images in the imaging platform to obtain preprocessing platform metadata.

[0030] S102 , performing one-hot encoding on LWIR / SWIR / NIR in the imaging bands to obtain pre-processed band metadata.

[0031] S103 , performing normalization processing, aspect ratio processing, and periodic function embedding processing on the image resolution in sequence to obtain pre-processing resolution metadata.

[0032] S104: input the pre-processed platform metadata, pre-processed band metadata, and pre-processed resolution metadata into a learnable linear mapping layer to generate platform features, band features, and resolution features, respectively.

[0033] S105. Use a multi-layer perceptron to map the platform features, band features, and resolution features into initial auxiliary features.

[0034] Among them, the platform features, band features and resolution features are mapped into initial auxiliary features using a multi-layer perceptron, including:

[0035] The platform feature, band feature and resolution feature are channel-joined to obtain the first joint feature.

[0036] The first concatenated features are processed by the MLP layer based on the residual connection to obtain the initial auxiliary features.

[0037] In the actual execution process, the processor reads the infrared image x and auxiliary metadata, performs one-hot encoding on the imaging platform (Air-based / Land-based / Space-based) and imaging band (LWIR / SWIR / NIR), normalizes the image resolution, calculates the aspect ratio (w / h), and embeds the periodic function. After that, each pre-processed metadata is respectively used to obtain the platform feature z through a learnable linear mapping. p , band characteristics z b , resolution feature z r , the process can be expressed as follows:

[0038] z p =F p (OneHot(platform)),

[0039] z b =F b (OneHot(band)),

[0040]

[0041] Among them, F p 、F b 、F r Represents a learnable linear transformation layer that uniformly transforms the channel dimension to d1=C / 8, σ w , σ h Represents a dataset-specific normalization factor.

[0042] Afterwards, the processor sets the platform feature z p , band characteristics z b , resolution feature z r After channel splicing, the initial fusion is performed by the MLP layer with a residual structure to obtain the initial auxiliary feature z:

[0043]

[0044] Among them, z Represents an MLP layer with a residual structure, which consists of a linear layer, a normalization layer, and a ReLU function. Indicates channel splicing.

[0045] S20. Extract multi-level image features through the backbone network, downsample the shallow features in the multi-level image features and perform point-by-point subtraction with the deep features in the multi-level image features to generate detail compensation features, and perform secondary fusion of the detail compensation features with the initial auxiliary features to generate the final auxiliary features.

[0046] It should be noted that the processor feeds the preprocessed image features into the backbone network to extract multi-layer image features. The shallow image features are downsampled and then subtracted point by point with the deep image features to obtain detail compensation features. The detail compensation features are then fused with the initial auxiliary features to obtain the final auxiliary features. This step aims to bridge the gap between metadata (primarily high-level semantics) and the lack of detailed features.

[0047] The process of downsampling shallow features in the multi-level image features and performing point-by-point subtraction with deep features in the multi-level image features to generate detail compensation features may include:

[0048] Perform learnable linear channel conversion on the deep features to obtain the deep features after channel conversion.

[0049] Perform spatially aligned downsampling on the shallow features to obtain downsampled shallow features.

[0050] The inter-layer feature differences between the converted deep features and the downsampled shallow features are calculated by element-by-element subtraction to obtain the detail compensation features.

[0051] In one embodiment of the present application, performing secondary fusion of the detail compensation feature and the initial auxiliary feature to generate the final auxiliary feature may include:

[0052] The detail compensation feature is spliced ​​with the initial auxiliary feature to obtain a second splicing feature.

[0053] The second concatenated feature is processed by the MLP layer based on the 3-layer residual connection to obtain the final auxiliary feature

[0054] In the actual execution process, the processor extracts the infrared image x into multi-level image features x through the backbone network. i ,i∈{1,2,3,4}, and then obtain the detail compensation feature e according to the inter-layer difference i :

[0055] e i =F e (C ↓ (x i )-x4),

[0056] Among them, F e Represents a learnable linear transformation layer that transforms the channel dimension to d1, C ↓ represents the downsampling convolution for spatial alignment.

[0057] Afterwards, the processor will compensate the detail features e i Fuse it with the initial auxiliary feature z again to get the final auxiliary feature

[0058]

[0059] Among them, aux It represents a stack of three residual MLP layers, d = 4·d1.

[0060] S30. Use a multimodal dynamic modulation algorithm to iteratively perform dynamic feature fusion on the multi-level image features after dimensionality reduction and the final auxiliary features to generate dynamic modulation parameters, and perform channel-space dual modulation on the multi-level image features based on the dynamic modulation parameters to obtain channel modulation features and spatial modulation features, and calculate the final modulation features based on the channel modulation features, spatial modulation features and shallow features.

[0061] The processor inputs shallow image features (the first two layers) and auxiliary features into the Multimodal Dynamic Modulation Module (M2DM). The processor then performs iterative dynamic feature fusion via a dynamic MLP to obtain dynamic modulation parameters. The processor then performs parallel per-sample channel modulation and per-sample dynamic modulation on the image features. This step aims to achieve full interaction between the auxiliary features and the image features, generating reasonable modulation parameters.

[0062] In one embodiment of the present application, the multimodal dynamic modulation algorithm is used to perform iterative dynamic feature fusion on the shallow features and the final auxiliary features, and the generation of dynamic modulation parameters may include:

[0063] The shallow features are average pooled and linearly transformed to obtain the shallow features after dimensionality reduction.

[0064] The shallow features after dimensionality reduction and the final auxiliary features are subjected to two dynamic MLP iterations and dimension conversion processes to generate dynamic modulation parameters.

[0065] In the actual execution process, the processor converts the image features Dimensionality reduction is achieved by average pooling and linear transformation. It is related to auxiliary feature a i Dynamic modulation parameters are obtained by iterative (2-times) dynamic feature fusion and dimension conversion through dynamic MLP

[0066] In one embodiment of the present application, performing channel-space dual modulation on multi-level image features based on dynamic modulation parameters to obtain channel modulation features and spatial modulation features may include:

[0067] The dynamic modulation parameters are deconstructed to obtain sample-by-sample channel modulation parameters and sample-by-sample spatial modulation parameters.

[0068] The result obtained by processing the sample-by-sample channel modulation parameters using the Sigmoid function is multiplied channel by channel with the shallow features to obtain the channel modulation features.

[0069] The sample-by-sample spatial modulation parameters are reconstructed as two dynamic convolution kernels with zero-angle constraints.

[0070] Shallow features are processed by maximum pooling and average pooling to obtain a multi-scale contextual attention map.

[0071] Grouped convolution is constructed based on dynamic convolution kernels, and multi-scale context attention maps are spatially modulated based on grouped convolution to obtain spatial modulation features.

[0072] In the actual execution process, the processor will dynamically modulate the parameter y′ i Deconstruction to obtain the sample-by-sample channel modulation parameter cm i and the sample-by-sample spatial modulation parameter sm i :

[0073]

[0074] cm i Directly adjust the channel importance to obtain channel modulation characteristics

[0075]

[0076] Among them, σ represents the Sigmoid function, Element-wise multiplication.

[0077] The processor then sends sm i Reconstructed into two dynamic convolution kernels with zero angle constraints, whose center weight is equal to the negative of the sum of the edge weights:

[0078]

[0079] Extract image features x through maximum pooling and average pooling i Multi-scale contextual attention map The dynamic convolution kernel is used in the form of grouped convolution to achieve sample-by-sample dynamic spatial modulation:

[0080]

[0081] Then we get the spatial modulation characteristics

[0082]

[0083] In one embodiment of the present application, the final modulation feature is calculated based on the channel modulation feature, the spatial modulation feature, and the shallow feature, including:

[0084] The channel modulation features, spatial modulation features and shallow features are summed to obtain the final modulation features.

[0085] That is, the final modulation characteristics for

[0086] S40. Use a decomposition-type edge convolution operator chain to perform edge optimization on the final modulation features to obtain edge-optimized modulation features. Based on the feature pyramid, process the input deep features and edge-optimized modulation features to obtain multi-scale fused image features. Use the detection head to process the multi-scale fused image features to obtain the auxiliary metadata-driven wide-area infrared small target detection results.

[0087] The processor feeds the modulated shallow image features (the first two layers) into the Lightweight Edge Enhancement Module (LEEM). This performs fine-grained feature additions through a factorized vertical-horizontal 1D convolution chain to achieve detail / edge optimization. The initialization parameters of the vertical-horizontal 1D convolution chain are modeled after the Laplacian operator.

[0088] In one embodiment of the present application, the edge optimization of the final modulation feature using the decomposition edge convolution operator chain to obtain the edge-optimized modulation feature includes:

[0089] The final modulation features are processed using the vertical edge convolution and the horizontal edge convolution in the decomposition edge convolution operator chain and the preset gating parameters to obtain the vertical edge optimization features and the horizontal edge optimization features.

[0090] The results of lightweight instance normalization processing on the vertical edge optimization features and the horizontal edge optimization features are added to the final modulation features to obtain the edge optimized modulation features.

[0091] Among them, the initialization parameters of the learnable operators of vertical edge convolution and horizontal edge convolution are approximated based on the two-dimensional Gaussian-Laplacian operator.

[0092] In the actual implementation process, a decomposed edge convolution operator chain is set up, which is divided into vertical edge convolution and horizontal edge convolution:

[0093] x vert =σ(α)☉ReLU(W 3×1 *x)

[0094] x hori =W 1×3 *x vert

[0095] Among them, α is a learnable gating parameter, W 3×1 and W 1×3 is a learnable operator, ⊙ is element-wise multiplication (same below), and its initialization simulates the approximate effect of the two-dimensional Gaussian-Laplacian operator:

[0096]

[0097] Its implementation effect is similar to the following 3×3 convolution:

[0098]

[0099] This design ensures sensitivity to edges or detailed features.

[0100] After obtaining the edge features, the processor performs lightweight instance normalization on the edge features and adds the normalized edge features to the shallow features to obtain the edge-optimized modulation features:

[0101]

[0102] Among them, μ space and σ space Represent the mean and variance of the edge features of the sample image, γ is the scale control factor, ò=10 -5 Used to prevent the denominator from being zero and ensure numerical stability.

[0103] This application effectively addresses domain heterogeneity by customizing representations based on the unique features of each input sample. It refines target details using a one-dimensional vertical-horizontal convolution chain and a priori initialization method that simulates the two-dimensional Gaussian-Laplacian operator, improving detection accuracy in complex backgrounds with a very low number of parameters. This application demonstrates strong generalization capabilities across a full range of scenarios with significant domain differences.

[0104] See also Figure 3 , Figure 3 In [1], the input is an image and metadata. The metadata includes different types, such as platform and band metadata in text form, and resolution metadata in digital form. After metadata processing, the initial auxiliary features z are obtained. The image is processed by the backbone network to obtain deep features and shallow features. The deep features are subjected to a learnable linear channel conversion to obtain the deep features after channel conversion. The shallow features are spatially aligned downsampling operations to obtain the downsampled shallow features. The inter-layer feature difference x between the converted deep features and the downsampled shallow features is calculated by element-by-element subtraction. e , the inter-layer feature difference x e Fuse with the initial auxiliary feature z to obtain the final auxiliary feature Shallow features and final auxiliary features After processing by the dynamic modulation module (M2DM), the final modulation characteristics are obtained The final modulation characteristics After processing by the Lightweight Edge Enhancement Module (LEEM), edge-optimized modulation features are obtained. Finally, the edge-optimized modulation features and deep features are processed using a feature pyramid to obtain multi-scale fused image features. The multi-scale fused image features are then processed using a detection head to obtain auxiliary metadata-driven wide-area infrared small target detection results.

[0105] Based on the above embodiments, this application also provides a training method for a wide-area infrared small target detection network driven by auxiliary metadata. The specific practice environment of the training method is:

[0106] Hardware: NVIDIA GeForce RTX 3090 GPU, 24GB video memory.

[0107] Training parameters: SGD optimizer, initial learning rate 0.005, batch size 4.

[0108] The specific implementation steps of the training method are:

[0109] Step 1: Data preprocessing: normalize and randomly flip the input infrared image, resize it to 1024×1024, and pre-encode the metadata specifically.

[0110] Step 2: Extract multi-level features x from the infrared image through the backbone network i ,i∈{1,2,3,4}, the metadata features are fused and the image detail features are compensated to generate auxiliary features a i .

[0111] Step 3: Transform the shallow multi-level features x i With auxiliary feature a i Enter M2DM for dynamic feature interaction and dynamic feature modulation.

[0112] Step 4: The modulation features are optimized for detail through LEEM, and then enter FPN for multi-scale fusion.

[0113] Step 5: The detection head performs target position regression and classification.

[0114] Step 6: Loss calculation and back propagation.

[0115] An auxiliary metadata-driven wide-area infrared small target detection device includes an initial auxiliary feature generation module, a final auxiliary feature generation module, a modulation feature generation module, and a detection module. The initial auxiliary feature generation module is used to preprocess the input infrared image and perform specialized precoding and multi-layer perceptron mapping on three types of metadata corresponding to the infrared image: imaging platform, imaging band, and image resolution, to generate initial auxiliary features. The final auxiliary feature generation module is used to extract multi-level image features through a backbone network, downsample the shallow features in the multi-level image features, and perform point-by-point subtraction with the deep features in the multi-level image features to generate detail compensation features, and then perform secondary fusion of the detail compensation features with the initial auxiliary features to generate final auxiliary features. The modulation feature generation module is used to use a multimodal dynamic modulation algorithm to iteratively perform dynamic feature fusion on the reduced multi-level image features and the final auxiliary features to generate dynamic modulation parameters. Based on the dynamic modulation parameters, the multi-level image features are subjected to channel-spatial dual modulation to obtain channel modulation features and spatial modulation features. The final modulation features are calculated based on the channel modulation features, spatial modulation features, and shallow features. The detection module is used to perform edge optimization on the final modulation features using a decomposition-based edge convolution operator chain to obtain edge-optimized modulation features. The input deep features and edge-optimized modulation features are processed based on the feature pyramid to obtain multi-scale fused image features. The multi-scale fused image features are processed using the detection head to obtain auxiliary metadata-driven wide-area infrared small target detection results.

[0116] It should be noted that, in this application, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0117] Each embodiment in this specification is described in a related manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are described briefly because they are generally similar to the method embodiments. For related portions, reference can be made to the description of the method embodiments.

[0118] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A metadata-assisted wide-area infrared small target detection method, characterized in that: include: The input infrared image is preprocessed, and the three types of metadata corresponding to the infrared image, namely the imaging platform, imaging band, and image resolution, are specifically pre-coded and mapped by a multi-layer perceptron to generate initial auxiliary features; The backbone network extracts multi-level image features, downsamples the shallow features in the multi-level image features, and performs point-by-point subtraction with the deep features in the multi-level image features to generate detail compensation features. The detail compensation features are then fused with the initial auxiliary features for a second time to generate the final auxiliary features. The multimodal dynamic modulation algorithm is used to iteratively fuse the shallow features with the final auxiliary features to generate dynamic modulation parameters. Based on the dynamic modulation parameters, the multi-level image features are subjected to channel-space dual modulation to obtain channel modulation features and spatial modulation features. The final modulation features are calculated based on the channel modulation features, spatial modulation features and shallow features. A decomposition-based edge convolution operator chain is used to perform edge optimization on the final modulation features to obtain edge-optimized modulation features. The input deep features and edge-optimized modulation features are processed based on the feature pyramid to obtain multi-scale fused image features. The multi-scale fused image features are processed using the detection head to obtain the auxiliary metadata-driven wide-area infrared small target detection results.

2. The metadata-assisted wide-area infrared small target detection method according to claim 1, wherein: The three types of metadata corresponding to the infrared image, namely, the imaging platform, the imaging band, and the image resolution, are specifically pre-coded and mapped using a multi-layer perceptron to generate initial auxiliary features, including: Perform one-hot encoding on the Air-based, Land-based, and Space-based images in the imaging platform to obtain the pre-processing platform metadata. Perform one-hot encoding on LWIR / SWIR / NIR in the imaging bands to obtain preprocessing band metadata; The image resolution is normalized, aspect-ratio processed, and periodic function embedded in sequence to obtain pre-processed resolution metadata; The pre-processed platform metadata, pre-processed band metadata, and pre-processed resolution metadata are input into the learnable linear mapping layer to obtain the corresponding platform features, band features, and resolution features; Multi-layer perceptron is used to map platform features, band features and resolution features into initial auxiliary features.

3. The metadata-assisted wide-area infrared small target detection method according to claim 2, wherein: The method of mapping the platform features, band features, and resolution features into initial auxiliary features using a multi-layer perceptron includes: Perform channel splicing on the platform feature, the band feature, and the resolution feature to obtain a first splicing feature; The first concatenated features are processed by the MLP layer based on the residual connection to obtain the initial auxiliary features.

4. The metadata-assisted wide-area infrared small target detection method according to claim 1, wherein: The step of downsampling shallow features in the multi-level image features and performing point-by-point subtraction with deep features in the multi-level image features to generate detail compensation features includes: Perform learnable linear channel conversion on the deep features to obtain the deep features after channel conversion; Perform spatially aligned downsampling on the shallow features to obtain downsampled shallow features; The inter-layer feature differences between the converted deep features and the downsampled shallow features are calculated by element-by-element subtraction to obtain the detail compensation features.

5. The metadata-assisted wide-area infrared small target detection method according to claim 1, wherein: The secondary fusion of the detail compensation feature and the initial auxiliary feature to generate the final auxiliary feature includes: Splicing the detail compensation feature with the initial auxiliary feature to obtain a second splicing feature; The second concatenated features are processed by the MLP layer based on 3 layers of residual connections to obtain the final auxiliary features.

6. The metadata-assisted wide-area infrared small target detection method according to claim 1, wherein: The multimodal dynamic modulation algorithm is used to iteratively fuse the shallow features with the final auxiliary features to generate dynamic modulation parameters, including: Perform average pooling and linear transformation on shallow features to obtain shallow features after dimensionality reduction; The shallow features after dimensionality reduction and the final auxiliary features are subjected to two dynamic MLP iterations and dimension conversion processes to generate dynamic modulation parameters.

7. The metadata-assisted wide-area infrared small target detection method according to claim 1, wherein: The method of performing channel-space dual modulation on the multi-level image features based on the dynamic modulation parameters to obtain channel modulation features and spatial modulation features includes: Deconstructing the dynamic modulation parameters to obtain sample-by-sample channel modulation parameters and sample-by-sample spatial modulation parameters; The result obtained by processing the channel modulation parameters of each sample using the Sigmoid function is multiplied by the shallow features channel by channel to obtain the channel modulation features; Reconstruct the sample-by-sample spatial modulation parameters into two dynamic convolution kernels with zero-angle constraints; By processing shallow features through maximum pooling and average pooling, a multi-scale context attention map is obtained; Grouped convolution is constructed based on dynamic convolution kernel, and the multi-scale context attention map is spatially modulated sample by sample based on grouped convolution to obtain spatial modulation features.

8. The metadata-assisted wide-area infrared small target detection method according to claim 1, wherein: The calculation of the final modulation feature based on the channel modulation feature, the spatial modulation feature and the shallow feature includes: The channel modulation features, spatial modulation features and shallow features are summed to obtain the final modulation features.

9. The metadata-assisted wide-area infrared small target detection method according to claim 1, wherein: The edge optimization of the final modulation feature by using a decomposition edge convolution operator chain to obtain an edge-optimized modulation feature includes: The final modulation features are processed using vertical edge convolution and horizontal edge convolution in a decomposition edge convolution operator chain and learnable gating parameters to obtain vertical edge optimization features and horizontal edge optimization features; Performing lightweight instance normalization processing on the vertical edge optimization features and the horizontal edge optimization features in the spatial dimension, and adding the processed results to the final modulation features to obtain the edge optimized modulation features; Among them, the initialization parameters of the learnable operators of vertical edge convolution and horizontal edge convolution are approximated based on the two-dimensional Gaussian-Laplacian operator.

10. A metadata-assisted wide-area infrared small target detection device, characterized in that: include: The initial auxiliary feature generation module is used to preprocess the input infrared image and perform specialized precoding and multi-layer perceptron mapping on the three types of metadata corresponding to the infrared image: imaging platform, imaging band, and image resolution, to generate initial auxiliary features. The final auxiliary feature generation module is used to extract multi-level image features through the backbone network, downsample the shallow features in the multi-level image features and perform point-by-point subtraction with the deep features in the multi-level image features to generate detail compensation features, and perform a secondary fusion of the detail compensation features with the initial auxiliary features to generate the final auxiliary features; A modulation feature generation module is used to perform iterative dynamic feature fusion on the multi-level image features after dimensionality reduction and the final auxiliary features using a multimodal dynamic modulation algorithm to generate dynamic modulation parameters. Based on the dynamic modulation parameters, the multi-level image features are subjected to channel-space dual modulation to obtain channel modulation features and spatial modulation features. The final modulation features are calculated based on the channel modulation features, spatial modulation features, and shallow features. The detection module is used to perform edge optimization on the final modulation features using a decomposition-based edge convolution operator chain to obtain edge-optimized modulation features. The deep features and edge-optimized modulation features of the input are processed based on the feature pyramid to obtain multi-scale fused image features. The multi-scale fused image features are processed using the detection head to obtain auxiliary metadata-driven wide-area infrared small target detection results.

Images