A target detection method based on infrared image fusion

Abstract

A target detection method based on infrared image fusion, comprising S1: acquiring infrared images and visible light images of the same scene; S2: constructing a generative adversarial network framework, including an infrared generator, a visible light generator, an infrared discriminator, and a visible light discriminator; S3: generating infrared and visible light foreground target images; S4: using and respectively discriminating and optimizing the two kinds of foreground target images, and outputting infrared features and visible light features; S5: compensating for spatial deformation through a deformable feature alignment module to realize alignment of the double-mode features; S6: outputting fusion features through a soft attention selection fusion module; S7: integrating the fusion features through a fusion generator to generate target reinforced semantic features; and S8: realizing multi-task feature decoupling of the detector based on the target reinforced semantic features to output a final target detection result. The application is suitable for high-precision target detection scenes in all-weather and complex environments.

Description

Technical Field

[0001] This invention relates to a target detection method based on infrared image fusion, belonging to the fields of image processing technology and pattern recognition technology. It is suitable for high-precision target detection scenarios in all weather and complex environments, and can be widely used in security monitoring, autonomous driving, industrial inspection and other fields. Background Technology

[0002] Target detection is a cornerstone task of computer vision, widely used in security monitoring, autonomous driving, military reconnaissance, and industrial inspection. However, single-spectrum (such as visible light) imaging sensors are greatly affected by environmental conditions. In complex scenarios such as nighttime, fog, snow, strong backlighting, or target camouflage, the quality of visible light images degrades severely, leading to a sharp decline in the performance of visible light-based target detection algorithms. In contrast, infrared imaging technology, by sensing the thermal radiation of objects, can penetrate obstructions and stably present the thermal contours of targets under harsh conditions such as low light and smoke. Therefore, the fusion of infrared and visible light images to construct a robust multi-spectral target perception system capable of operating in all weather and all scenarios has become a research hotspot.

[0003] The core of multispectral fusion target detection based on infrared images lies in the complementary nature of information from different spectral bands. Visible light images provide high spatial resolution, rich texture, and color information for target recognition and classification, while infrared images provide stable thermal radiation information beneficial for determining the presence of targets. By enhancing the background differentiation details of targets through multispectral fusion, high-precision target detection can be achieved in complex environments. Based on the order and depth of interaction between fusion and target detection, existing multispectral target detection systems can be divided into three categories: pixel-level fusion detection, feature-level fusion detection, and decision-level fusion detection.

[0004] Pixel-level multispectral fusion detection first fuses the original pixels of the registered infrared and visible light images to generate an information-enhanced fused image, which is then input into a traditional object detection network. Typical algorithms include multi-scale transformation, linear weighting, and generative adversarial networks (GANs). , ) and self-encoder (e.g. , Feature-level multispectral fusion detection uses a two-stream network to extract features from infrared and visible light images separately, and then fuses them in the middle layer of the network. Typical algorithms include two-stream encoders (such as...). , Attention mechanisms guide fusion (e.g.) , ) and feature alignment networks (such as , ); Decision-level multispectral fusion detection performs independent target detection on two types of images and fuses the results at the decision level. Typical algorithms include bounding box confidence fusion, confidence-weighted fusion, and expert system rule fusion.

[0005] Existing technologies have many shortcomings, specifically: First, the fusion and detection tasks are severely disconnected, and pixel-level fusion methods (such as...) , Focusing solely on improving the subjective visual quality of the fused image and employing fixed fusion rules (such as multi-scale coefficient weighting) without establishing a connection with the detection task results in fused images that, while visually excellent, fail to specifically enhance the semantic information of the target region, making them difficult for the detection model to utilize effectively. Feature-level fusion methods (such as...) , Although feature fusion is performed in the intermediate layer of the network, the parameter update of the fusion module only depends on the fusion loss and does not receive direct gradient feedback from the detection loss. The direction of parameter optimization deviates from the optimal detection requirements, resulting in problems such as redundant fused features and diluted effective detection information. Decision-level fusion methods (such as bounding box confidence-weighted fusion) suffer from severe information loss due to independent detection in the early stages, and the detection confidence calibration of different modalities is difficult, easily leading to missed or duplicate target detections. Secondly, the problem of cross-modal feature misalignment and conflict is prominent. The fundamental difference between infrared and visible light imaging principles leads to a natural semantic gap and spatial offset between bimodal features: semantically, the same target appears as a thermal radiation contour in infrared images and as texture and color information in visible light images. Existing attention fusion modules (such as...) It can only implicitly learn feature weights and cannot explicitly distinguish between conflicting and complementary regions; at the spatial level, factors such as sensor installation deviation and target motion blur cause pixel-level spatial misalignment in dual-modal images, which existing rigid registration methods (such as...) cannot address. Feature matching cannot compensate for non-rigid deformations, further exacerbating the difficulty of feature alignment. Third, multi-scale contextual information is not fully utilized. Existing fusion networks mostly adopt a single-scale fusion strategy, which cannot take into account the detailed features of small targets such as distant pedestrians and the global features of large targets such as large vehicles. This results in large differences in detection accuracy for targets of different scales, making it difficult to meet the full-scale detection needs of complex scenes.

[0006] Therefore, there is an urgent need to provide a target detection method based on infrared image multi-spectral fusion that can solve the problems of disconnect between fusion and detection and misalignment of cross-modal features, and improve the accuracy and robustness of target detection in complex environments. Summary of the Invention

[0007] The technical problem solved by this invention is to address the core shortcomings of existing multi-spectral fusion target detection technologies based on infrared images, such as the disconnect between fusion and detection tasks, misalignment and conflict of cross-modal features, insufficient utilization of multi-scale contextual information, loss of decision-level information, and challenges in confidence calibration. This invention provides a target detection method based on infrared image fusion.

[0008] The technical solution of this invention is: A target detection method based on infrared image fusion includes the following steps: S1: Acquire infrared and visible light images of the same scene, perform image preprocessing to ensure the quality and consistency of the input images; preprocessing includes image registration, noise removal, and size normalization; S2: Construct a multimodal generative adversarial network framework, including an infrared generator. Visible light generator Infrared discriminator Visible light discriminator Fusion Generator and detector ; S3: Via infrared generator Generate an infrared foreground target image using a visible light generator. Generate a visible light foreground target image; S4: Using an infrared discriminator and visible light discriminator The infrared foreground target image and the visible light foreground target image are respectively discriminated and optimized, and the optimized infrared features and visible light features are output. S5: Input the optimized infrared and visible light features into the deformable feature alignment module. Spatial deformation compensation is performed to achieve alignment of dual-modal features; S6: Input the aligned bimodal features into the soft attention selection fusion module. It uses a dual-path attention mechanism to handle feature conflicts and complementarities, and outputs fused features; S7: Through the fusion generator Multi-scale integration of fused features generates target-enhanced semantic features; S8: Detector Based on target-enhanced semantic features, multi-task features are decoupled through target localization and semantic classification branches. The weights of training samples are dynamically adjusted in conjunction with detection confidence, and the target localization and semantic guidance information are iteratively optimized to output the final target detection result.

[0009] Furthermore, infrared generator A two-stage network structure of "sparse coding-feature enhancement" is adopted: the first stage is a sparse coding layer, through... Regularization constraint learning of sparse representation of infrared images preserves significant thermal distribution features of pedestrian and vehicle targets and suppresses background noise interference; the second level is the feature enhancement layer, which first improves the gray-level contrast between the target and the background through histogram equalization, and then focuses the target region features through self-attention window to enhance the continuity of the target contour and generate an infrared foreground target image. Visible light generator The architecture of "frequency domain denoising-detail enhancement" is adopted: First, the visible light image is converted to the frequency domain through two-dimensional Fourier transform, and Gaussian low-pass filtering is used to suppress high-frequency noise. Then, the filtered frequency domain image is inversely transformed back to the spatiotemporal domain. Subsequently, the low-frequency approximate component and high-frequency detail component of the image are extracted by wavelet decomposition. The high-frequency detail component is enhanced by dilated convolution to enhance the edge and texture details of the target. Finally, the enhanced high-frequency detail component and the low-frequency approximate component are fused to generate a visible light foreground target image.

[0010] Further infrared discriminator A multi-scale discriminant network architecture is adopted, with the core objective of distinguishing between fused features and pure infrared features, so that the fusion result can inherit the target-background contrast advantage of the infrared image. The discriminant network contains three scale branches, which correspond to... Original size Original size The original-size feature map is used to capture feature differences at different scales; each scale branch contains 4 convolutional layers, and finally... Convolution outputs a single-channel discrimination result; using The gradient penalty strategy is used, and the loss function of the infrared discriminator is shown in Equation (1): (1) in, It has pure infrared characteristics. As a feature of fusion, for and The random interpolation characteristics, This is the gradient penalty coefficient; This represents the mean of the pure infrared features processed by the infrared discriminator. This indicates the fused features processed by the infrared discriminator. This represents the corresponding gradient.

[0011] Visible light discriminator Using an infrared discriminator The architecture remains largely the same across all scales, with the only difference being the addition of a frequency domain analysis module in each discriminative branch. This module extracts the frequency domain information of visible light features using Fast Fourier Transform and combines it with texture-aware convolutional layers to enhance the ability to discriminate high-frequency texture features of visible light. The loss function employs adversarial loss and pre-training. The perceptual loss of the network is weighted and summed, where the adversarial loss constrains the visible light modal consistency of the fused features, the perceptual loss ensures the authenticity of the texture details of the fused features, and the dual constraint avoids modal shift. Visible light discriminator The total loss function is shown in formula (2): (2) in, To mitigate losses, the value is set to... , It is a characteristic of pure visible light. To perceive loss weights, To perceive the loss, the calculation method is pre-training. The fusion features extracted by the network and the pure visible light features are in Layer distance.

[0012] Furthermore, the training of the multimodal generative adversarial network framework employs an alternating training strategy: first, two generators are fixed. , The parameters are used to train two discriminators. , Optimize the resistance loss; then fix two discriminators. , The parameters are used to train two generators. , This allows the generated foreground target image to better deceive the discriminator while minimizing the reconstruction loss between the generated image and the real target region. Through multiple rounds of alternating training, a dynamic balance between the generator and the discriminator is achieved, ensuring that the generated infrared and visible light foreground target images retain the core features of their respective modalities while possessing good target highlighting capabilities.

[0013] Furthermore, deformable feature alignment module Using the visible light feature map as a reference and the infrared feature map as the object to be aligned, non-rigid alignment is achieved by learning the dynamic displacement field. Deformable Feature Alignment Module The specific structure consists of three parts: a feature encoding layer, a displacement field learning layer, and a deformable convolutional layer. The feature encoding layer uses two convolutional layers to encode the dual-modal features, outputting a feature map with a dimension of 256. The displacement field learning layer uses three convolutional layers, with an output channel number of... , The deformable convolution kernel size corresponds to the size of each convolution kernel position. , Directional offset; the deformable convolutional layer dynamically adjusts the convolution sampling position of the infrared feature map based on the learned displacement field, achieving pixel-level non-rigid deformation.

[0014] Furthermore, the soft attention selection fusion module It includes two pathways: conflict resolution attention and complementary reinforcement attention. a) Conflict resolution attention: First, calculate the response difference value of the aligned bimodal features, and set a threshold. When the response difference value is greater than When a potential conflict region is identified, a conflict mask is generated for that region, and the conflict mask value is determined by... function The calculation yielded: (3) in , These are learnable parameters; This represents the collision mask value.

[0015] Dynamically assign dual-modal feature weights using collision mask values: Infrared weights can be expressed as... The visible light weight can be expressed as This enables adaptive selection of features in conflict areas.

[0016] b) Complementary Enhancement Attention: Calculate the maximum and minimum response values ​​of the bimodal features. When the maximum / minimum ratio is ≥3, it is identified as a potential complementary region, and a complementary mask is generated. The complementary mask value is then processed... function calculate: (4) in Representing the complementary mask value, the strong response mode feature is enhanced by using the complementary mask value. The enhanced feature... It can be represented as: (5) The method for enhancing complementary regions by utilizing reinforcement features and further fusing features can be expressed as follows:

[0017] (6) Among them, the weight selection feature Refers to infrared weighting or visible light weighting. This represents the standard attention-weighted feature.

[0018] Furthermore, the fusion generator The fusion features are integrated at multiple scales to generate target-enhanced semantic features, specifically: Fusion Generator use The architecture extracts multi-scale fusion features through the encoder, performs feature upsampling and fusion in the decoder, and generates target-enhanced semantic features to highlight the semantic information of the target region.

[0019] Furthermore, the detector Based on target-enhanced semantic features, multi-task feature decoupling is achieved through target localization and semantic classification branches. Training sample weights are dynamically adjusted in conjunction with detection confidence, iteratively optimizing target localization and semantic guidance information, and outputting the final target detection result. Specifically: detector use The two-stage detection framework, the first stage being the regional proposal network. Candidate target boxes are generated based on target-enhanced semantic features, and the position coordinates and preliminary confidence scores of the candidate target boxes are output. The second stage is the detection head, which inputs the candidate target box features into the target localization branch and the semantic classification branch respectively to achieve feature decoupling: the localization branch uses a regression head to output the offset of the candidate target box, and the classification branch uses a fully connected layer to output the target category; For low-confidence regions with weak contrast and blurred edge targets, adaptive iterative optimization is initiated: First, the feature response map of the low-confidence target is calculated and fed back to the decoder of the fusion generator to perform secondary enhancement on the features of the region; then, it is re-input into the detector for detection, and the number of iterations is dynamically adjusted according to the confidence level to ensure the detection accuracy of low-quality targets.

[0020] In a second aspect, the present invention also proposes a non-volatile storage medium comprising: a computer program product, wherein the method is executed when the computer program product is executed.

[0021] Thirdly, the present invention also proposes a computer program product, which includes a computer program that, when executed by a processor, implements the method described.

[0022] The advantages of this invention compared to the prior art are: (1) This invention achieves "detection-perception-oriented" fusion optimization by constructing a triple adversarial learning framework, thus solving the problem of disconnect between fusion and detection; and by designing a deformable feature alignment module ( ) and soft attention selection fusion module ( This approach explicitly addresses the issues of non-rigid alignment and feature conflict across modal features; it fully utilizes multi-scale contextual information through multi-scale feature fusion and iterative optimization strategies to improve multi-scale target detection performance; and ultimately achieves high-precision and robust target detection in complex environments.

[0023] (2) This invention solves the problem of disconnect between fusion and detection and improves the quality of detection-oriented fusion: Through the gradient-guided loss modulation mechanism, the detection loss gradient is used as the core prior, and the adversarial loss weight is dynamically adjusted so that the optimization direction of the fusion network is deeply coupled with the requirements of the detection task, avoiding the misconception of "fusion for the sake of fusion".

[0024] (3) This invention overcomes the problem of cross-modal feature misalignment and conflict: The module achieves non-rigid pixel-level alignment through a dynamic displacement field, effectively compensating for target deformation and registration errors; The module explicitly distinguishes conflicting / complementary regions through dual attention masks and adopts a differentiated fusion strategy to improve the semantic consistency of fused features.

[0025] (4) This invention fully utilizes multi-scale contextual information to improve the robustness of multi-scale target detection: The U-shaped structure and skip connections are fused with multi-scale features to generate enhanced features that combine global semantics and local details; the dynamic hard example mining strategy optimizes weak response target learning, improves the detection accuracy of small-scale and low-contrast targets, and achieves full coverage detection of multi-scale targets. Attached Figure Description

[0026] Figure 1 This is a flowchart of the multi-band infrared image fusion detection method provided by the present invention; Figure 2 This is a framework diagram of the multi-band infrared image fusion detection method provided by the present invention; Figure 3 This invention provides a deformable feature alignment module ( ) Structural diagram; Figure 4 The soft attention selection fusion module provided by this invention ( ) Structural diagram. Detailed Implementation

[0027] The specific embodiments of the present invention will now be described in further detail with reference to the accompanying drawings.

[0028] This invention provides a target detection method based on infrared image fusion. It constructs a multi-spectral fusion detection framework based on triple adversarial learning, integrates deformable feature alignment and soft attention selection fusion mechanisms, and introduces a gradient-guided loss modulation strategy to achieve deep coupling between the fusion process and the detection task. Simultaneously, it solves the problems of cross-modal feature misalignment and conflict, improving the accuracy and robustness of target detection in complex environments. The specific technical solution includes the following four core parts: Part 1: Multimodal Image Input and Preprocessing Infrared and visible light images of the same scene are acquired and preprocessed to ensure the quality and consistency of the input images. Preprocessing steps include image registration, noise removal, and size normalization. (1) Image registration: using Algorithm combination A robust matching strategy is employed to achieve spatial alignment between infrared and visible light images. The specific process involves first extracting the data from both images. Feature points are used to calculate the Euclidean distance between feature points to obtain initial matching pairs; then, using... The algorithm eliminates erroneous matching pairs and estimates the homography matrix from the infrared image to the visible light image. Finally, it performs a perspective transformation on the infrared image based on the homography matrix to achieve spatial alignment with the visible light image.

[0029] (2) Noise Removal: To address the salt-and-pepper noise characteristics of infrared images, the following methods are employed: Median filtering of the window is used for noise reduction; to address the Gaussian noise characteristics of visible light images, a method is employed... Denoising was achieved using a Gaussian filter with a window size and a standard deviation of 0.5.

[0030] (3) Size normalization: The preprocessed infrared and visible light images are uniformly scaled to a preset resolution using a bilinear interpolation algorithm (e.g., ...). This ensures that image details are not lost.

[0031] Part Two: Construction of a Triple Adversarial Learning Framework and Foreground Target Image Generation Constructing an infrared generator Visible light generator Infrared discriminator Visible light discriminator The proposed triple adversarial learning framework generates high-quality infrared and visible light foreground target images through adversarial competition between the generator and discriminator, laying the foundation for subsequent feature fusion and detection. The specific design of each module is as follows: (1) Infrared generator This system employs a sparse feature extraction network architecture. Its core objective is to preserve significant thermal distribution features in infrared images and enhance the contrast between the target and the background. The network structure consists of an input layer, three sparse coding convolutional layers, a self-attention layer, a histogram equalization layer, and an output layer. The input layer receives the preprocessed infrared image; the three sparse coding convolutional layers sequentially extract shallow, middle, and deep features from the infrared image, with convolutional kernel sizes of [sizes to be filled in]. , , The step size is 1, and the fill method is... The activation function uses (The negative slope is set to 0.2), and through Sparse regularization constraints (weight coefficient of 0.001) ensure the sparsity of extracted features and preserve key hot distribution features; the self-attention layer generates attention weights by calculating the cosine similarity at different locations in the feature map, enhancing the feature response of the target region and suppressing background interference; the histogram equalization layer adopts an adaptive histogram equalization algorithm (…). The clipping limit parameter is set to 2.0, and the grid size is set to... This further enhances the contrast between the target and the background; the output layer through... Convolution maps the feature map to an infrared foreground target image with the same size as the input image.

[0032] (2) Visible light generator The core objective is to extract high-frequency texture features from visible light images, suppress noise interference, and enhance the edge and detail information of the target. The network structure consists of an input layer, a Fourier transform layer, a wavelet decomposition layer, a dilated convolutional layer, and an output layer: the input layer receives the preprocessed visible light image; the Fourier transform layer uses a two-dimensional fast Fourier transform (FFT). The visible light image is converted to the frequency domain, and high-frequency texture features are filtered by a preset high-frequency threshold (adaptively determined according to the image frequency distribution, usually set to 0.3 times the maximum frequency) to suppress low-frequency noise; the wavelet decomposition layer uses... Three-level wavelet decomposition of wavelet basis is performed to extract Low-frequency components and , , For high-frequency components, soft thresholding (threshold set to 0.02) is applied for denoising, followed by inverse wavelet transform with low-frequency components to obtain the denoised visible light image. The dilated convolutional layer consists of three consecutive dilated convolutions with dilation rates of 2, 4, and 8, respectively, and the kernel size is [missing value]. The step size is 1, and the fill method is... The activation function uses It can expand the receptive field and enhance the edge and detail features of the target without increasing the computational load; the output layer through Convolution maps the feature map to a visible light foreground target image with the same size as the input image.

[0033] (3) Infrared discriminator The network employs a multi-scale discriminative network architecture. Its core objective is to distinguish between fused features and pure infrared features, ensuring that the fused result inherits the target-background contrast advantage of the infrared image. The network contains three scale branches (discriminative branches), each corresponding to... Original size Original size The original-size feature map can capture feature differences at different scales; each scale branch contains 4 convolutional layers with a kernel size of [missing information]. The step size is 2, and the fill method is... The activation function uses (The negative slope is set to 0.2), and finally passed. The convolution outputs a single-channel discrimination result. To improve the stability of the discriminator, a method is adopted... The gradient penalty strategy is adopted, with the penalty coefficient set to 10. The discriminator loss function is shown in Equation (1): (1) in, It has pure infrared characteristics. As a feature of fusion, for and The random interpolation feature (interpolation coefficients are uniformly sampled between [0,1]). Gradient penalty coefficient ( ). This represents the mean of the pure infrared features processed by the infrared discriminator. Indicates the fusion features processed by the infrared discriminator, This represents the corresponding gradient.

[0034] (4) Visible light discriminator : Adoption and The core objective of the matching multi-scale architecture is to distinguish between fused features and pure visible light features, ensuring that the fusion result retains the detailed texture information of the visible light image. The difference lies in the introduction of a frequency domain analysis module and a texture-aware convolutional layer: the frequency domain analysis module uses... The filter bank, configured with 8 directions (0°, 22.5°, 45°, ..., 157.5°) and 5 scales (1, 2, 3, 4, 5), extracts texture frequency features from visible light images, enhancing the ability to discriminate texture details. The texture-aware convolutional layer employs grouped convolution (8 groups) to better capture local texture features. To further improve the quality of the fused features, a dual constraint is applied, combining adversarial loss and the perceptual loss of the pre-trained network: the adversarial loss uses... Loss, perceived loss is achieved through pre-training Network, Extract Layer feature calculation The loss, specifically the perceived loss weight, is set to 0.01. Visible light discriminator. The total loss function is shown in formula (2): (2) in, To mitigate losses, the value is set to... ( (characteristics of pure visible light) For perceived loss weights ( ), To perceive the loss, the calculation method is pre-training. The fusion features extracted by the network and the pure visible light features are in Layer distance.

[0035] The training of the triple adversarial learning framework employs an alternating training strategy: first, fix the generator... , The parameters are used to train the discriminator. , Optimize the countermeasure loss; then fix the discriminator. , Parameters for training the generator , This allows the generated foreground target image to better deceive the discriminator, while minimizing the reconstruction loss between the generated image and the real target region (using...). The loss is calculated with a weighting factor of 0.5. Through multiple rounds of alternating training, a dynamic balance between the generator and the discriminator is achieved, ensuring that the generated infrared and visible light foreground target images retain the core features of their respective modalities while possessing good target highlighting capabilities.

[0036] Part 3: Deformable Alignment and Soft Attention Fusion To address the issues of non-rigid spatial offset and semantic conflict in cross-modal features, this invention designs a deformable feature alignment module (…). ) and soft attention selection fusion module ( The dual optimization structure: first through Achieve pixel-level non-rigid alignment of dual-modal features, and then through The system dynamically distinguishes between conflicting and complementary regions, and adaptively performs feature fusion. The specific design is as follows: (1) Deformable feature alignment module ( ) Using the visible light feature map as a reference, the infrared foreground feature map undergoes non-rigid deformation adjustment to compensate for spatial offsets caused by target deformation, pose changes, or registration residual errors, achieving dual-modal feature semantic alignment. The module consists of a feature encoding layer, an offset prediction layer, and a deformable convolutional layer. The specific workflow is as follows: ① Feature encoding: ... Output infrared foreground feature map and Output visible light foreground feature map The input feature encoding layer contains two consecutive convolutional layers (convolutional kernels). Step size 1, fill , (Activation), extract bimodal deep semantic features to obtain infrared encoded features. With visible light coding features This provides reliable discriminative support for offset prediction.

[0037] ② Offset prediction: and Perform channel splicing, input Convolutional layer (output channel 2, corresponding to) , Directional offset), via Activation maps the offset to the [-1,1] interval, generating a displacement field. The calculation logic is shown in formula (3): (3) In the formula, for function, For 1×1 convolution, For channel splicing, These are the spatial coordinates of the feature map.

[0038] ③ Deformable convolution sampling: Input the displacement field into the deformable convolutional layer (convolution kernel) ), dynamically adjust the sampling point position of the convolution kernel, for Non-rigid sampling adjustment is performed to compensate for spatial offset. The sampling formula is shown in formula (4): (4) In the formula, For convolution kernel weights, , For the displacement field in The offset at that point This is the aligned infrared feature map; zero padding is used when the sampling exceeds the boundary.

[0039] (2) Soft attention selection fusion module ( ) By generating dual-attention masks to dynamically distinguish conflicting / complementary regions, a fusion effect of "conflict resolution and complementarity enhancement" is achieved. The process consists of two steps: mask generation and adaptive fusion, as detailed below: ① Dual attention mask generation: collision mask With complementary mask The values ​​of all are in the range [0,1]. Activation generation, size and input features Figure 1 To.

[0040] Collision mask: Used to define regions with significant bimodal semantic differences (such as glass regions), calculated by... and The pixel-level absolute difference, after After convolution and channel compression, through Activate generation; the larger the difference, the closer the mask value is to 1 (indicating a more severe collision). The calculation formula is shown in formula (5): (5) Complementary mask: Used to define regions of complementary information (e.g., nighttime scenes). It is achieved by calculating the difference between the maximum and minimum values ​​of bimodal features. After convolution and channel compression, through Activation generation: the larger the difference, the closer the mask value is to 1 (indicating stronger complementarity). The calculation formula is shown in formula (6): (6) ② Adaptive Feature Fusion: Based on the region type divided by the double mask, a differentiated strategy is used to calculate the final fused features. The calculation formula is shown in formula (7): (7) In the formula, , Adaptive learning weights for the conflict region (initial value set to 0.5); Enhance complementary information through feature overlay; For masks of non-conflicting and non-complementary regions, The channel attention weighting function is calculated as shown in formula (8): (8) in, For global average pooling, The channel attention weights are achieved by multiplying them with the original features channel by channel.

[0041] Part Four: Target Enhancement Semantic Feature Generation and Detection This section uses a fusion generator. By integrating multi-scale fusion features, highly discriminative target-enhancing semantic features are generated, which are then passed through a two-stage detector. To achieve accurate target localization and classification, and to optimize the detection performance of weak-response targets by combining a dynamic sample weight adjustment strategy, the specific process is as follows: (1) Fusion Generator Employing a U-shaped structure of "encoder-fusion bottleneck layer-decoder," this approach deeply mines multi-scale contextual information to generate data that combines global semantics with local details. Feature map.

[0042] Encoder: Pre-trained As the backbone network, the fully connected layers are removed, retaining 5 feature extraction stages. );Will After inputting into the encoder, feature maps at 5 scales are extracted. The scales are 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input, respectively, and the effectiveness of feature extraction is improved by using pre-trained weights.

[0043] Fusion bottleneck layer: Introducing a cross-modal self-attention mechanism for the deepest features (Possessing strong global semantic information) Calculate the semantic relevance of spatial location, generate global attention weights and perform weighted enhancement to strengthen the response of the target region and suppress background interference, thus obtaining... .

[0044] Decoder: Contains 4 deconvolutional layers (4×4 kernel, stride 2, padding) , (Activation) The feature map resolution is gradually restored through upsampling, and skip connections are made with the feature map at the corresponding scale of the encoder to supplement detailed information. The final output has the same size as the input image. .

[0045] (2) Detector :use The two-stage detection framework achieves multi-task feature decoupling and introduces a dynamic sample weight adjustment strategy to optimize the detection performance of weak response targets: Regional Proposal Network ( ):Will enter ,through After feature extraction via convolution, it is processed through two parallel... Convolutional layers predict candidate boxes respectively ( Branches) and target confidence ( Branches). Anchor boxes are set to 3 scales ( , , There are 9 types in total, including 3 aspect ratios (1:1, 1:2, 2:1); ( (With the threshold set to 0.7) After filtering, 2000 high-confidence candidate boxes are output.

[0046] Detection head: Maps candidate boxes to ,pass Scaling candidate bounding box features of different sizes uniformly to After feature extraction through two fully connected layers, target classification is achieved through parallel fully connected layers (using...). Activation output class probability) and bounding box regression (using Activate output coordinate offset).

[0047] Dynamic sample weight adjustment: A hard example mining strategy is adopted, with a confidence threshold of 0.5. Hard example samples with confidence below the threshold (such as pedestrians at night or vehicles blurred by fog) are assigned 1.5 times the training weight, while easy example samples with confidence above the threshold are assigned 1.0 times the training weight. The sample confidence is recalculated and the weight is adjusted after each iteration, and the iteration is repeated 3 times to fully learn the features of hard example samples.

[0048] (3) Gradient-guided loss modulation and model optimization This invention introduces a gradient-guided loss modulation mechanism to achieve deep coupling between fusion and detection tasks, and designs a multi-loss collaborative optimization strategy to ensure stable model convergence. The specific scheme is as follows: (3.1) Gradient-guided adversarial loss modulation: During backpropagation, the detection loss is calculated. right gradient (Reflecting the degree of influence of fusion features on detection performance), this is used as a modulation factor and multiplied with the adversarial loss, so that the fusion network prioritizes optimizing the feature regions that are key to detection. The modulated adversarial loss is as shown in formula (9): (9) In the formula, Original combat loss ( of or of ), gradient Norm. Corresponding to and The modulation losses are respectively , .

[0049] (3.2) Calculation of detection loss: The weighted sum of the classification loss and the bounding box regression loss is shown in Equation (10): (10) In the formula, Cross-entropy loss (measures the difference between the predicted category and the true label). For smoothing loss; , .

[0050] (3.3) Total Loss and Optimization Strategy: The total loss is the weighted sum of multiple loss terms, as shown in formula (11): (11) In the formula, L2 reconstruction loss of GFUSE (measured) (Differences from the real target semantic feature map) ; Optimization Optimizer (initial learning rate) Cosine annealing decay , ), Train 200 Adopt an early stop strategy ( To prevent overfitting.

[0051] Based on the above technical solution, this invention proposes a target detection method based on infrared image fusion, such as... Figure 1 As shown, it includes the following steps: S1: Acquire infrared and visible light images of the same scene; Specifically, images of the target scene are simultaneously acquired using infrared and visible light imaging sensors. The acquired raw images are preprocessed (e.g., denoising and size normalization) to ensure consistent spatial resolution between the two modalities, laying the foundation for subsequent fusion and detection. In this embodiment, Gaussian filtering is used for preprocessing to remove noise, and the image size is uniformly adjusted to... Pixel.

[0052] S2: Construct a multimodal generative adversarial network framework, the framework including an infrared generator. Visible light generator Infrared discriminator Visible light discriminator Fusion Generator and detector ; S3: Via infrared generator Generate an infrared foreground target image using a visible light generator. Generate a visible light foreground target image; Among them, infrared generator A two-stage network structure of "sparse coding-feature enhancement" is adopted: the first stage is a sparse coding layer, through... Regularization constraints are used to learn the sparse representation of infrared images, preserving the significant thermal distribution features of targets such as pedestrians and vehicles while suppressing background noise interference. The second level is a feature enhancement layer, which first improves the grayscale contrast between the target and the background (roads, green belts) through histogram equalization, and then uses a self-attention module (with the attention window size set to...) Focusing on the features of the target area enhances the continuity of the target outline. Visible light generator. The architecture of "frequency domain denoising-detail enhancement" is adopted: First, the visible light image is converted to the frequency domain through two-dimensional Fourier transform, and Gaussian low-pass filtering (cutoff frequency set to 0.3) is used to suppress high-frequency noise. Then, the filtered frequency domain image is inversely transformed back to the spatial domain. Subsequently, the low-frequency approximate component and high-frequency detail component of the image are extracted by wavelet decomposition. The high-frequency detail component is enhanced by dilated convolution (dilation rate set to 2) to enhance the edge and texture details of the target (such as vehicle outlines and pedestrian clothing textures). Finally, the enhanced high-frequency component and low-frequency component are fused to generate a visible light foreground target image.

[0053] S4: Using an infrared discriminator and visible light discriminator The infrared foreground target image and the visible light foreground target image are respectively discriminated and optimized, and the optimized infrared features and visible light features are output. Infrared discriminator A multi-scale discriminant network structure is adopted, specifically including three parallel discriminant branches, each corresponding to a small ( ),middle( ),big( Three receptive fields, with the following kernel sizes for each branch: , , The step size is 2 for all cases. During the discrimination process, each branch extracts the local-global contrast difference between the fused features and the pure infrared features, integrates the multi-scale difference features through a feature stitching layer, and finally outputs the discrimination result through a fully connected layer. To avoid mode collapse, a method is adopted... The gradient penalty strategy applies a penalty to the discriminator parameters. A regularization constraint (regularization coefficient set to 10) maximizes the difference in distribution between the fused features and pure infrared features in the discriminator output, ensuring that the fusion result inherits the high target-background contrast advantage of the infrared image. Visible light discriminator. Adopted and The architecture is identical across all scales, the difference being the addition of a frequency domain analysis module to each discriminative branch. This module extracts frequency domain information of visible light features using Fast Fourier Transform (FFT) and combines this with texture-aware convolutional layers (with Gaussian kernel initialization) to enhance the discriminative ability for high-frequency texture features of visible light. The loss function employs adversarial loss and pre-training. The perceptual loss of the network is weighted and summed (with a weight ratio of 1:0.8). The adversarial loss constrains the visible light modal consistency of the fused features, while the perceptual loss ensures the authenticity of the texture details of the fused features. The dual constraint avoids modal shift.

[0054] In this embodiment, the gradient guidance mechanism is implemented as follows: During the backpropagation phase, the detection loss output by the detector is first calculated (using...). As a classification loss, As the localization loss, with a weight ratio of 1:1, the gradient of the fused features output by the fusion generator is used to obtain the gradient map. Then, for the gradient map conduct After normalization, the gradient guiding coefficients are obtained. Finally, The modified adversarial loss, incorporated into the adversarial loss function of the dual discriminator, is shown in Equation (12): (12) in For the original combat losses, This is the regularization loss for the discriminator parameters. This mechanism guides the discriminator's optimization process to the detection task, prioritizing the retention of features beneficial to target detection, thus achieving collaborative optimization between fusion and detection tasks.

[0055] S5: Input the optimized infrared and visible light features into the deformable feature alignment module ( Spatial deformation compensation is performed to achieve cross-modal feature alignment; Deformable Feature Alignment Module ( Using visible light feature maps as a reference and infrared feature maps as the objects to be aligned, the core lies in achieving non-rigid alignment by learning a dynamic displacement field. The specific structure includes three parts: a feature encoding layer, a displacement field learning layer, and a deformable convolutional layer. The feature encoding layer uses two convolutional layers (convolutional kernels...). The bimodal features are encoded using a stride of 1 and padding of 1, outputting a feature map with a dimension of 256; the displacement field learning layer uses three convolutional layers (the first two being...). Convolution, output dimension 256, the last layer is Convolution), with the number of output channels being ( For deformable convolution kernel size, this embodiment ), corresponding to each convolution kernel position , Directional offset, activation function adopted The offset range is ensured to be [-1, 1]. The deformable convolutional layer dynamically adjusts the convolution sampling position of the infrared feature map based on the learned displacement field, achieving pixel-level non-rigid deformation. For example, when a pedestrian target in the infrared image is slightly distorted due to motion, the displacement field learning layer generates a corresponding offset. The deformable convolutional layer adjusts the sampling point position to align the pedestrian contour in the infrared feature map with the visible light feature map, improving the alignment accuracy by more than 10% compared to the traditional rigid registration method.

[0056] S6: Input the aligned bimodal features into the soft attention selection fusion module ( The dual-path attention mechanism handles feature conflicts and complementarities, and outputs fused features. Soft attention selection fusion module ( This includes two pathways: conflict resolution attention and complementary reinforcement attention. a) Conflict resolution attention: First, calculate the response difference value of the aligned bimodal features (the difference value between the infrared feature response and the visible light feature response), and set a threshold. ( Cross-validation determined the value to be 0.6, when the difference is greater than... When a potential collision region is identified (e.g., glass region: high visible light characteristic response value, infrared characteristic response value close to 0; haze region: chaotic visible light characteristic response, stable infrared characteristic response), a collision mask is generated for that region. ), mask value through The function calculates to obtain: (13) in , These are learnable parameters. Dual-modal feature weights are dynamically assigned using mask values: the infrared weights can be expressed as... The visible light weight can be expressed as This enables adaptive selection of features in conflict areas.

[0057] b) Complementary Enhancement Attention: Calculate the maximum and minimum response values ​​of the bimodal features. When the maximum / minimum ratio is ≥3 (indicating a strong response in one modality and a weak response in the other), it is identified as a potential complementary region (e.g., vehicles in dark environments: strong infrared feature response, weak visible light feature response; road signs in well-lit areas: strong visible light feature response, weak infrared feature response). Generate a complementary mask. The mask value is obtained through Function calculation: (14) Strong response mode features are enhanced using mask values; enhanced features It can be represented as (15) The method for enhancing complementary regions by utilizing reinforcement features and further fusing features can be expressed as follows:

[0058] (16) Among them, the weight selection feature Refers to infrared weighting or visible light weighting. This represents the standard attention-weighted feature.

[0059] S7: Fusion Generator Multi-scale integration of fused features generates target-enhanced semantic features; Fusion Generator The U-Net architecture is adopted, and multi-scale fusion features are extracted by the encoder. The features are then upsampled and fused in the decoder to generate target-enhanced semantic features, highlighting the semantic information of the target region.

[0060] S8: Detector Based on target-enhanced semantic features, multi-task features are decoupled through target localization and semantic classification branches. The weights of training samples are dynamically adjusted in conjunction with detection confidence, and the target localization and semantic guidance information are iteratively optimized to output the final target detection result.

[0061] detector use A two-stage detection framework innovatively introduces a multi-task feature decoupling and adaptive iterative optimization mechanism: the first stage is a region proposal network ( The first stage generates candidate bounding boxes based on target-enhanced semantic features, outputting the position coordinates and preliminary confidence scores of the candidate boxes. The second stage is the detection head, which inputs the candidate box features into the target localization branch and the semantic classification branch respectively to achieve feature decoupling: the localization branch uses a regression head to output the offset of the target box, and the classification branch uses a fully connected layer to output the target category (pedestrian, vehicle). For low-confidence regions with weak contrast (confidence below 0.5) and blurred edge targets, adaptive iterative optimization is initiated: first, the feature response map of the low-confidence target is calculated and fed back to the decoding layer of the fusion generator for secondary enhancement of the features in this region; then, it is re-input into the detector for detection, and the number of iterations is dynamically adjusted according to the confidence score (3 iterations when the confidence score is 0.3-0.5, and 5 iterations when the confidence score is below 0.3) to ensure the detection accuracy of low-quality targets. In this embodiment, through this mechanism, the detection accuracy of low-confidence targets is improved by more than 20%, and the bounding box localization error of blurred edge targets is reduced by 18%.

[0062] Example 1: Application Case of Target Detection in Nighttime Security Monitoring This embodiment applies the method of the present invention to a nighttime security monitoring scenario on urban roads, enabling accurate detection of pedestrians and vehicles, such as... Figure 2 As shown.

[0063] Scenario: 11:00 PM, on a main urban road with no streetlights and some smog. Traditional visible light monitoring systems cannot clearly identify targets, while infrared monitoring systems can capture the target's thermal outline but lack detailed information.

[0064] Application process: 1. Image Acquisition: Using infrared imaging sensors installed beside the road (detection band: ...) Simultaneously acquiring road scene images with a visible light imaging sensor, the original image size is [size missing]. Pixel; 2. Preprocessing: Gaussian filtering is applied to the acquired raw image for noise reduction (Gaussian kernel size is...). Adjust the image size to Pixels, complete image registration (using The algorithm implements feature point matching and registration. 3. Foreground target image generation: Infrared generator Extracting the thermal distribution features of pedestrians and vehicles from infrared images of road scenes, and enhancing the contrast between the target and the background (road, green belt) through histogram equalization to generate an infrared foreground target image; a visible light generator... The edge texture features of pedestrians and vehicles in visible light images are extracted by Fourier transform, and the details are enhanced by wavelet decomposition to generate visible light foreground target images. 4. Discriminator Optimization: Infrared Discriminator By comparing the contrast difference between infrared foreground target images and pure infrared images, through Gradient penalty policy for feature optimization; visible light discriminator Combining adversarial loss and pre-training The perceptual loss is optimized to improve the features of the visible light foreground target image and preserve texture details; at the same time, a gradient guidance mechanism is introduced to modulate the adversarial loss of the dual discriminator according to the detection loss gradient. 5. Feature Alignment and Fusion: Deformable Feature Alignment Module ( Using the visible light feature map as a reference, the infrared feature map undergoes non-rigid deformation to correct the feature space offset caused by sensor installation angle deviation; the soft attention selection fusion module ( Generate conflict masks for haze areas (bimodal feature conflict), generate complementary masks for pedestrian contour areas (bimodal feature complementarity), and calculate fusion features; 6. Object Detection: Fusion Generator A four-level encoder-decoder structure is used to integrate the fused features at multiple scales to generate target-enhanced semantic features; detector A multi-task feature decoupling mechanism is initiated, with the localization branch accurately outputting the target bounding box and the classification branch identifying the target category. For distant low-confidence pedestrian targets (confidence 0.3-0.5), adaptive iterative optimization is initiated. After three iterations of feature enhancement and detection, the target confidence is improved to above 0.7.

[0065] Test results show that the method of this invention achieves a comprehensive detection accuracy of 92.3% for pedestrians and vehicles, with a detection accuracy of 88.7% for small targets (pixel size < 64×64) and 90.2% for weak contrast targets. Compared with the traditional pixel-level fusion method (DenseFuse+Faster R-CNN) which has a comprehensive detection accuracy of 78.5%, a small target detection accuracy of 72.1%, and a weak contrast target detection accuracy of 75.3%, all indicators are significantly improved. Furthermore, in terms of detection speed, the method of this invention has a single-frame image detection time of 32ms, meeting the requirements of real-time security monitoring (real-time requirement < 50ms).

[0066] This application example demonstrates that the method of the present invention has significant advantages in security monitoring target detection in complex environments such as nighttime fog and haze. It can effectively solve the problems of low detection accuracy and poor robustness of traditional methods, and provide reliable target detection results for security monitoring systems.

[0067] Example 2: Deformable Feature Alignment Module Detailed implementation This embodiment describes the deformable feature alignment module ( DFAM The implementation details of ) are specified, such as Figure 3As shown: The module includes 4 convolutional layers ( ) and 1 deformable convolutional layer ( The parameters for each layer are as follows: Kernel size Step size 1, padding 1, output channels 64, activation function is ; Kernel size Step size 1, padding 1, number of output channels 128, activation function is ; Kernel size Step size 1, padding 1, number of output channels 256, activation function is ReLU; Kernel size Step size 1, padding with 0s, number of output channels ( For deformable convolution kernel size, this embodiment The activation function is Output displacement field; Kernel size The step size is 1, the padding is 1, the input is the infrared feature map and the displacement field, and the output is the aligned infrared feature map.

[0068] The module's working process is as follows: Infrared feature image is transmitted through... After extracting deep features, input Generate a displacement field (offset range [-1, 1]). By adjusting the sampling position of the convolution kernel according to the displacement field, non-rigid deformation is performed on the infrared feature map to achieve semantic alignment with the visible light feature map.

[0069] Example 3: Soft Attention Selection Fusion Module Detailed implementation This embodiment describes the soft attention selection fusion module ( The implementation details of ) are specified, such as Figure 4 As shown: The module includes an attention weight generation network and a fusion computation unit. The attention weight generation network adopts a two-layer convolutional layer structure: First convolutional layer: kernel size Step size 1, padding 1, output channels 64, activation function is ; Second convolutional layer: kernel size Step size 1, padding with 0s, output channel number 2 (corresponding to collision mask and complementary mask respectively), activation function is ; The input to the attention weight generation network is an aligned bimodal feature map (the number of channels is the number of infrared feature channels + the number of visible light feature channels), and the output is a conflict mask and a complementary mask. The mask size is the same as the input feature map size.

[0070] In the fusion computation unit, the weighted selection feature is obtained by weighted summation of the bimodal features using a conflict mask; the enhanced feature is obtained by amplifying the strong response modality feature using a complementary mask (amplification factor of 2); and the standard attention weighted feature is obtained by weighted summation of the bimodal features after calculating the global weights of the bimodal features using global average pooling. Finally, the fused feature is obtained according to the fusion feature calculation formula.

[0071] While specific embodiments of the invention have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

[0072] The parts of this invention not described in detail are common knowledge to those skilled in the art.

Claims

1. A target detection method based on infrared image fusion, characterized in that, include: S1: Acquire infrared and visible light images of the same scene, perform image preprocessing to ensure the quality and consistency of the input images; preprocessing includes image registration, noise removal, and size normalization; S2: Construct a multimodal generative adversarial network framework, including an infrared generator. Visible light generator Infrared discriminator Visible light discriminator Fusion Generator and detector ; S3: Via infrared generator Generate an infrared foreground target image using a visible light generator. Generate a visible light foreground target image; S4: Using an infrared discriminator and visible light discriminator The infrared foreground target image and the visible light foreground target image are respectively discriminated and optimized, and the optimized infrared features and visible light features are output. S5: Input the optimized infrared and visible light features into the deformable feature alignment module. Spatial deformation compensation is performed to achieve alignment of dual-modal features; S6: Input the aligned bimodal features into the soft attention selection fusion module. It uses a dual-path attention mechanism to handle feature conflicts and complementarities, and outputs fused features; S7: Through the fusion generator Multi-scale integration of fused features generates target-enhanced semantic features; S8: Detector Based on target-enhanced semantic features, multi-task features are decoupled through target localization and semantic classification branches. The weights of training samples are dynamically adjusted in conjunction with detection confidence, and the target localization and semantic guidance information are iteratively optimized to output the final target detection result.

2. The target detection method based on infrared image fusion according to claim 1, characterized in that: Infrared generator A two-stage network structure of "sparse coding-feature enhancement" is adopted: the first stage is the sparse coding layer, which learns the sparse representation of infrared images through L1 regularization constraints, preserves the significant thermal distribution features of pedestrian and vehicle targets, and suppresses background noise interference; the second stage is the feature enhancement layer, which first improves the gray-level contrast between the target and the background through histogram equalization, and then focuses the features of the target region through a self-attention window to enhance the continuity of the target contour and generate an infrared foreground target image. Visible light generator The architecture of "frequency domain denoising-detail enhancement" is adopted: First, the visible light image is converted to the frequency domain through two-dimensional Fourier transform, and Gaussian low-pass filtering is used to suppress high-frequency noise. Then, the filtered frequency domain image is inversely transformed back to the spatiotemporal domain. Subsequently, the low-frequency approximate component and high-frequency detail component of the image are extracted by wavelet decomposition. The high-frequency detail component is enhanced by dilated convolution to enhance the edge and texture details of the target. Finally, the enhanced high-frequency detail component and the low-frequency approximate component are fused to generate a visible light foreground target image.

3. The target detection method based on infrared image fusion according to claim 1, characterized in that: Infrared discriminator The system employs a multi-scale discriminant network architecture. Its core objective is to distinguish between fused features and pure infrared features, ensuring that the fused result inherits the target-background contrast advantage of the infrared image. The discriminant network comprises three scale branches, each corresponding to a different scale. Original size Original size The original-size feature map is used to capture feature differences at different scales; each scale branch contains 4 convolutional layers, and finally... Convolution outputs a single-channel discrimination result; using The gradient penalty strategy is used, and the loss function of the infrared discriminator is shown in Equation (1): （1） in, It has pure infrared characteristics. As a feature of fusion, for and The random interpolation characteristics, This is the gradient penalty coefficient; This represents the mean of the pure infrared features processed by the infrared discriminator. This indicates the fused features processed by the infrared discriminator. Indicates the corresponding gradient; Visible light discriminator Using an infrared discriminator The architecture remains largely the same across all scales, with the only difference being the addition of a frequency domain analysis module in each discriminative branch. This module extracts the frequency domain information of visible light features using Fast Fourier Transform and combines it with texture-aware convolutional layers to enhance the ability to discriminate high-frequency texture features of visible light. The loss function employs adversarial loss and pre-training. The perceptual loss of the network is weighted and summed, where the adversarial loss constrains the visible light modal consistency of the fused features, the perceptual loss ensures the authenticity of the texture details of the fused features, and the dual constraint avoids modal shift. Visible light discriminator The total loss function is shown in formula (2): （2） in, To combat losses , It is a characteristic of pure visible light. To perceive loss weights, To perceive the loss, the calculation method is pre-training. The fusion features extracted by the network and the pure visible light features are in Layer distance.

4. The target detection method based on infrared image fusion according to claim 3, characterized in that: The training of the multimodal generative adversarial network framework adopts an alternating training strategy: first, fix two generators. , The parameters are used to train two discriminators. , Optimize the resistance loss; then fix two discriminators. , The parameters are used to train two generators. , This allows the generated foreground target image to better deceive the discriminator while minimizing the reconstruction loss between the generated image and the real target region. Through multiple rounds of alternating training, a dynamic balance between the generator and the discriminator is achieved, ensuring that the generated infrared and visible light foreground target images retain the core features of their respective modalities.

5. The target detection method based on infrared image fusion according to claim 3, characterized in that: Deformable Feature Alignment Module Using the visible light feature map as a reference and the infrared feature map as the object to be aligned, non-rigid alignment is achieved by learning the dynamic displacement field. Deformable Feature Alignment Module The specific structure consists of three parts: a feature encoding layer, a displacement field learning layer, and a deformable convolutional layer. The feature encoding layer uses two convolutional layers to encode the dual-modal features, outputting a feature map with a dimension of 256. The displacement field learning layer uses three convolutional layers, with an output channel number of... , The deformable convolution kernel size corresponds to the size of each convolution kernel position. , Directional offset; the deformable convolutional layer dynamically adjusts the convolution sampling position of the infrared feature map based on the learned displacement field, achieving pixel-level non-rigid deformation.

6. The target detection method based on infrared image fusion according to claim 3, characterized in that: Soft attention selection fusion module It includes two pathways: conflict resolution attention and complementary reinforcement attention. a) Conflict resolution attention: First, calculate the response difference value of the aligned bimodal features, and set a threshold. When the response difference value is greater than At that time, it was identified as a potential conflict zone; A collision mask is generated for this potential collision region. The collision mask value is obtained through... function The calculation yielded: （3） in , These are learnable parameters; Indicates the collision mask value; Dynamically assign dual-modal feature weights using collision mask values: Infrared weights can be expressed as... The visible light weight can be expressed as To achieve adaptive feature selection in conflict regions; b) Complementary Enhancement Attention: Calculate the maximum and minimum response values ​​of the bimodal features. When the maximum / minimum ratio is ≥3, it is identified as a potential complementary region, and a complementary mask is generated. The complementary mask value is then processed... function calculate: （4） in Representing the complementary mask value, the strong response mode feature is enhanced by using the complementary mask value. The enhanced feature... It can be represented as: （5） The method for enhancing complementary regions by utilizing reinforcement features and further fusing features can be expressed as follows: （6） Among them, the weight selection feature Refers to infrared weighting or visible light weighting. This represents the standard attention-weighted feature.

7. The target detection method based on infrared image fusion according to claim 6, characterized in that: The fusion generator The fusion features are integrated at multiple scales to generate target-enhanced semantic features, specifically: Fusion Generator use The architecture extracts multi-scale fusion features through the encoder, performs feature upsampling and fusion in the decoder, and generates target-enhanced semantic features to highlight the semantic information of the target region.

8. The target detection method based on infrared image fusion according to claim 7, characterized in that: The detector Based on target-enhanced semantic features, multi-task feature decoupling is achieved through target localization and semantic classification branches. Training sample weights are dynamically adjusted in conjunction with detection confidence, iteratively optimizing target localization and semantic guidance information, and outputting the final target detection result. Specifically: detector use The two-stage detection framework, the first stage being the regional proposal network. Candidate target boxes are generated based on target-enhanced semantic features, and the position coordinates and preliminary confidence scores of the candidate target boxes are output. The second stage is the detection head, which inputs the candidate target box features into the target localization branch and the semantic classification branch respectively to achieve feature decoupling: the localization branch uses a regression head to output the offset of the candidate target box, and the classification branch uses a fully connected layer to output the target category; For low-confidence regions with weak contrast and blurred edge targets, adaptive iterative optimization is initiated: first, the feature response map of the low-confidence target is calculated and fed back to the decoder of the fusion generator to perform secondary enhancement on the features of the region; Then the data is re-entered into the detector for detection, and the number of iterations is dynamically adjusted according to the confidence level to ensure the detection accuracy of low-quality targets.

9. A non-volatile storage medium, characterized in that, include: A computer program product that, when executed, performs the method described in any one of claims 1 to 8.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images