Unsupervised infrared and visible image fusion method based on mask guided flow matching

Abstract

The present application belongs to the technical field of computer vision and deep learning, and discloses an unsupervised infrared and visible light image fusion method based on mask guided flow matching. The method fully utilizes the bidirectional reversible characteristics of ODE solving in the flow matching method, takes the self-supervised cyclic reconstruction as the core closed loop, does not introduce any artificially defined fusion loss, and only relies on the reversible mapping of the flow matching model itself to complete the end-to-end unsupervised fusion. The present application innovatively introduces a physical prior mask extracted by a lightweight network, accurately constrains the reversible flow space through the mask guided reconstruction loss, and forces the flow matching model to automatically retain the two types of most physically valuable complementary information of infrared thermal targets and visible light textures in the limited capacity of the fusion channel. The present application introduces an information bottleneck technology, divides the information channel into a fusion channel and a residual channel, and suppresses the redundant information into meaningless distribution, and filters out the redundant features through the information bottleneck.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning technology, and relates to an unsupervised infrared and visible light image fusion method based on mask-guided flow matching. Background Technology

[0002] Currently, the technologies related to this patent mainly include two aspects: the first is unsupervised infrared and visible light image fusion technology (IVIF); the second is generative flow matching technology.

[0003] Infrared and Visible Image Fusion (IVIF) is one of the core technologies in the field of multimodal perception. Infrared imaging sensors capture thermal radiation information of a scene, clearly highlighting thermal targets under adverse lighting conditions such as nighttime, fog, and strong light obstruction. However, they have inherent drawbacks such as loss of texture details and low spatial resolution. Visible imaging sensors can capture rich scene textures, colors, and structural details, and the imaging effect conforms to human visual habits, but their performance degrades sharply in low light and in adverse weather conditions. Infrared and Visible Image Fusion technology aims to integrate complementary information from the two modalities into a single image, generating a fusion result that simultaneously possesses the saliency of thermal targets and the integrity of texture details. This provides a more reliable input for subsequent high-level vision tasks such as target detection, semantic segmentation, and scene understanding.

[0004] Since physically definable "ground truth" fused images do not exist in real-world scenarios, existing deep learning fusion models generally adopt an unsupervised learning paradigm. They rely solely on the constraints of the source images themselves to complete feature extraction, information decoupling, and image reconstruction. This approach, with its advantages of requiring no manual annotation and adapting to real-world scenarios, has become a mainstream research direction. Early unsupervised fusion methods were mainly based on traditional machine learning, such as traditional multi-scale decomposition, sparse representation, and low-rank representation. These methods rely on handcrafted features, have weak generalization ability, and are prone to problems such as blurred edges, lost textures, and severe artifacts in complex scenes, low lighting, and strong interference conditions. With the rise of deep learning, unsupervised fusion methods based on convolutional neural networks (CNNs) have become mainstream, with architectures based on autoencoders (AEs) and variational autoencoders (VAEs) forming the core framework of unsupervised fusion. Typical methods include U2Fusion, which proposes a unified unsupervised framework and dynamically balances the information retention strength of different modalities through adaptive metric loss; SwinFusion, which models long-range dependencies based on Transformer to improve global structural consistency; and DenseFuse, which proposes an end-to-end unsupervised fusion strategy that uses gradient and intensity loss to drive the network to learn complementary modal information. These methods have achieved significant progress under unsupervised constraints, but they generally suffer from problems such as incomplete decoupling of deep features, easy imbalance of modal information, and insufficient ability to preserve high-frequency components of details. Although unsupervised fusion methods based on CNN and Transformer are superior to traditional algorithms in feature representation, they are essentially still discriminative models, lacking the ability to model the overall distribution of the image, and thus cannot achieve true "information creation and enhancement".

[0005] Based on the above issues, generative models, with their powerful distribution modeling and feature mapping capabilities, have become a research hotspot in the field of image fusion. Among them, Generative Adversarial Networks (GANs) were the earliest generative models applied to image fusion. They achieve the generation of fused images through an adversarial game between the generator and the discriminator. Classic methods include FusionGAN and dual-discriminator GAN, but GANs suffer from defects such as unstable training, mode collapse, and insufficient detail generation capabilities. Diffusion models, with their progressive denoising generation mechanism, have achieved breakthroughs in generation quality and diversity. Existing techniques such as DDFM, Dif-Fusion, and Diff-IF have applied diffusion models to image fusion tasks, but they rely on tens to hundreds of iterative denoising processes, resulting in slow inference speeds and difficulty in meeting the requirements of real-time applications.

[0006] Flow matching, as a new generation of generative modeling paradigm, models the image generation process as a smooth ordinary differential equation (ODE) trajectory evolution by directly learning a continuous deterministic vector field from the source modality distribution to the target fusion distribution. Unlike diffusion models, which require iterative denoising, flow matching learns the precise mapping between data in a simpler and more stable way. It has advantages such as stable training, fast single-step generation, and strong trajectory controllability, making it an ideal choice for real-time multimodal image fusion. Building on this, Rectified Flow further optimizes the flow trajectory by straightening it, enabling near-optimal fusion results in a single generation step, significantly improving inference efficiency and stability in practical engineering deployments. The existing method FusionFM was the first to introduce flow matching into multimodal fusion, significantly improving generation efficiency by directly learning the probabilistic transfer process from the source image to the fused image. However, this method still relies on pseudo-label supervision and does not achieve truly unsupervised learning.

[0007] Flow matching models the image generation process as a smooth solution of an ordinary differential equation (ODE) by learning a continuous-time deterministic vector field. It inherently possesses the characteristic of bidirectional reversibility in both forward evolution and reverse reconstruction. In principle, flow matching defines a path from the initial state... To the target state Differentiable transformation path, and by vector field The following ODE evolution is driven by: Since this vector field is continuous and reversible everywhere in the time interval t∈[0,1], it naturally possesses the bidirectional invertibility of forward evolution and reverse reconstruction. Once the correct vector field is learned, it can be used to... Pushing along the positive direction of ODE to You can also start from Exactly return to the original time along the same ODE by reverse time integration No additional network is required, and no additional errors are introduced. This strong reversibility makes flow matching ideal for unsupervised infrared and visible light image fusion, and it can encode bimodal images into latent variables. Fusion characteristics are obtained in the forward evolution of ODE. Then, the original mode is reconstructed through reverse ODE, forming a self-supervised closed-loop learning.

[0008] While the reversibility of stream matching provides a natural framework for unsupervised fusion, directly applying it to infrared and visible light fusion still faces key challenges: Under unsupervised constraints, models can easily exploit the reversibility property to perform simple channel feature backups, forming trivial solutions that are "only saved and not reused," failing to achieve effective fusion of bimodal information; simultaneously, unreasonable residual processing methods can disrupt the continuity of reversible ODE trajectories, leading to instability in reverse reconstruction. Therefore, how to fully leverage the inherent advantages of stream matching—reversible generation, accurate mapping, and fast inference—to achieve stable, decoupled, and high-quality fusion under completely unsupervised conditions has become a core problem that urgently needs to be solved. Summary of the Invention

[0009] To address the issues of reliance on manually defined fusion losses and convergence difficulties in generative models during unsupervised infrared and visible light image fusion tasks, this invention proposes an unsupervised fusion method based on mask-guided flow matching. This method fully leverages the bidirectional reversibility of ODE solving in flow matching, using self-supervised cyclic reconstruction as the core closed loop. It does not introduce any manually defined fusion loss, relying solely on the reversible mapping of the flow matching model itself to achieve end-to-end unsupervised fusion. Furthermore, this invention innovatively introduces physical prior masks (infrared saliency masks and visible light texture masks) extracted by a lightweight network. The mask-guided reconstruction loss precisely constrains the reversible flow space, forcing the flow matching model to automatically retain the two most physically valuable complementary information types—infrared thermal targets and visible light textures—within a limited fusion channel capacity. In addition, this invention introduces an information bottleneck technique, dividing the information channel into fusion channels and residual channels, and suppressing redundant information into meaningless distributions, filtering out redundant features through the information bottleneck. The entire fusion process is driven by the reconstruction constraints of the reverse process of flow matching, without the need for external supervision or manual fusion rules. This fundamentally solves the problems of unsupervised generative fusion, such as easy collapse, difficulty in convergence, and imbalance of modal information.

[0010] The technical solution of the present invention: An unsupervised infrared and visible light image fusion method based on mask-guided flow matching, comprising the following steps: (1) Physical prior mask extraction; Lightweight pre-trained networks are used to process the input infrared images. and visible light images Infrared saliency soft mask and visible light texture soft mask are extracted to construct spatialized directional reconstruction constraints, where H and W are the height and width of the infrared image and the visible light image, respectively. For infrared saliency soft mask extraction, a pre-trained saliency target detection network is used to analyze infrared images. The process is performed to output the infrared saliency soft mask normalized by the Sigmoid activation function. The soft mask value is positively correlated with the pixel thermal radiation intensity, allowing for the extraction of thermal target regions in infrared images and the localization of core semantic information. For visible light texture soft mask extraction, the pre-trained edge detection network PiDiNet is used to process visible light images. The process involves extracting high-frequency edge and texture features from the visible light image, and outputting a visible light texture soft mask normalized by the Sigmoid activation function. The mask value is positively correlated with the pixel gradient magnitude, extracting texture details and structural edge regions in visible light images, thus locking in the core visual information of visible light images; finally, infrared saliency soft masking is applied. With visible light texture soft mask Perform min-max normalization to constrain the mask values ​​to the [0,1] interval; (2) Forward-backward cyclic flow evolution based on the reversible Euler equation (ODE); Infrared and visible light images are mapped to a unified latent space. The RectifiedFlow method is used to perform forward evolution and channel segmentation of the joint features of the infrared and visible light modes' ODEs. Inverse reconstruction is then performed through the ODE inverse process of the RectifiedFlow method, forming a self-supervised closed loop. The entire process requires no manual design of the fusion loss; it relies solely on the bidirectional invertibility of the ODEs in the RectifiedFlow method to reconstruct the source image. Combined with subsequent information bottlenecks and mask guidance, the reconstruction loss achieves unsupervised feature alignment and information decoupling. RectifiedFlow provides a stable vector field foundation. The specific steps are as follows: 2.1) Feature Encoding of Infrared and Visible Images: Since infrared and visible images originate from different imaging mechanisms and exhibit significant modal differences and distribution gaps in the pixel domain, the infrared and visible images are first projected into a unified latent space to achieve preliminary alignment and transmission of infrared and visible modes within the latent space. A pre-trained variational autoencoder with weighted freezing is then used to encode the features of the visible image... With infrared images Mapping to the latent space yields latent features of the visible light and infrared modes with consistent dimensions. and This enables infrared and visible light modes to interact, transmit, and fuse within the same vector space, and finally, it analyzes the latent features of the visible light and infrared modes along the channel dimension. and By splicing the features together, we obtain the joint hidden features. : Where C is the number of channels for the visible light mode and the infrared mode latent features, and H′ and W′ are the spatial dimensions of the visible light mode and the infrared mode latent features; 2.2) Modified Flow Matching Forward Evolution: The modified flow matching method is used to learn the trajectories of two motions distributed over consecutive time intervals, requiring a vector field prediction network. The velocity changes of the two distributions are calculated, and then the evolution trajectory is fitted using the Euler equation, where t∈[0,1] is the time step of the diffusion model. Let be the latent variable state at time t. The training parameters for the prediction network are used; a single-stream diffused Transformer (DiTs) is used as the vector field prediction network to predict from joint latent features. to target hidden features velocity field Based on the modified flow matching method, joint latent features are analyzed. Perform forward flow derivation and solve the ordinary differential equations. ,in, It is the change at the time step; The linear interpolation path of the forward process is: in, This is the starting state for forward evolution. This is the final state of forward evolution. To minimize the regularized variance, To add Gaussian noise, It follows a standard normal distribution; Training the velocity field prediction network The joint latent features calculated based on the modified flow matching method to target hidden features Theoretical value of velocity change for: Finally, the stream matching loss Network for constrained velocity field prediction Theoretical values ​​for the fitted velocity field: Among them, the target hidden features at the start of training The first and last C channels are initialized with visible light mode and infrared mode latent features. and The mean; 2.3) Feature channel segmentation: segmenting the latent features of the target. Perform equal channel segmentation, and define the first C channels as the fusion feature. The last C channels are defined as residual features. : in, ; 2.4) Reverse flow reconstruction based on reversible ODE: To achieve unsupervised self-supervised closed loop, the target latent features are reconstructed. Performing the "residual discarding" operation yields the target latent features that discard redundant information. ; using target hidden features As the starting point for reverse evolution, the ODE reverse process is executed. The velocity field obtained by the vector field prediction network Substitute the values, perform the inverse velocity field solution, derive up to time t=0, and then reverse the reconstruction of the joint latent features. The reconstructed joint latent features The latent features are divided into infrared and visible light modes based on the channel's front and back edges. and Finally, the latent features of the infrared and visible light modes are extracted using a decoder based on a pre-trained variational autoencoder. and Mapping back to pixel space yields the reconstructed infrared image. With reconstructed visible light images ; Treating the variational autoencoder and single-stream diffusion Transformer above as a whole, and since the flow matching method is used for training, this invention simply refers to this whole as the flow matching model. (3) Information bottleneck construction and residual feature distribution constraints based on Gaussian prior; 3.1) KL Loss Constraints Construct Information Bottlenecks: Flow matching models have a strong ability to fit and perceive data distributions, while the standard normal distribution N(0,I) is the simplest distribution without structure or deterministic information; residual features... Defined as an information garbage can, for residual features Perform Gaussian distribution modeling and calculate the mean of its channel dimensions. With variance By constraining the approximation of the standard normal distribution N(0,I) through KL divergence loss, the ability of residual features to carry deterministic structural information is stripped away from the training level. as follows: 3.2) Gaussian replacement of the reverse starting point: For the target latent features obtained in step (2) The residual features obtained from the forward process are directly discarded. and noise sampled from a standard normal distribution N(0,I) constructs new target latent features as the starting point for reverse evolution. : (4) Multi-criteria loss optimization based on mask guidance; A multi-constraint joint loss function is constructed to achieve game-theoretic optimization between information bottleneck constraints and mask-guided reconstruction. The velocity field of the network is predicted by updating the vector field through the loss function. The driving flow matching model completes the purification and fusion of infrared and visible light modes; The multi-constraint joint loss function consists of four parts: flow matching loss. Global reconstruction minimum loss Mask-guided reconstruction loss and constrained residual characteristics KL divergence loss ; Global Reconstruction Minimum Loss The reconstructed image output by the L1 loss-constrained flow matching model achieves global pixel consistency with the input source image, ensuring that the reconstructed image maintains consistency with the original image in overall brightness, contrast, global structure, and energy distribution. Specifically: Mask-guided reconstruction loss Based on the infrared saliency soft mask and visible light texture soft mask extracted in step (1), the forced flow matching model accurately restores the infrared thermal target and visible light texture details, thereby achieving true cross-modal information fusion; specifically as follows: in, The product of Hadama; The joint loss function with multiple constraints is: in, , , To balance the weights of the various losses, a game between mask-guided reconstruction loss and KL divergence loss forces the flow matching model to maximize complementary information, squeezing in visible light texture details and infrared thermal targets. To achieve true decoupling and fusion of physical characteristics; Finally, the AdamW optimizer is used to optimize the joint loss function with multiple constraints. End-to-end optimization is performed, the trainable parameters of the vector field prediction network are updated, and the weights of the autoencoder are frozen throughout the process. (5) Implementation process of the reasoning stage; Once the flow matching model has completed training, the inference phase does not require inverse reconstruction and loss calculation; only the forward process is needed to output the fused image. The specific steps are as follows: First, the infrared images to be fused are... With visible light images The input stream matching model encodes latent features from infrared and visible light images using a pre-trained variational autoencoder, and then concatenates them to obtain joint latent features. The trained single-flow diffusion Transformer performs forward flow evolution to obtain the target hidden features. The first C channel features are directly taken as the fusion features. , will integrate features The input is a variational autoencoder decoder, which decodes and outputs the final fused image. The decoder used in the inference phase is the same as the decoder used to reconstruct the image in the training phase.

[0011] The beneficial effects of this invention are as follows: The unsupervised infrared and visible light image fusion method based on mask-guided flow matching achieves infrared and visible light image fusion without relying on manually defined fusion losses, and utilizes the bidirectional reversibility of ODE solving in the flow matching method to achieve unsupervised fusion. This invention can generate high-quality fused images that simultaneously possess infrared thermal target saliency and visible light texture integrity, providing more reliable input data for downstream perception tasks such as autonomous driving and security monitoring. Attached Figure Description

[0012] Figure 1 This is the overall architecture diagram. Detailed Implementation

[0013] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0014] Figure 1 This is a schematic diagram of the overall architecture of the unsupervised image fusion method based on mask-guided flow matching and information bottleneck of the present invention. First, an infrared image is input. With visible light images The samples are fed into the Salience Target Detection Network (SOD) and the Edge Detection Network (PiDiNet) respectively to extract the infrared salience soft mask. With visible light texture soft mask First, physical prior constraints are constructed. Then, the infrared and visible light images are mapped to a unified latent space by the encoder E of a weight-frozen variational autoencoder to obtain latent features. and And spliced ​​along the channel into joint latent variables The data is fed into a vector field prediction network based on a single-stream diffusion Transformer (DiTs). The forward flow evolution is completed based on the modified flow matching, and the final state is obtained by solving the ordinary differential equation. And equal channels are segmented into fusion features With residual characteristics Next, the residual characteristics are approximated to a standard normal distribution using KL divergence constraints, and Gaussian noise is used as the starting point for the inverse reconstruction. Replace the residual channel and construct Then, reverse flow sampling is performed, and the reconstructed latent features are obtained up to time t=0. The image is reconstructed by mapping the decoder D of the variational autoencoder back to the pixel space. Finally, a total loss function is constructed, including flow matching loss, global reconstruction loss, mask-guided reconstruction loss, and KL divergence loss, to perform end-to-end optimization of the DiTs network. During the inference stage, only the infrared and visible light images to be fused need to be input. After encoding by the encoder and evolution by the forward flow, the fused features are directly extracted. The final fused image can be output by sending it to the decoder. Our fusion network structure can achieve infrared and visible light image fusion without requiring a manually defined fusion loss, while ensuring that the fused image has complementary information from both modes.

Claims

1. An unsupervised infrared and visible light image fusion method based on mask-guided flow matching, characterized in that, The steps are as follows: (1) Physical prior mask extraction; Lightweight pre-trained networks are used to process the input infrared images. and visible light images Infrared saliency soft mask and visible light texture soft mask are extracted to construct spatialized directional reconstruction constraints, where H and W are the height and width of the infrared image and the visible light image, respectively. (2) Forward-backward cyclic flow evolution based on the reversible Euler equation ODE; Infrared and visible light images are mapped to a unified latent space. The forward evolution and channel segmentation of the joint features of infrared and visible light modes are completed by modifying the flow matching method. Inverse reconstruction is performed through the ODE inverse process of flow matching to form a self-supervised closed loop. (3) Information bottleneck construction and residual feature distribution constraints based on Gaussian prior; (4) Multi-criteria loss optimization based on mask guidance; (5) Implementation process of the reasoning stage.

2. The unsupervised infrared and visible light image fusion method based on mask-guided flow matching according to claim 1, characterized in that, The specific implementation process of step (1) is as follows: For infrared saliency soft mask extraction, a pre-trained saliency target detection network is used to analyze infrared images. The process is performed to output the infrared saliency soft mask normalized by the Sigmoid activation function. The soft mask value is positively correlated with the pixel thermal radiation intensity, and the thermal target area in the infrared image is extracted to locate the core semantic information of the infrared image; For visible light texture soft mask extraction, the pre-trained edge detection network PiDiNet is used to process visible light images. The process involves extracting high-frequency edge and texture features from the visible light image, and outputting a visible light texture soft mask normalized by the Sigmoid activation function. The mask value is positively correlated with the pixel gradient magnitude, extracting texture details and structural edge regions in visible light images, thus locking in the core visual information of visible light images; finally, infrared saliency soft masking is applied. With visible light texture soft mask Perform min-max normalization to constrain the mask values ​​to the [0,1] interval.

3. The unsupervised infrared and visible light image fusion method based on mask-guided flow matching according to claim 1, characterized in that, The specific implementation process of step (2) is as follows: 2.1) Feature Encoding of Infrared and Visible Images: First, the infrared and visible images are projected into a unified latent space to achieve preliminary alignment and transmission of infrared and visible modes within the latent space. Then, a pre-trained variational autoencoder with weighted freezing is used to encode the features of the visible image... With infrared images Mapping to the latent space yields latent features of the visible light and infrared modes with consistent dimensions. and This enables infrared and visible light modes to interact, transmit, and fuse within the same vector space, and finally, it analyzes the latent features of the visible light and infrared modes along the channel dimension. and By splicing the features together, we obtain the joint hidden features. : Where C is the number of channels for the visible light mode and the infrared mode latent features, and H′ and W′ are the spatial dimensions of the visible light mode and the infrared mode latent features; 2.2) Modified Flow Matching Forward Evolution: The modified flow matching method is used to learn the trajectories of two motions distributed over consecutive time intervals, requiring a vector field prediction network. The velocity changes of the two distributions are calculated, and then the evolution trajectory is fitted using the Euler equation, where t∈[0,1] is the time step of the diffusion model. Let be the latent variable state at time t. The training parameters for the prediction network are used; a single-stream diffused Transformer is used as the vector field prediction network to predict from joint latent features. to target hidden features velocity field Based on the modified flow matching method, joint latent features are analyzed. Perform forward flow derivation and solve the ordinary differential equations. ,in, It is the change at the time step; The linear interpolation path of the forward process is: in, This is the starting state for forward evolution. This is the final state of forward evolution. To minimize the regularized variance, To add Gaussian noise, It follows a standard normal distribution; Training the velocity field prediction network The joint latent features calculated based on the modified flow matching method to target hidden features Theoretical value of velocity change for: Finally, the stream matching loss Network for constrained velocity field prediction Theoretical values ​​for the fitted velocity field: Among them, the target hidden features at the start of training The first and last C channels are initialized with visible light mode and infrared mode latent features. and The mean; 2.3) Feature channel segmentation: segmenting the latent features of the target. Perform equal channel segmentation, and define the first C channels as the fusion feature. The last C channels are defined as residual features. : in, ; 2.4) Reverse flow reconstruction based on reversible ODE: To achieve unsupervised self-supervised closed loop, the target latent features are reconstructed. Performing the "residual discard" operation yields the target latent features that discard redundant information. ; using target hidden features As the starting point for reverse evolution, the ODE reverse process is executed. The velocity field obtained by the vector field prediction network Substitute the values, perform the inverse velocity field solution, derive up to time t=0, and then reverse the reconstruction of the joint latent features. The reconstructed joint latent features The latent features are divided into infrared and visible light modes based on the channel's front and back edges. and Finally, the latent features of the infrared and visible light modes are extracted using a decoder based on a pre-trained variational autoencoder. and Mapping back to pixel space yields the reconstructed infrared image. With reconstructed visible light images ; Treating the variational autoencoder and the single-stream diffusion Transformer as a whole, we arrive at a flow matching model.

4. The unsupervised infrared and visible light image fusion method based on mask-guided flow matching according to claim 1, characterized in that, The specific implementation process of step (3) is as follows: 3.1) Constructing an information bottleneck using KL loss constraints: This involves modifying residual features... Defined as an information garbage can, for residual features Perform Gaussian distribution modeling and calculate the mean of its channel dimensions. With variance By constraining the approximation of the standard normal distribution N(0,I) through KL divergence loss, the ability of residual features to carry deterministic structural information is stripped away from the training level. as follows: 3.2) Gaussian replacement of the reverse starting point: For the target latent features obtained in step (2) The residual features obtained from the forward process are directly discarded. and noise sampled from a standard normal distribution N(0,I) constructs new target latent features as the starting point for reverse evolution. : 。 5. The unsupervised infrared and visible light image fusion method based on mask-guided flow matching according to claim 1, characterized in that, The specific implementation process of step (4) is as follows: A multi-constraint joint loss function is constructed to achieve game-theoretic optimization between information bottleneck constraints and mask-guided reconstruction. The velocity field of the network is predicted by updating the vector field through the loss function. The driving flow matching model completes the purification and fusion of infrared and visible light modes; The multi-constraint joint loss function consists of four parts: flow matching loss. Global reconstruction minimum loss Mask-guided reconstruction loss and constrained residual characteristics KL divergence loss ; Global Reconstruction Minimum Loss The reconstructed image output by the L1 loss-constrained flow matching model achieves global pixel consistency with the input source image, ensuring that the reconstructed image maintains consistency with the original image in overall brightness, contrast, global structure, and energy distribution. Specifically: Mask-guided reconstruction loss Based on the infrared saliency soft mask and visible light texture soft mask extracted in step (1), the forced flow matching model accurately restores the infrared thermal target and visible light texture details, thereby achieving true cross-modal information fusion; specifically as follows: in, The product of Hadama; The joint loss function with multiple constraints is: in, , , To balance the weights of the various losses, a game between mask-guided reconstruction loss and KL divergence loss forces the flow matching model to maximize complementary information, squeezing in visible light texture details and infrared thermal targets. To achieve true decoupling and fusion of physical characteristics; Finally, the AdamW optimizer is used to optimize the joint loss function with multiple constraints. End-to-end optimization is performed, the trainable parameters of the vector field prediction network are updated, and the weights of the autoencoder are frozen throughout the process.

6. The unsupervised infrared and visible light image fusion method based on mask-guided flow matching according to claim 1, characterized in that, The specific implementation process of step (5) is as follows: Once the flow matching model has completed training, the inference phase does not require inverse reconstruction and loss calculation; only the forward process is needed to output the fused image. The specific steps are as follows: First, the infrared images to be fused are... With visible light images The input stream matching model encodes latent features from infrared and visible light images using a pre-trained variational autoencoder, and then concatenates them to obtain joint latent features. ; The trained single-flow diffusion Transformer performs forward flow evolution to obtain the target hidden features. The first C channel features are directly taken as the fusion features. , will integrate features The input is a variational autoencoder decoder, which decodes and outputs the final fused image. The decoder used in the inference phase is the same as the decoder used to reconstruct the image in the training phase.

Images