A non-aligned multi-focus image fusion method and system based on a diffusion model

Abstract

The application belongs to the technical field of image processing, and provides a non-aligned multi-focus image fusion method and system based on a diffusion model. A conditional control branch is used to extract multi-scale features of source images, and each scale feature extracted is respectively subjected to spatial alignment in a feature domain to obtain aligned multi-scale conditional features. The obtained multi-scale conditional features are spliced and then injected into a diffusion decoding layer corresponding to a scale of the image fusion main branch. In the image fusion main branch, the source images are subjected to latent space encoding and coarse fusion and are subjected to noise adding processing. In the diffusion decoding layer, the injected multi-scale conditional features are used to guide an iterative denoising process of the noise-added features to generate fused latent features. The alignment and fusion are coupled into a unified latent feature space based on the diffusion model through the image fusion main branch and the conditional control branch, thereby avoiding the problem that the alignment error is unreservedly propagated to the subsequent fusion stage when the alignment is performed first and then the fusion is performed.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method and system for fusion of unaligned multi-focus images based on a diffusion model. Background Technology

[0002] Most existing multi-focus image fusion methods are based on the assumption that the source images are strictly aligned at the pixel level. In actual shooting, due to hand shake, stage or motor vibration, lens breathing effect, and slight changes in the optical system, there are usually translations, rotations, scaling, or perspective distortions between different focus images acquired in the same scene. If the misaligned source images are directly input into a traditional fusion network, ghosting, edge misalignment, texture breaks, and artifacts are likely to occur in the fusion result.

[0003] For unaligned multifocal images, an aligned image is first obtained through feature point matching, optical flow estimation, or depth homography estimation. Then, the aligned pixel domain image is input into the fusion network. The alignment error will propagate to the subsequent fusion stage without reservation. The out-of-focus area in the multifocal image lacks stable matching texture, which will destroy the feature consistency assumption on which the alignment algorithm depends, resulting in unstable homography matrix estimation. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a non-aligned multi-focus image fusion method and system based on a diffusion model. This invention couples alignment and fusion into a unified latent feature space based on a diffusion model by using a main image fusion branch and a conditional control branch. This avoids the problem of alignment errors propagating unretained to subsequent fusion stages when alignment is performed first and then fusion is performed, thereby improving the stability and detail fidelity of non-aligned multi-focus image fusion.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution: In a first aspect, the present invention provides a method for unaligned multi-focus image fusion based on a diffusion model, comprising: Acquire multiple source images of the same scene with different focal areas and geometric displacements; Multiple source images are processed to obtain a combined image. Based on the combined image and a preset alignment and fusion network, fusion latent features are generated. The alignment and fusion network includes an image fusion main branch and a conditional control branch. The conditional control branch extracts multi-scale features from the source images and performs spatial alignment on each extracted scale feature within the feature domain to obtain aligned multi-scale conditional features. The obtained multi-scale conditional features are concatenated and injected into the diffusion decoding layer of the corresponding scale of the image fusion main branch. In the image fusion main branch, the source images undergo latent spatial encoding and coarse fusion, and noise addition is performed. In the diffusion decoding layer, the injected multi-scale conditional features guide the iterative denoising process of the noise-added features to generate fusion latent features. The generated fusion latent features are decoded to obtain a panoramic depth fusion image.

[0006] Furthermore, the processing of multiple source images includes: unifying the size of the multiple source images to adjust them to the same spatial resolution; normalizing the pixel values ​​of the size-unified source images to map the pixel values ​​to a preset value range; and combining the normalized source images according to the channel dimension.

[0007] Furthermore, the injection of the diffusion decoding layer includes: inputting the processed source image into the image fusion main branch and obtaining the corresponding latent space features through the encoder; inputting each processed source image into the conditional control branch to obtain feature inputs for subsequent alignment and conditional injection; and mapping the image fusion main branch and the conditional control branch to scale, so that features of different scales in the conditional control branch are injected into the diffusion decoding layer of the corresponding scale in the image fusion main branch.

[0008] Furthermore, the alignment of the multi-scale conditional features includes: extracting features layer by layer from each source image using conditional control branches to obtain a set of multi-scale conditional features; selecting the multi-scale features corresponding to one of the source images as reference features, and using the multi-scale features of the remaining source images as features to be aligned, and predicting the spatial offset field at each scale; performing feature domain spatial sampling on the features to be aligned based on the spatial offset field to obtain the aligned multi-scale features; and aggregating the alignment features of each source image at the same scale to obtain the alignment conditional features at that scale.

[0009] Furthermore, the concatenation of the multi-scale conditional features includes: concatenating the multi-source aligned conditional features at the same scale to obtain the concatenated conditional features; performing linear mapping on the concatenated conditional features to generate the keys and values ​​in the cross-attention; performing linear mapping on the diffusion decoding features at the corresponding scale in the main branch of image fusion to generate the query vector; calculating the conditional enhancement features through a pixel-aware cross-attention mechanism; and injecting the attention enhancement features into the diffusion decoding layer at the corresponding scale of the main branch of image fusion to obtain the conditionally guided diffusion decoding features.

[0010] Furthermore, the enhanced features are: ; in, Indicates the first Attention enhancement features at various scales; Indicates the feature channel dimension; For query vector; and These are the keys and values ​​in cross-attention.

[0011] Furthermore, the generation of the fused latent features includes: inputting the source images into the latent space encoder of the image fusion main branch to obtain the latent space representation corresponding to each source image; performing coarse fusion on the latent space features of each source image to obtain initial fused latent features; performing noise processing on the initial fused latent features according to the forward noise addition process of the diffusion model to obtain noisy latent features; in the reverse denoising process, using multi-scale conditional enhancement features as guiding information input to the diffusion decoding layer to predict the noise in the current noisy latent features; performing stepwise denoising and updating the noisy latent features based on the predicted noise; repeating noise prediction and denoising update until the diffusion time step is updated to zero to obtain the final fused latent features.

[0012] Furthermore, the determination of the panoramic depth fusion image includes: inputting the fusion latent features into the decoder of the main branch of image fusion to perform image space reconstruction; and performing pixel range constraint processing on the fusion image to map it to the effective image pixel range.

[0013] Secondly, the present invention also provides an unaligned multi-focus image fusion system based on a diffusion model, comprising: The data acquisition module is configured to acquire multiple source images of the same scene with different focal areas and geometric displacements. The alignment and fusion module is configured to: process multiple source images to obtain a combined image; generate fusion latent features based on the combined image and a preset alignment and fusion network; wherein the alignment and fusion network includes an image fusion main branch and a conditional control branch, the conditional control branch extracts multi-scale features from the source images, and performs spatial alignment on each extracted scale feature within the feature domain to obtain aligned multi-scale conditional features; the obtained multi-scale conditional features are concatenated and injected into the diffusion decoding layer of the corresponding scale of the image fusion main branch; in the image fusion main branch, the source images are latently spatially encoded and coarsely fused, and noise is added; in the diffusion decoding layer, the injected multi-scale conditional features are used to guide the iterative denoising process of the noise-added features to generate fusion latent features; The decoding module is configured to decode the generated fusion latent features to obtain a panoramic depth fusion image.

[0014] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the diffusion-based unaligned multifocus image fusion method described in the first aspect.

[0015] Fourthly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to implement the steps of the diffusion-based unaligned multifocus image fusion method described in the first aspect.

[0016] Fifthly, the present invention also provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the steps of the diffusion-based unaligned multi-focus image fusion method described in the first aspect.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: The alignment and fusion network of this invention includes an image fusion main branch and a conditional control branch. The conditional control branch extracts multi-scale features from the source image and performs spatial alignment on each extracted scale feature within the feature domain to obtain aligned multi-scale conditional features. These multi-scale conditional features are then concatenated and injected into the diffusion decoding layer of the corresponding scale in the image fusion main branch. In the image fusion main branch, latent spatial encoding and coarse fusion of the source image are performed, along with noise addition. In the diffusion decoding layer, the injected multi-scale conditional features guide the iterative denoising process of the noise-added features, generating fused latent features. By coupling alignment and fusion into a unified latent feature space based on a diffusion model through the image fusion main branch and the conditional control branch, the problem of alignment errors propagating unretained to subsequent fusion stages when alignment is performed before fusion is avoided, thus improving the stability and detail fidelity of unaligned multi-focus image fusion. Attached Figure Description

[0018] The accompanying drawings, which form part of this embodiment, are used to provide a further understanding of this embodiment. The illustrative embodiments and their descriptions are used to explain this embodiment and do not constitute an improper limitation of this embodiment.

[0019] Figure 1 This is a schematic diagram of the structure of the unified alignment fusion network in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the unaligned multi-focus source image input and preprocessing process in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the multi-scale feature domain spatial alignment process in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of pixel-aware cross-attention in Embodiment 1 of the present invention. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0022] Example 1: Multi-focus image fusion aims to combine multiple source images with different focus positions in the same scene into a single, clear image with full depth of field, thus solving the problem of limited depth of field in single imaging. This technology can be applied in fields such as microscopic imaging, industrial inspection, robot vision, autonomous driving perception, and scene understanding.

[0023] Most existing multi-focus image fusion methods are based on the assumption that the source images are strictly aligned at the pixel level. However, in real-world shooting, due to hand shake, stage or motor vibration, lens breathing, and minute changes in the optical system, images captured from different focus points in the same scene often exhibit translation, rotation, scaling, or perspective distortion. If misaligned source images are directly input into a traditional fusion network, the fusion result is prone to ghosting, edge misalignment, texture breaks, and artifacts.

[0024] For unaligned multifocal images, existing solutions typically employ a sequential "align first, then fuse" approach. This involves first obtaining an aligned image through feature point matching, optical flow estimation, or depth homography estimation, and then inputting the aligned pixel-domain image into the fusion network. This approach has two problems: first, alignment errors propagate unreservedly to subsequent fusion stages; second, the lack of stable, matchable textures in out-of-focus regions of multifocal images undermines the feature consistency assumption upon which the alignment algorithm relies, leading to unstable homography matrix estimation.

[0025] To solve at least one of the above problems, such as Figure 1 As shown, this embodiment provides a non-aligned multi-focus image fusion method based on a diffusion model, including the following steps: S1, such as Figure 2 As shown, at least two source images with different focal areas and geometric displacements in the same scene are acquired, and the source images are preprocessed. The specific steps are as follows: S1.1 Obtain at least two source images with different focus areas in the same scene, denoted as: ; in, Indicates the first Source image; Indicates the number of source images; there is at least one geometric mismatch among the different source images caused by camera shake, viewpoint changes, or scene micro-motions, including spatial translation, rotation, or scale changes.

[0026] S1.2. Perform size unification processing on the source images, adjusting each source image to have the same spatial resolution, denoted as: ; in, This indicates a size adjustment operation; and These represent the height and width of the unified image, respectively.

[0027] S1.3. Perform pixel value normalization on the source image after unifying the size, mapping the pixel values ​​to a preset numerical range, denoted as: ; in, and These represent the minimum and maximum values ​​of the image pixel range, respectively.

[0028] S1.4. Combine the normalized source images according to the channel dimension to form a network input image group, denoted as: ; in, This indicates a splicing operation. This represents the preprocessed image group input to the unified alignment fusion network.

[0029] S2. Construct a unified alignment and fusion network, and input the preprocessed source images into the image fusion main branch and conditional control branch of this network, respectively. The specific steps are as follows: S2.1 Construct a unified alignment fusion network including an image fusion main branch and a conditional control branch. The image fusion main branch is used to perform latent space coding, coarse fusion, diffusion noise addition, and iterative denoising operations. The conditional control branch is used to extract multi-scale structural features and sharp region features of the source image.

[0030] S2.2, The preprocessed image group obtained in step S1 The input image is fused into the main branch, and the corresponding latent space features are obtained through the encoder, denoted as: ; in, This represents the latent space encoder in the main branch of image fusion; Represents the initial latent space features of the source image group; S2.3. Input each preprocessed source image into the conditional control branch to obtain the feature inputs used for subsequent alignment and conditional injection, denoted as: ; in, The feature encoder representing the conditional control branch. Indicates the first Conditional features corresponding to the source image; S2.4. The image fusion main branch and the conditional control branch are scale-corresponding so that features of different scales in the conditional control branch can be injected into the diffusion decoding layer of the corresponding scale in the image fusion main branch.

[0031] S3, such as Figure 3 As shown, multi-scale features of the source image are extracted using conditional control branches, and spatial alignment is performed on each extracted scale feature within the feature domain. The specific steps to obtain the aligned multi-scale conditional features are as follows: S3.1. Using conditional control branches, perform layer-by-layer feature extraction on each source image to obtain a multi-scale conditional feature set, denoted as: ; in, Indicates the ( k The source image is in the first Conditional features at each scale Indicates the first Feature extraction functions at various scales Indicates the number of feature scales.

[0032] S3.2 Select the multi-scale features corresponding to one of the source images as the reference features, and use the multi-scale features of the remaining source images as the features to be aligned, and predict the spatial offset fields at each scale respectively, denoted as: ; in, Indicates the reference source image at the 1st Features at each scale Indicates the source image to be aligned at the . Features at each scale This represents the offset prediction function. This represents the spatial offset field at the corresponding scale.

[0033] S3.3. Based on the spatial offset field, perform feature domain spatial sampling on the features to be aligned to obtain the aligned multi-scale features, denoted as: ; in, Indicates the spatial location in the feature map. Indicates the first The source image in the first Conditional features aligned at each scale.

[0034] S3.4. Aggregate the alignment features of each source image at the same scale to obtain the alignment condition features at that scale, denoted as: ; in, This indicates a feature aggregation operation. Indicates the first Conditional features aligned at each scale.

[0035] S4, such as Figure 4 As shown, the specific steps for stitching together the aligned multi-scale conditional features and injecting them into the diffusion decoding layer of the corresponding scale in the main branch of image fusion through a pixel-aware cross-attention module are as follows: S4.1. Perform channel concatenation on the multi-source aligned conditional features at the same scale to obtain the concatenated conditional features, denoted as: ; in, Indicates the first Features of splicing conditions at various scales.

[0036] S4.2. Perform linear mapping on the concatenated conditional features to generate the keys and values ​​in the cross-attention function, denoted as: ; in, and They represent the first Key mapping matrix and value mapping matrix at each scale.

[0037] S4.3. Perform linear mapping on the diffusion decoding features of the corresponding scale in the main branch of image fusion to generate a query vector, denoted as: ; in, Indicates the first branch in the main branch of image fusion Diffusion decoding features at various scales, This represents the query mapping matrix.

[0038] S4.4 Calculate conditional enhancement features using a pixel-aware cross-attention mechanism, denoted as: ; in, Indicates the first Attention enhancement features at various scales Indicates the feature channel dimension.

[0039] S4.5. Inject the attention-enhanced features into the diffusion decoding layer at the corresponding scale of the main branch of image fusion to obtain the conditionally guided diffusion decoding features, denoted as: ; in, This represents the diffusion decoding feature after injecting conditional information. Indicates the first Conditional weighting at each scale.

[0040] S5. Perform latent space coding and coarse fusion on the preprocessed source image and add noise. Then, use the injected multi-scale conditional features to guide the iterative denoising process to generate fused latent features. The specific steps are as follows: S5.1. Input the preprocessed source image group into the latent space encoder of the image fusion main branch to obtain the latent space representation corresponding to each source image, denoted as: ; in, Indicates the first Latent space features of the source image; S5.2. Perform coarse fusion of the latent space features of each source image to obtain the initial fused latent features, denoted as: ; in, Indicates the first The fusion weights of the source and source images satisfy: ; S5.3. Following the forward noise addition process of the diffusion model, the initial fusion latent features are noise-added to obtain the first... The noisy latent features of the step are denoted as: ; in, Indicates Gaussian noise. This represents the noise scheduling parameters during the diffusion process.

[0041] S5.4 In the reverse denoising process, the multi-scale conditional enhancement features obtained in step 4 are used as guiding information input to the diffusion decoding layer to predict the noise in the current noisy latent features, denoted as: ; in, This represents a diffusion denoising network. This represents a multi-scale condition-guided feature set.

[0042] S5.5. Based on the predicted noise, the noisy latent features are progressively denoised and updated, denoised as follows: ; in, and These are all scheduling parameters during the diffusion process. This represents random noise.

[0043] S5.6 Repeat steps 5.4 to 5.5 until the diffusion time step is updated to... The final fusion latent features are obtained, denoted as: ; in, This represents the fused latent features generated after condition-guided iterative denoising.

[0044] S6. Decode the fusion latent features to obtain the panoramic depth fusion image. The specific steps are as follows: S6.1. Input the latent fusion features generated in step 5 into the decoder of the main branch of image fusion to perform image spatial reconstruction, denoted as: ; in, This represents the decoder in the main branch of image fusion. This represents the fused image obtained from the reconstruction.

[0045] S6.2. Perform pixel range constraint processing on the reconstructed fused image, mapping it to the effective image pixel range, denoted as: ; in, Represents the pixel truncation function. This represents the final output panoramic depth-blended image.

[0046] S6.3 Output the panoramic deep fusion image, which simultaneously retains the clear focus areas in at least two source images and suppresses ghosting, blurring, and structural misalignment caused by misalignment of the source images.

[0047] This embodiment couples alignment and fusion into a unified latent feature space based on a diffusion model, avoiding the direct propagation of pixel domain alignment errors to the fusion stage in the traditional "align first, then fuse" paradigm.

[0048] This embodiment embeds the multi-scale alignment module into different downsampling scales of the conditional branch, so that the coarse-to-fine geometric estimation and the source image structure condition extraction are performed simultaneously.

[0049] This embodiment uses pixel-aware cross-attention instead of simple feature addition, injecting the high-fidelity structure and clear texture of the source image into the diffusion model denoising process, suppressing diffusion model illusions and improving the detail consistency of the fused image.

[0050] Example 2: This embodiment provides an unaligned multi-focus image fusion system based on a diffusion model, including: The data acquisition module is configured to acquire multiple source images of the same scene with different focal areas and geometric displacements. The alignment and fusion module is configured to: process multiple source images to obtain a combined image; generate fusion latent features based on the combined image and a preset alignment and fusion network; wherein the alignment and fusion network includes an image fusion main branch and a conditional control branch, the conditional control branch extracts multi-scale features from the source images, and performs spatial alignment on each extracted scale feature within the feature domain to obtain aligned multi-scale conditional features; the obtained multi-scale conditional features are concatenated and injected into the diffusion decoding layer of the corresponding scale of the image fusion main branch; in the image fusion main branch, the source images are latently spatially encoded and coarsely fused, and noise is added; in the diffusion decoding layer, the injected multi-scale conditional features are used to guide the iterative denoising process of the noise-added features to generate fusion latent features; The decoding module is configured to decode the generated fusion latent features to obtain a panoramic depth fusion image.

[0051] The working method of the system is the same as that of the diffusion-based non-aligned multi-focus image fusion method in Example 1, and will not be repeated here.

[0052] Example 3: This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the diffusion-based unaligned multifocus image fusion method described in Embodiment 1.

[0053] Example 4: This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, it implements the steps of the diffusion-based unaligned multi-focus image fusion method described in Embodiment 1.

[0054] Example 5: This embodiment provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the non-aligned multi-focus image fusion method based on the diffusion model described in Embodiment 1.

[0055] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A non-aligned multi-focus image fusion method based on a diffusion model, characterized in that, include: Acquire multiple source images of the same scene with different focal areas and geometric displacements; Multiple source images are processed to obtain a combined image. Based on the combined image and a preset alignment and fusion network, fusion latent features are generated. The alignment and fusion network includes an image fusion main branch and a conditional control branch. The conditional control branch extracts multi-scale features from the source images and performs spatial alignment on each extracted scale feature within the feature domain to obtain aligned multi-scale conditional features. The obtained multi-scale conditional features are concatenated and injected into the diffusion decoding layer of the corresponding scale of the image fusion main branch. In the image fusion main branch, the source images undergo latent spatial encoding and coarse fusion, and noise addition is performed. In the diffusion decoding layer, the injected multi-scale conditional features guide the iterative denoising process of the noise-added features to generate fusion latent features. The alignment of the multi-scale conditional features includes: extracting features layer by layer from each source image using conditional control branches to obtain a set of multi-scale conditional features; selecting the multi-scale features corresponding to one source image as a reference feature and the multi-scale features of the remaining source images as features to be aligned, and predicting the spatial offset field at each scale; performing feature domain spatial sampling on the features to be aligned based on the spatial offset field to obtain the aligned multi-scale features; and aggregating the alignment features of each source image at the same scale to obtain the alignment conditional features at that scale. The concatenation of multi-scale conditional features includes: concatenating multi-source aligned conditional features at the same scale to obtain concatenated conditional features; performing linear mapping on the concatenated conditional features to generate keys and values ​​in cross-attention; performing linear mapping on the diffusion decoding features at the corresponding scale in the main branch of image fusion to generate query vectors; calculating conditional enhancement features through a pixel-aware cross-attention mechanism; and injecting the attention enhancement features into the diffusion decoding layer at the corresponding scale of the main branch of image fusion to obtain conditionally guided diffusion decoding features. The generated fusion latent features are decoded to obtain a panoramic depth fusion image.

2. The non-aligned multi-focus image fusion method based on a diffusion model as described in claim 1, characterized in that, The processing of multiple source images includes: unifying the size of the multiple source images to achieve the same spatial resolution; normalizing the pixel values ​​of the unified source images to map the pixel values ​​to a preset range; and combining the normalized source images according to the channel dimension.

3. The non-aligned multi-focus image fusion method based on a diffusion model as described in claim 2, characterized in that, The injection of the diffusion decoding layer includes: inputting the processed source image into the image fusion main branch and obtaining the corresponding latent space features through the encoder; inputting each processed source image into the conditional control branch to obtain feature inputs for subsequent alignment and conditional injection; and mapping the image fusion main branch and the conditional control branch to scale, so that features of different scales in the conditional control branch are injected into the diffusion decoding layer of the corresponding scale in the image fusion main branch.

4. The non-aligned multi-focus image fusion method based on a diffusion model as described in claim 1, characterized in that, Enhanced features are: ； in, Indicates the first Attention enhancement features at various scales; Indicates the feature channel dimension; For query vector; and These are the keys and values ​​in cross-attention.

5. The non-aligned multi-focus image fusion method based on a diffusion model as described in claim 1, characterized in that, The generation of the fused latent features includes: inputting the source images into the latent space encoder of the image fusion main branch to obtain the latent space representation corresponding to each source image; performing coarse fusion on the latent space features of each source image to obtain the initial fused latent features; performing noise processing on the initial fused latent features according to the forward noise addition process of the diffusion model to obtain noisy latent features; in the reverse denoising process, using the multi-scale conditional enhancement features as guiding information input to the diffusion decoding layer to predict the noise in the current noisy latent features; performing stepwise denoising and updating the noisy latent features based on the predicted noise; repeating the noise prediction and denoising update until the diffusion time step is updated to zero to obtain the final fused latent features.

6. The non-aligned multi-focus image fusion method based on a diffusion model as described in claim 1, characterized in that, The determination of the panoramic depth fusion image includes: inputting the fusion latent features into the decoder of the main branch of image fusion to perform image space reconstruction; and performing pixel range constraint processing on the fusion image to map it to the effective image pixel range.

7. A non-aligned multi-focus image fusion system based on a diffusion model, characterized in that, include: The data acquisition module is configured to acquire multiple source images of the same scene with different focal areas and geometric displacements. The alignment and fusion module is configured to: process multiple source images to obtain a combined image; generate fusion latent features based on the combined image and a preset alignment and fusion network; wherein the alignment and fusion network includes an image fusion main branch and a conditional control branch, the conditional control branch extracts multi-scale features from the source images, and performs spatial alignment on each extracted scale feature within the feature domain to obtain aligned multi-scale conditional features; the obtained multi-scale conditional features are concatenated and injected into the diffusion decoding layer of the corresponding scale of the image fusion main branch; in the image fusion main branch, the source images are latently spatially encoded and coarsely fused, and noise is added; in the diffusion decoding layer, the injected multi-scale conditional features are used to guide the iterative denoising process of the noise-added features to generate fusion latent features; The alignment of the multi-scale conditional features includes: extracting features layer by layer from each source image using conditional control branches to obtain a set of multi-scale conditional features; selecting the multi-scale features corresponding to one source image as a reference feature and the multi-scale features of the remaining source images as features to be aligned, and predicting the spatial offset field at each scale; performing feature domain spatial sampling on the features to be aligned based on the spatial offset field to obtain the aligned multi-scale features; and aggregating the alignment features of each source image at the same scale to obtain the alignment conditional features at that scale. The concatenation of multi-scale conditional features includes: concatenating multi-source aligned conditional features at the same scale to obtain concatenated conditional features; performing linear mapping on the concatenated conditional features to generate keys and values ​​in cross-attention; performing linear mapping on the diffusion decoding features at the corresponding scale in the main branch of image fusion to generate query vectors; calculating conditional enhancement features through a pixel-aware cross-attention mechanism; and injecting the attention enhancement features into the diffusion decoding layer at the corresponding scale of the main branch of image fusion to obtain conditionally guided diffusion decoding features. The decoding module is configured to decode the generated fusion latent features to obtain a panoramic depth fusion image.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the diffusion-based unaligned multifocus image fusion method as described in any one of claims 1-6.

Images