Image fusion method and system based on paired mean residual decomposition and statistical weighting

Abstract

The application discloses an image fusion method and system based on paired mean residual decomposition and statistical weighting, and relates to the technical field of medical image processing. The method comprises the following steps: acquiring at least two medical images to be fused; obtaining corresponding mean components based on a plurality of mean kernels of different scales and / or directions; determining residual components according to the difference between the medical images to be fused and the mean components, thereby forming a plurality of paired mean residual branches; determining mean statistical responses and residual statistical responses respectively, and determining mean modal weights and residual modal weights according to the statistical responses; fusing the mean components and the residual components respectively, and combining to obtain a branch fusion result; determining branch activities and branch weights according to the statistical responses of the branches, and aggregating to obtain a fused medical image. The structural information and the detailed information of different images are utilized respectively, and the effective responses of different scales and directions are adaptively utilized, so that the structure and the details are reserved, the information omission and redundancy are reduced, and the fusion processing complexity is reduced.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to an image fusion method and system based on paired mean residual decomposition and statistical weighting. Background Technology

[0002] Medical images can reflect the structural or functional information of human tissues, organs, and diseased areas in a non-invasive or minimally invasive manner, playing a crucial role in disease screening, clinical diagnosis, treatment planning, and efficacy evaluation. Due to differences in imaging principles, tissue contrast mechanisms, spatial resolution, and sensitive information types among various medical imaging modalities, different modalities of medical images often provide complementary imaging information. To comprehensively utilize the effective information from different medical images, multimodal medical image fusion technology registers, processes, and combines information from different modalities of the same examination object and the same anatomical region to generate a fused image containing multi-source image features, thus providing an image foundation for medical image observation, lesion analysis, and computer-aided diagnosis.

[0003] Currently, multimodal medical image fusion typically employs pixel weighting, multi-scale transformation, sparse representation, or neural networks to extract and combine structural and detail information from images of different modalities. Multi-scale transformation methods usually decompose an image into image components of different scales or frequencies, then fuse them according to predefined rules. Deep learning methods utilize network structures such as convolutional neural networks, encoder-decoders, attention mechanisms, or Transformers to extract image features and generate a fused image based on the learned feature representations or fusion weights. However, different image components represent different information characteristics. Using the same or fixed fusion rules makes it difficult to simultaneously capture effective information from different modalities regarding tissue structure, grayscale contrast, lesion boundaries, and local texture. Furthermore, image responses at different scales and in different directions vary with tissue regions and lesion morphologies; using a fixed aggregation method can easily lead to the omission of effective features or information redundancy. In addition, some fusion methods lack a clear correspondence between structural components and detailed components, which can easily lead to information loss during fusion and reconstruction. Fusion methods that rely on complex feature extraction networks, reconstruction networks, or multi-stage training usually have problems such as a large number of model parameters, complex training processes, high computational resource requirements, and insufficient interpretability of the fusion process.

[0004] Therefore, in the process of multimodal medical image fusion, the difficulty in coordinating and preserving structural and detailed information, the difficulty in adaptively utilizing effective information at different scales and orientations, and the high complexity and computational cost of the fusion model have become urgent problems to be solved. Summary of the Invention

[0005] This application provides an image fusion method and system based on paired mean residual decomposition and statistical weighting, aiming to solve the problems in the existing technology of multimodal medical image fusion, such as the difficulty in coordinating and preserving structural and detailed information, the difficulty in adaptively utilizing effective information at different scales and directions, and the high complexity and computational cost of the fusion model.

[0006] In a first aspect, an image fusion method based on paired mean residual decomposition and statistical weighting is provided, the method comprising: At least two medical images to be fused are acquired. The average of each medical image to be fused is processed based on multiple mean kernels with different scales and / or orientations to obtain the mean components corresponding to each mean kernel. The residual components are determined based on the difference between each medical image to be fused and the corresponding mean components, forming multiple paired mean residual branches corresponding to each mean kernel. The mean statistical response of the mean component and the residual statistical response of the residual component corresponding to different medical images to be fused in each of the paired mean residual branches are determined respectively, and the mean mode weight and the residual mode weight are determined respectively based on the mean statistical response and the residual statistical response. For each of the paired mean residual branches, the mean components corresponding to different medical images to be fused are fused according to the mean mode weights to obtain fused mean components, and the residual components corresponding to different medical images to be fused are fused according to the residual mode weights to obtain fused residual components. The fused mean components and the fused residual components are combined to obtain the corresponding branch fusion result. The branch activity is determined based on the mean statistical response and residual statistical response corresponding to each of the paired mean residual branches, and the corresponding branch weight is determined based on the branch activity. The fusion results of the corresponding branches are aggregated according to the weights of each branch to obtain a fused medical image.

[0007] Optionally, in the above scheme, acquiring at least two medical images to be fused includes: Acquire at least two original medical images corresponding to the same examination object and the same anatomical region to obtain a set of original medical images; Spatial registration is performed on each original medical image in the original medical image set to obtain a registered medical image corresponding to the spatial location. The registered medical images are subjected to size unification and intensity normalization to obtain the medical images to be fused.

[0008] In the above scheme, optionally, the multiple mean kernels with different scales and / or orientations are determined in the following way: Multiple directional anchor point kernels are constructed based on multiple preset scales and multiple preset directions to obtain a set of directional anchor point kernels; For each of the directional anchor point kernels, the corresponding learnable kernel parameters are non-negatively normalized to obtain the learnable normalized kernel. The mean kernel is obtained by combining each directional anchor kernel and the corresponding learnable normalized kernel according to the adjustment ratio; wherein the adjustment ratio is used to limit the degree of adjustment of the learnable normalized kernel relative to the corresponding directional anchor kernel, the kernel elements in each mean kernel are non-negative, and the sum of the kernel elements in each mean kernel is 1.

[0009] Optionally, in the above scheme, forming multiple paired mean residual branches corresponding to each of the mean kernels includes: For any target mean kernel, mean filtering is performed on each of the medical images to be fused using the target mean kernel to obtain the target mean component corresponding to each of the medical images to be fused. The difference between each of the medical images to be fused and the corresponding target mean component is calculated to obtain the target residual component corresponding to each of the medical images to be fused. Each target mean component and each target residual component corresponding to the same target mean kernel are associated according to the corresponding medical image to be fused to form a paired mean residual branch corresponding to the target mean kernel; wherein, for each medical image to be fused, the sum of the pixel values ​​of the target mean component and the target residual component at the same pixel position is equal to the pixel value of the medical image to be fused at that pixel position.

[0010] In the above scheme, optionally, the mean statistical response and the residual statistical response are determined in the following way: For each of the mean components, the local mean and local second moment in the corresponding local region are determined, and the local variance is determined based on the local mean and the local second moment to obtain the local statistical information of the mean. The mean statistical response is obtained by statistically combining the local mean and local variance in the local statistical information of the mean. For each residual component, the local response of the absolute value of the residual component and the local quadratic response of the residual component in the corresponding local region are determined to obtain the local statistical information of the residual. The residual statistical response is obtained by statistically combining the local response of the absolute value of the residual component and the local quadratic response of the residual component in the local statistical information of the residual.

[0011] Optionally, in the above scheme, determining the mean mode weight and the residual mode weight based on the mean statistical response and the residual statistical response respectively includes: For the same paired mean residual branch, the mean statistical response corresponding to different medical images to be fused is non-negatively normalized to obtain a set of mean modal weights; wherein, the sum of the mean modal weights in the set of mean modal weights is 1; Non-negative normalization is performed on the residual statistical responses corresponding to different medical images to be fused to obtain a set of residual mode weights; wherein the sum of the residual mode weights in the set of residual mode weights is 1.

[0012] Optionally, in the above scheme, obtaining the corresponding branch fusion result includes: Based on the mean modality weight set, the mean components corresponding to different medical images to be fused in the same paired mean residual branch are weighted and fused to obtain the fused mean components; Based on the residual modal weight set, the residual components corresponding to different medical images to be fused in the same paired mean residual branch are weighted and fused to obtain the fused residual components. The fusion mean component and the fusion residual component are directly added together to obtain the corresponding branch fusion result.

[0013] Optionally, in the above scheme, determining the branch activity based on the mean statistical response and residual statistical response corresponding to each of the paired mean residual branches, and determining the corresponding branch weight based on the branch activity, includes: For each of the paired mean residual branches, the mean statistical responses corresponding to different medical images to be fused are aggregated to obtain the aggregated mean statistical response value. The residual statistical responses corresponding to different medical images to be fused are aggregated to obtain the aggregated value of the residual statistical response. The aggregated value of the mean statistical response and the aggregated value of the residual statistical response are combined to obtain the corresponding branch activity. The branch activity degrees corresponding to the multiple paired mean residual branches are normalized to obtain a set of branch weights; The fusion results of each branch are weighted and aggregated according to the set of branch weights to obtain the aggregated fused image; The intensity range of the aggregated and fused image is constrained to obtain the fused medical image.

[0014] Optionally, in the above scheme, the method further includes: The intensity preservation loss is determined based on the intensity preservation relationship between the fused medical image and each of the medical images to be fused; Multiple directional gradient components corresponding to each of the medical images to be fused are determined to obtain the directional gradient information corresponding to each of the medical images to be fused. The complete gradient activity of each of the medical images to be fused is determined based on the directional gradient information, and the corresponding gradient target mode is determined based on the complete gradient activity to obtain gradient target information; wherein, multiple directional gradient components in the gradient target information come from the same gradient target mode; The directional gradient preservation loss is determined based on multiple directional gradient components of the fused medical image and the gradient target information; The fusion training loss is determined based on the strength preservation loss and the orientation gradient preservation loss; The learnable parameters in the image fusion method based on paired mean residual decomposition and statistical weighting are jointly updated in a single stage according to the fusion training loss to obtain the updated image fusion parameters; wherein, the learnable parameters include mean kernel parameters, statistical combination parameters for determining the mean statistical response and the residual statistical response, and branch aggregation parameters for determining the branch weights.

[0015] Secondly, an image fusion system based on paired mean residual decomposition and statistical weighting, the system comprising: The image acquisition module is used to acquire at least two medical images to be fused. The pairing decomposition module is used to perform mean processing on each of the medical images to be fused based on multiple mean kernels with different scales and / or directions, to obtain the mean components corresponding to each mean kernel, and to determine the residual components based on the difference between each medical image to be fused and the corresponding mean components, thereby forming multiple paired mean residual branches corresponding to each mean kernel. The statistical weighting module is used to determine the mean statistical response of the mean component and the residual statistical response of the residual component corresponding to different medical images to be fused in each of the paired mean residual branches, and to determine the mean mode weight and the residual mode weight according to the mean statistical response and the residual statistical response, respectively. The branch fusion module is used to fuse the mean components corresponding to different medical images to be fused according to the mean mode weight for each of the paired mean residual branches to obtain fused mean components, and fuse the residual components corresponding to different medical images to be fused according to the residual mode weight to obtain fused residual components. The fused mean components and the fused residual components are combined to obtain the corresponding branch fusion result. The branch weight determination module is used to determine the branch activity degree based on the mean statistical response and residual statistical response corresponding to each of the paired mean residual branches, and to determine the corresponding branch weight based on the branch activity degree. The fused image generation module is used to aggregate the fusion results of the corresponding branches according to the weights of each branch to obtain a fused medical image.

[0016] Compared with the prior art, this application has at least the following beneficial effects: Based on further analysis and research of existing technical problems, this application recognizes the difficulties in coordinating the preservation of structural and detail information, the difficulty in adaptively utilizing effective information at different scales and orientations, and the high complexity and computational cost of fusion models during multimodal medical image fusion. This application addresses these issues by applying mean processing to the medical images to be fused using multiple mean kernels with different scales and / or orientations. The residual components are determined using the difference between the medical image to be fused and its corresponding mean component, forming paired mean-residual branches with a defined correspondence between the mean and residual components. This avoids the inconsistencies that can easily arise when structural and detail components are extracted independently. Furthermore, the application separately determines the mean statistical response of the mean component and the residual statistical response of the residual component, and accordingly determines the mean modal weight and residual modal weight. This allows different medical images to participate in the fusion process based on their respective contributions to structural and detail information, thus taking into account tissue structure, grayscale contrast, and lesion boundaries. The method preserves local textures. Based on this, it fuses the mean and residual components separately and combines them to form branch fusion results for each paired mean-residual branch. This allows structural and detail information to be fused with a clear correspondence. Furthermore, it determines branch activity and weights based on the mean and residual statistical responses of each paired mean-residual branch, adaptively aggregating the branch fusion results at different scales and / or directions. This ensures that scales and directions with greater actual information contributions receive corresponding fusion contributions, thus reducing information omissions or redundancy caused by fixed aggregation methods, improving the utilization of effective information at different scales and directions, and achieving coordinated preservation of structural and detail information in medical images without relying on complex fusion networks for information selection and combination. This solves the problems of difficulty in simultaneously preserving structural and detail information, difficulty in adaptively utilizing effective information at different scales and directions, and high complexity in existing medical image fusion processes. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an image fusion method based on paired mean residual decomposition and statistical weighting, provided in one embodiment of this application. Figure 2 This application provides a flowchart of an image fusion process based on paired mean residual decomposition and statistical weighting, as an embodiment of the present application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] The following is combined with Figure 2 This paper describes the specific implementation of an image fusion method and system based on paired mean residual decomposition and statistical weighting. It should be noted that, for consistency of terminology, the medical images involved in the fusion will be uniformly referred to as "medical images to be fused" and the final medical image will be uniformly referred to as "fused medical images." The medical images to be fused can be CT, MRI, PET, SPECT, or ultrasound images, or they can be medical images obtained from the same medical imaging modality through different scanning sequences or different scanning parameters. Therefore, the term "modal weight" used below is used to represent the information contribution of different medical images to be fused in the corresponding fusion process, and is not limited to the requirement that the medical images to be fused must originate from different medical imaging modalities. Figure 2 Modality A and Modality B represent the two medical images to be fused, corresponding to the two values ​​of the input image index below; this figure is an example of two input images, and the number of images to be fused can be more than two.

[0020] Figure 2 From left to right, the process of registration and normalization, directional mean kernel processing, paired mean residual decomposition, statistical weighting, branch reconstruction, activity aggregation, and finally forming the overall processing relationship of the medical images to be fused into the fused medical images is shown. Figure 2 The lower section further illustrates the processing relationship where intensity loss and gradient loss are established solely based on the final fused medical images, and training is performed using a single-stage optimization approach.

[0021] In one embodiment, such as Figure 1 As shown, an image fusion method based on paired mean residual decomposition and statistical weighting is provided to acquire at least two medical images to be fused. These at least two images provide complementary medical imaging information; for example, different images can highlight tissue background, grayscale distribution, larger-scale anatomical structures, tissue edges, lesion boundaries, or local textures, respectively. For medical image processing of the same examination subject, images located in the same anatomical region and with established spatial correspondence can be selected as images to be fused, so that the same spatial location in different images can participate in subsequent statistical evaluation and fusion processing.

[0022] like Figure 2 As shown, multiple mean kernels with different scales and / or orientations are set for each medical image to be fused. Different mean kernels correspond to image responses of different scales and / or orientations, thus forming multiple processing branches. For the first... Medical images to be fused and the Each mean kernel The mean kernel is used to perform mean processing on the medical images to be fused to obtain the mean components. The residual component is obtained by subtracting the corresponding mean component from the medical image to be fused. The correspondence is as follows:

[0023]

[0024] And satisfy:

[0025] Therefore, the mean and residual components of different medical images to be fused corresponding to the same mean kernel together form a paired mean-residual branch. The residual components are not obtained through learnable residual kernels that are independent of the mean kernel, but are determined by the difference between the medical image to be fused and the corresponding mean component. Therefore, there is a clear pairing relationship between the mean component and the residual component.

[0026] In the above formula, For indexing medical images to be fused, The index of the mean kernel and its corresponding paired mean residual branches; superscript This indicates the branch number, not the exponentiation operation. and The first The medical images to be fused in the first The mean and residual component images in each branch have the same pixel location correspondence with the corresponding medical image to be fused. Additions and subtractions between images or components, as well as subsequent multiplications between weights and components, are all performed at the corresponding pixel locations. Since the residual component is obtained by subtracting its mean component from the input image, the pixel values ​​of the two types of components at any identical pixel location are added together to recover the pixel value of the input image at that location. Figure 2 In , , These are the medical image to be fused after omitting the input image index and branch index, the mean component, and the residual component, respectively; "Mean" and "Residual" represent the mean component and the residual component, respectively, and "same kernel pairing" means that they correspond to the same mean kernel.

[0027] For each paired mean-residual branch, the mean and residual components corresponding to different medical images to be fused are evaluated. Specifically, the local statistical characteristics of the mean component are evaluated to obtain the mean statistical response; the local statistical characteristics of the residual component are evaluated to obtain the residual statistical response. The mean statistical response is mainly used to characterize the local information contribution of different medical images to be fused in terms of tissue background, grayscale distribution, and large-scale anatomical structures, while the residual statistical response is mainly used to characterize the local information contribution of different medical images to be fused in terms of tissue edges, lesion boundaries, local texture, and detail changes.

[0028] Based on the mean statistical responses of different medical images to be fused within the same paired mean residual branch, the mean mode weights for each medical image to be fused are determined; based on the corresponding residual statistical responses, the residual mode weights for each medical image to be fused are determined. The mean mode weights and residual mode weights are independent of each other, thus allowing the same medical image to make different information contributions during mean component fusion and residual component fusion. For example, one medical image to be fused may contribute significantly to the tissue background or anatomical structure, while another may contribute significantly to the lesion boundaries or texture details.

[0029] For each paired mean residual branch, the mean components corresponding to different medical images to be fused are fused according to the mean modality weight to obtain the fused mean component; the residual components corresponding to different medical images to be fused are fused according to the residual modality weight to obtain the fused residual component; then the fused mean component and the fused residual component are combined to obtain the branch fusion result corresponding to the paired mean residual branch. Figure 2 The specific implementation shown in the figure completes the branch reconstruction by directly adding the fused mean component and the fused residual component, so there is no need to set up a separate image encoder or decoder to re-restore the image of the fused features.

[0030] After obtaining multiple branch fusion results, the statistical activity of each branch is evaluated based on the mean statistical response and residual statistical response corresponding to each paired mean residual branch, thus obtaining the corresponding branch activity. The corresponding branch weights are determined based on the branch activity of different paired mean residual branches, and the multiple branch fusion results are aggregated according to each branch weight. Therefore, based on the actual statistical responses of anatomical structures, lesion regions, and edge directions at different spatial locations, the contribution of paired mean residual branches at different scales and / or directions to the fusion results can be adaptively adjusted to obtain a fused medical image.

[0031] This embodiment utilizes the same mean kernel to form mean and residual components with a defined correspondence, enabling structural and detail components to jointly characterize the medical images to be fused. Furthermore, by statistically evaluating the mean and residual components and determining their modal weights, different medical images to be fused can participate in the fusion process according to their actual contributions to structural and detail information. Based on this, branch contributions are determined according to the statistical activity of branches at different scales and / or directions, thus preserving lesion boundaries and local textures while maintaining tissue structure and grayscale contrast, and reducing information omissions or redundancy caused by fixed-branch aggregation. Simultaneously, branch fusion results can be directly formed within a branch based on the pairing relationship between the mean and residual components, reducing dependence on complex encoders, decoders, and image reconstruction networks, thereby lowering model complexity and improving the interpretability and verifiability of the fusion process.

[0032] In this embodiment, at least two original medical images corresponding to the same examination object and the same anatomical region are acquired to form an original medical image set. The original medical images can come from different medical imaging methods such as CT, MRI, PET, SPECT, or ultrasound, or from different scanning sequences or different scanning parameters under the same medical imaging method. For example, different MRI scan sequences of the same examination object can be selected as original medical images, or MRI images and CT images of the corresponding region of the same examination object can be selected as original medical images.

[0033] After acquiring the original medical image set, spatial registration is performed on each original medical image to establish a correspondence between the positions of the same anatomical structures in different original medical images, resulting in a registered medical image corresponding to the spatial location. Spatial registration is used to avoid positional deviations in subsequent local statistical responses and fusion results caused by inconsistencies in the acquisition locations or spatial coordinates of different medical images.

[0034] Subsequently, the registered medical images are subjected to size unification and intensity normalization processing to ensure that each registered medical image has a corresponding image size and an intensity range suitable for subsequent statistical comparison and weighted fusion, thereby obtaining the medical images to be fused. Figure 2 The "registration normalization" part refers to the above processing procedure before the original medical image enters the directional mean kernel processing.

[0035] This embodiment performs spatial registration, size unification, and intensity normalization sequentially on original medical images of the same examination object and the same anatomical region. This makes different medical images to be fused comparable in terms of spatial location, image size, and data range. This provides a unified data foundation for subsequent paired mean residual decomposition based on the same mean kernel and for comparing the local statistical responses of different medical images to be fused, reducing the interference caused by differences in the spatial location and intensity range of the input images themselves on the fusion results.

[0036] In this embodiment, multiple directional anchor kernels are constructed based on multiple preset scales and multiple preset directions to form a set of directional anchor kernels. These directional anchor kernels are used to perform mean processing on the medical images to be fused at different scales and directions, enabling different paired mean residual branches to respond to medical image structures of different spatial ranges and directions. Preferably, three scales and four directions can be set, thereby forming twelve directional mean kernels, corresponding to… Figure 2 The multi-scale, multi-directional mean kernel and subsequent branches are shown in the figure.

[0037] For the Each mean kernel Its nuclear elements satisfy nonnegativity and normalization constraints:

[0038] in, This represents the position of the kernel element in the mean kernel. By ensuring that all kernel elements are non-negative and that the sum of the kernel elements is 1, the mean kernel can maintain the basic properties of local mean processing. Figure 2 The non-negativity constraint of the mean kernel applies to each kernel element, while the constraint that sums to 1 applies to all kernel elements of the same mean kernel; the summation symbol in the figure omits the kernel element position index.

[0039] For each directional anchor kernel, the corresponding learnable kernel parameters are non-negatively normalized to obtain the corresponding learnable normalized kernel. The mean kernel is obtained by combining the directional anchor kernel and the learnable normalized kernel, and their correspondence is as follows:

[0040] in, Indicates the first A directional anchor core, This represents the learnable normalized kernel obtained based on the corresponding learnable kernel parameters. The limited adjustment ratio is used to restrict the degree of adjustment of the learnable normalized kernel relative to the directional anchor kernel. Therefore, the mean kernel can be adjusted to a limited extent based on medical image data while preserving a preset scale or directional prior. Indicates the first The set of learnable kernel parameters corresponding to each mean kernel is used. Here, softmax normalizes the parameters of each kernel element in this set to obtain a non-negative learnable normalized kernel whose sum is 1. The kernel elements of the directional anchor kernel are non-negative, and the sum of all kernel elements of the same directional anchor kernel is 1. The restricted adjustment ratio takes values ​​between 0 and 1 to ensure that the combined mean kernel still satisfies the constraint that the kernel elements are non-negative and the sum of the kernel elements is 1.

[0041] Since the mean kernel satisfies the above normalization constraints, its corresponding residual kernel can be expressed as:

[0042] in, This represents the unit pulse kernel. The residual kernel has a zero-sum property. Therefore, the mean kernel is used to obtain local structural components, while the corresponding residual reflects the detailed changes relative to the local mean, and there is a definite mathematical correspondence between the two. When the unit pulse kernel is convolved with the input image, the input image remains unchanged; therefore, the result of convolving the residual kernel with the input image is equal to the input image minus the mean component, and the sum of all kernel elements is 0.

[0043] This embodiment combines the directional anchor kernel and the learnable normalization kernel according to a restricted adjustment ratio. This allows the mean kernel to retain the prior characteristics of different scales and directions while being able to make limited parameter adjustments based on medical image data. At the same time, by applying non-negativity and normalization constraints to the mean kernel, the mean kernel maintains a clear local mean processing meaning, and the residual kernel determined by it has zero-sum characteristics. This reduces the possibility of unconstrained degradation of the mean kernel and residual components, and improves the interpretability and verifiability of the multi-scale, multi-directional paired decomposition process.

[0044] In this embodiment, for any target mean kernel, mean filtering is performed on each medical image to be fused using the target mean kernel. Taking the first... Medical images to be fused and the Target mean kernel For example, its target mean component is:

[0045] in, This indicates that the mean value is applied to the medical images to be fused using the corresponding mean kernel.

[0046] After obtaining the target mean component, instead of setting a learnable residual kernel independent of the target mean kernel, the difference between the medical image to be fused and the corresponding target mean component is directly calculated to obtain the corresponding target residual component.

[0047] Therefore, the target mean component and the target residual component corresponding to the same medical image to be fused satisfy:

[0048] For the same target mean kernel, the above processing is performed on all medical images to be fused, and each target mean component and each target residual component corresponding to the target mean kernel are associated according to the corresponding medical images to be fused, thereby forming a paired mean residual branch corresponding to the target mean kernel. Different target mean kernels form different paired mean residual branches, and multiple paired mean residual branches are formed by multiple target mean kernels of different scales and / or directions.

[0049] In the above processing relationship, each target residual component is uniquely determined by the corresponding medical image to be fused and its target mean component. Therefore, the target mean component and target residual component of the same medical image to be fused in a paired mean residual branch can accurately recover the corresponding medical image to be fused, without needing to re-establish the correspondence between them through another set of independent low-pass and high-pass filtering relationships. The above accurate recovery relationship refers to the paired components of the same medical image to be fused before fusion, representing pixel-by-pixel addition recovery, not the summation of pixel values ​​of the entire image, nor the summation of components from different medical images to be fused to recover a single input image.

[0050] This embodiment generates a target mean component using a target mean kernel, and obtains a unique corresponding target residual component by subtracting the target mean component from the medical image to be fused. This ensures that each medical image to be fused satisfies the definite relationship that the sum of the pixel values ​​of the mean component and the residual component at the same pixel position in each paired mean residual branch is equal to the pixel value of the original medical image to be fused at that pixel position. This avoids the inconsistency in decomposition relationships that may occur when setting independent low-pass kernels and high-pass kernels separately, which helps to reduce grayscale shift, loss of detail, and local reconstruction errors. It also provides input components with clear mathematical relationships for the subsequent separate statistical analysis and fusion of structural information and detail information.

[0051] In this embodiment, for each mean component, the local mean and local second moment in its corresponding local region are determined, and the local variance is determined based on the local mean and local second moment, thereby obtaining the local statistical information of the mean. The mean component is used to represent the local structure obtained after processing with the mean at the corresponding scale and direction. Therefore, its local mean and local variance can reflect the local statistical characteristics of the tissue background, grayscale distribution, and larger-scale anatomical structures.

[0052] By statistically combining the local mean and local variance from the local statistical information of the mean, we obtain the mean statistical response:

[0053] in, Represents the local mean. Represents local variance. and These are non-negative normalized statistical coefficients. The mean statistical response is used to evaluate the local information contribution of the corresponding medical image to be fused in terms of tissue background, grayscale distribution, and larger-scale anatomical structures. Indicates the first The medical images to be fused in the first The mean statistical response in each branch; the local second moment is the second moment obtained by performing local statistics on the squares of the pixel values ​​of the mean component, and the local variance is the local second moment minus the square of the local mean. and are the statistical combination coefficients of the local mean square term and the local variance term, respectively, both of which are non-negative and sum to 1. Figure 2 middle , and These correspond to the mean statistical response, local mean, and local variance, respectively, after omitting the input image index and branch index, as shown in the figure. It represents the square of the local mean.

[0054] For each residual component, the local response of the absolute value of the residual component and the local quadratic response of the residual component within the corresponding local region are determined to obtain the local statistical information of the residual. Since the residual component represents the local change of the medical image to be fused relative to the corresponding mean component, the local response and local quadratic response of the absolute value of the residual component can be used to reflect the local statistical characteristics of tissue edges, lesion boundaries, local texture, and detail changes.

[0055] The residual statistical response is obtained by statistically combining the local responses of the absolute values ​​of the residual components and the local quadratic responses of the residual components:

[0056] in, This represents the local absolute residual response. This represents the local quadratic residual response. and These are non-negative normalized statistical coefficients. The residual statistical response is used to evaluate the information contribution of the corresponding medical image to be fused in terms of tissue edges, lesion boundaries, local texture, and detail changes. Indicates the first The medical images to be fused in the first The residual statistical response in each branch; the local absolute residual response is the response obtained by performing local statistics on the absolute values ​​of the residual components, and the local quadratic residual response is the response obtained by performing local statistics on the squares of the residual components. and are the statistical combination coefficients of the squared term of the local absolute residual response and the local quadratic residual response, respectively, both of which are non-negative and their sum is 1. Figure 2 In , and These correspond to the residual statistical response, local absolute residual response, and local quadratic residual response, respectively, after omitting the input image index and branch index. This represents the square of the local absolute residual response.

[0057] This embodiment constructs local statistical responses for different information attributes of the mean and residual components. The mean statistical response focuses on reflecting the contributions of tissue background, grayscale distribution, and larger-scale anatomical structures, while the residual statistical response focuses on reflecting the contributions of tissue edges, lesion boundaries, texture, and detail changes. This avoids using the same statistical index to uniformly evaluate structural and detail components, allowing the subsequent modal weights to be determined based on the actual statistical characteristics of the corresponding image components. This is beneficial for simultaneously preserving structural information and lesion detail information in the fused medical image.

[0058] In this embodiment, for the same paired mean residual branch, the mean modal weight is determined based on the mean statistical response corresponding to different medical images to be fused. For the first... The medical images to be fused in the first The mean statistical response in each paired mean residual branch is determined by the following formula:

[0059] in, For the first The medical images to be fused in the first Mean mode weights in each branch; The mean statistical response of the image in this branch; For the first The mean statistical response of the medical images to be fused in the same branch. The input image index is used for summing the denominator, and the summation iterates through all the medical images to be fused in this branch. A positive numerical stabilizing term is used to keep the base of the exponentiation operation positive and to avoid the normalized denominator being zero when all statistical responses are zero. This is the power-law adjustment parameter for the mean statistical response, used to adjust the relative differences when the statistical response is converted into modal weights. Normalization is performed across input images at the same branch and the same pixel location, with the numerator and denominator using the same... and .

[0060] Through the above processing, the mean statistical response corresponding to different medical images to be fused in the same paired mean residual branch is non-negatively normalized to obtain the mean modal weight set, and the sum of the mean modal weights is 1.

[0061] For the same paired mean residual branch, the residual mode weights are determined based on the residual statistical responses corresponding to different medical images to be fused. For the first... The medical images to be fused in the first The residual statistical response in each paired mean residual branch is determined by the following formula:

[0062] in, For the first The medical images to be fused in the first Residual mode weights in each branch; The residual statistical response of the image in this branch; For the first The residual statistical response of medical images to be fused in the same branch. Its summation range is the same as that of the mean-mode weight formula. This is the aforementioned positive numerical stability term; This is the power-law adjustment parameter for the residual statistical response, used to adjust the relative differences when the residual statistical response is converted into modal weights. This normalization is also performed across input images at the same branch and the same pixel location, using the same power-law adjustment for both the numerator and denominator. and ; and They are applied to the mean statistical response and the residual statistical response, respectively.

[0063] Through the above processing, the residual statistical responses corresponding to different medical images to be fused are non-negatively normalized to obtain a set of residual mode weights, and the sum of the weights of each residual mode is 1.

[0064] The mean mode weight set and the residual mode weight set are independent of each other. Therefore, for the same medical image to be fused, the mean mode weight and the residual mode weight do not need to be the same. The mean mode weight is used to control the contribution of the mean component in the medical image to the fusion mean component, and the residual mode weight is used to control the contribution of the residual component in the medical image to the fusion residual component. Thus, different medical images to be fused can participate in the fusion separately based on their actual statistical responses in structural and detail information.

[0065] This embodiment generates two sets of independent non-negative normalized modal weights based on the mean statistical response and the residual statistical response, respectively. This allows the same medical image to be fused to have different information contributions in structural component fusion and detail component fusion. This avoids the information trade-off caused by a certain medical image having advantages in structure but not in detail when using a unified fusion weight. It is beneficial to maintain tissue structure, grayscale contrast, lesion boundaries and local texture at the same time.

[0066] In this embodiment, for any paired mean residual branch, the mean components corresponding to different medical images to be fused in the paired mean residual branch are weighted and fused according to the mean modality weight set to obtain the fused mean component. The fusion mean component in each paired mean residual branch is:

[0067] in, Indicates the first The mean modal weights corresponding to the medical images to be fused. Indicates the first The medical images to be fused in the first The mean component in each paired mean residual branch. The summation is performed on the fusion mean component of this branch, traversing all medical images to be fused. Figure 2 The mean fusion formula in the image is a simplified version of the two input images: and Let these represent the mean components of mode A and mode B in the same branch, respectively. and These represent the corresponding mean-mode weights. This represents the fused mean component of the branch; the branch index is omitted in this illustrated formula.

[0068] Based on the residual modal weight set, the residual components corresponding to different medical images to be fused in the same paired mean residual branch are weighted and fused to obtain the fused residual components:

[0069] in, Indicates the first The residual mode weights corresponding to the medical images to be fused Indicates the first The medical images to be fused in the first The residual components in each paired mean residual branch. For the fusion residual components of this branch, sum them up and iterate through all the medical images to be fused. Figure 2The “Mean fusion” and “Residual fusion” in the text correspond to the mean component weighted fusion and residual component weighted fusion mentioned above, respectively, and each uses the corresponding modal weights.

[0070] After obtaining the fusion mean component and the fusion residual component, the branch fusion result of the current branch is formed by using the reconstruction method corresponding to the aforementioned pairing decomposition relationship between the two. Figure 2 In the embodiment shown, the fusion mean component and the fusion residual component are directly added together:

[0071] in, Indicates the first The branch fusion results correspond to each paired mean residual branch. This process does not require a separate encoder or decoder to restore the high-dimensional fusion features to a medical image; instead, the branch fusion results are directly formed from the statistically weighted fusion mean component and the fusion residual component. Figure 2 The addition symbol in the figure indicates that the fused mean component and the fused residual component are added at the corresponding pixel positions; Indicates the first The branch fusion results for each branch have the same sign as the branch fusion results in the main text. The branch result sequence in the figure is... to This represents the fusion result of the twelve branches formed by the aforementioned three scales and four directions.

[0072] This embodiment fuses structural and detail components using mean mode weights and residual mode weights within the same paired mean residual branch, and then directly combines the fused mean component and the fused residual component to form the branch fusion result. This makes the branch fusion result directly correspond to the aforementioned statistical weighting process. It can simultaneously introduce structural and detail information from different medical images to be fused, shorten the processing path from the input medical image to the branch fusion result, reduce information loss that may be caused by complex feature mapping and independent decoding and reconstruction, and reduce the number of model parameters and computational overhead.

[0073] In this embodiment, after obtaining the branch fusion results of each paired mean residual branch, the activity level of the corresponding branch is further evaluated based on the mean statistical response and residual statistical response of each paired mean residual branch.

[0074] Regarding the first Each paired mean residual branch aggregates the mean statistical responses corresponding to different medical images to be fused, obtaining aggregated mean statistical response values; it also aggregates the residual statistical responses corresponding to different medical images to be fused, obtaining aggregated residual statistical response values; subsequently, it combines the aggregated mean statistical response values ​​and the aggregated residual statistical response values ​​to obtain the corresponding branch activity.

[0075] in, Indicates the first Branch activity corresponding to each paired mean residual branch and This refers to the combination coefficient in the calculation of the branch activity degree. (Two points...) The summation is performed by iterating through all the medical images to be fused in the same branch. and Adjust the contributions of the aggregated mean statistical response and the aggregated residual statistical response to the branch activity, respectively. Figure 2 In Indicates the first The branch activity of each branch is indicated by the same sign as the branch activity in the main text, and its meaning is distinct from that of the local absolute residual response. .

[0076] The branch activity is normalized based on the branch activity corresponding to each paired mean residual branch, forming a set of branch weights. For example... Figure 2 As shown, in one specific implementation, the softmax function is used to convert branch activity into non-negative normalized branch weights:

[0077] in, Indicates the first The branch weights correspond to the paired mean residual branches. Here, softmax normalizes the activity of all branches at the same pixel position: it takes the natural exponent of the activity of each branch, and then divides the exponent value of the current branch by the sum of the exponent values ​​of all branches to obtain the weight of the current branch. Thus, the weights of each branch are non-negative and their sum at the same pixel position is 1. Figure 2 In For the first The branch weights of each branch have the same sign as the branch weights in the main text; the normalization here applies to different branches, unlike the normalization of kernel element parameters when constructing the mean kernel.

[0078] After obtaining the set of branch weights, the fusion results of the corresponding branches are weighted and aggregated according to the weights of each branch to obtain the aggregated fused image:

[0079] Since different paired mean residual branches correspond to different scales and / or directions, the branch weights formed based on the branch activity can enable the scale or direction branch with a more obvious actual statistical response to obtain a fusion contribution corresponding to its activity level, rather than uniformly processing all branches by selecting or preset fixed branch weights according to fixed average, maximum value, or fixed branch weights. Figure 2 The branch-weighted summation formula in the above formula represents the same processing relationship, where for The summation method involves iterating through all paired mean residual branches and multiplying the branch weights by the branch fusion results at the corresponding pixel positions before summing them up.

[0080] After obtaining the aggregated and fused image, an intensity range constraint is applied to it. In the implementation where the medical image to be fused has been normalized, the fused medical image is determined according to the following formula:

[0081] in, Indicates the aggregation and fusion of images. This represents a fused medical image obtained after intensity range constraints. Here, `clip` represents a pixel-by-pixel intensity range truncation operation, with 0 and 1 being the lower and upper limits of the output pixel values, respectively: pixel values ​​less than 0 in the aggregated fused image are set to 0, pixel values ​​greater than 1 are set to 1, and pixel values ​​within the range of 0 to 1 remain unchanged. Figure 2 In and Consistent with the meaning here.

[0082] This embodiment aggregates the mean statistical response and residual statistical response in each paired mean residual branch, and uses the two types of statistical responses to jointly determine the branch activity, so that the branch evaluation considers both structural and detailed information. Furthermore, it uses the branch activity to form normalized branch weights and adaptively aggregates the branch fusion results of different scales and / or directions, so that different anatomical structures, lesion morphologies and edge directions can select more effective branch contributions based on the actual local statistical response, thereby reducing information omissions or branch redundancy that may be caused by fixed aggregation methods, and improving the adaptability of the fusion method to medical image features of different scales and directions.

[0083] In this embodiment, the training objective is established only for the final obtained fused medical image; no pseudo-labels or independent supervision are set for the intermediate mean component, residual component, or the fusion results of each branch. For example... Figure 2 As shown in the lower part, during the training process, the intensity preservation loss and orientation gradient preservation loss are determined based on the final fused medical image, and the two types of losses are used to form the fusion training loss.

[0084] Among them, the intensity preservation loss is determined based on the intensity preservation relationship between the fused medical image and each medical image to be fused. The intensity preservation loss is used to constrain the preservation of effective intensity information in the medical image to be fused in the final fused medical image. Figure 2 The training objective is represented by "intensity loss".

[0085] For the gradient-preserving loss, multiple gradient components are determined for each medical image to be fused, forming corresponding gradient information. Further, the complete gradient activity of each medical image to be fused is determined based on the gradient information, and the corresponding gradient target mode is determined based on the complete gradient activity, obtaining gradient target information. After determining the gradient target mode, multiple gradient components of that gradient target mode are simultaneously selected as gradient target information, instead of selecting gradient components from different directions from different medical images to be fused. This ensures that the multiple gradient components used as training targets originate from the same gradient target mode, maintaining the consistency of the target gradient direction.

[0086] The gradient preservation loss is determined based on multiple directional gradient components and gradient target information from the fused medical images. . Figure 2 The training objective is referred to as "gradient loss". It should be noted that the above implementation method selects the corresponding medical image to be fused using the full gradient activity, and selects the gradient components of the medical image to be fused in two directions at the same time, instead of selecting gradient components of different directions from different medical images to be fused.

[0087] The fusion training loss is determined based on the strength preservation loss and the orientation gradient preservation loss. Figure 2 The given relationship for fusion training loss is as follows:

[0088] in, Indicates the fusion training loss, Indicates strength retention loss, This represents the gradient preservation loss. This represents the weighting coefficient of the gradient preservation loss in the fusion training loss. Figure 2 The loss formula in the code represents adding the intensity preservation loss to the weighted gradient-preserving loss to form a unified fusion training loss; the weight coefficients... Constrained adjustment ratio in the construction of mean kernel Different parameters are used for training loss combinations and mean kernel combinations, respectively.

[0089] Based on the fusion training loss, the learnable parameters in the image fusion process based on paired mean residual decomposition and statistical weighting are jointly updated in a single stage to obtain the updated image fusion parameters. The learnable parameters include mean kernel parameters, statistical combination parameters for determining the mean statistical response and residual statistical response, and branch aggregation parameters for determining branch weights. These parameters are uniformly optimized in the same training process based on the fusion training loss corresponding to the final fused medical image, without separately training the mean component branch, residual component branch, and branch fusion result.

[0090] Figure 2 The training loop is shown in dashed lines; that is, the training loss generated from the final fusion of medical images is used inversely to optimize the mean kernel and related fusion parameters. In a preferred embodiment, the image fusion process based on paired mean residual decomposition and statistical weighting is mainly completed through directional mean kernels, local statistical parameters, modal weights, and branch weights. There are fewer than 500 trainable parameters, eliminating the need for a large-scale feature extraction network and a separate image reconstruction network.

[0091] This embodiment establishes intensity preservation loss and directional gradient preservation loss solely based on the final fused medical image, and utilizes these two to form a unified fusion training loss, ensuring that the model optimization objective directly corresponds to the final fusion result. Simultaneously, by selecting multiple directional gradient components of the same gradient objective mode through complete gradient activity, it helps maintain the directional consistency of the gradient objective. Furthermore, by uniformly updating the mean kernel parameters, statistical combination parameters, and branch aggregation parameters in a single-stage manner, it eliminates the need to set additional pseudo-labels or independent supervision for intermediate mean components, residual components, and branch fusion results. This reduces the training complexity caused by pre-training, multi-stage training, and multiple intermediate supervision objectives, and mitigates the impact of inconsistencies between different supervision objectives on the final fusion result.

[0092] In one implementation, the image fusion system based on paired mean residual decomposition and statistical weighting includes an image acquisition module, a paired decomposition module, a statistical weighting module, a branch fusion module, a branch weight determination module, and a fused image generation module. Each module is configured according to... Figure 2 The image fusion processing flow based on paired mean residual decomposition and statistical weighting is shown to transmit the corresponding processing results.

[0093] The image acquisition module is used to acquire at least two medical images to be fused. The medical images to be fused can be spatially registered, size-uniformed, and intensity-normalized medical images, which can originate from different medical imaging methods such as CT, MRI, PET, SPECT, or ultrasound, or from different scan sequences or different scan parameters under the same medical imaging method.

[0094] The paired decomposition module performs mean processing on each medical image to be fused based on multiple mean kernels with different scales and / or orientations, obtaining the mean component corresponding to each mean kernel. For each medical image to be fused, it determines the residual component based on the difference between the medical image to be fused and the corresponding mean component, thus forming multiple paired mean residual branches corresponding to each mean kernel. The mean components and residual components of each medical image to be fused corresponding to the same mean kernel belong to the same paired mean residual branch, and the mean components and residual components corresponding to each medical image to be fused have a defined pairing relationship.

[0095] The statistical weighting module is used to determine the mean statistical response and residual statistical response of the mean component and residual component of different medical images to be fused in each paired mean residual branch, and to determine the mean mode weight and residual mode weight based on the mean statistical response and residual statistical response, respectively. The mean statistical response is used to evaluate the contribution of structural information in the corresponding medical image to be fused, and the residual statistical response is used to evaluate the contribution of edge, lesion boundary, texture and detail information in the corresponding medical image to be fused. The mean mode weight and residual mode weight are determined independently based on the corresponding statistical response.

[0096] The branch fusion module is used to fuse the mean components corresponding to different medical images to be fused according to the mean mode weights for each paired mean residual branch, obtaining the fused mean component. Then, it fuses the residual components corresponding to different medical images to be fused according to the residual mode weights, obtaining the fused residual component. Finally, the fused mean component and the fused residual component are combined to obtain the corresponding branch fusion result. Figure 2 In the embodiment shown, the fused mean component and the fused residual component are directly added together to form the branch fusion result, without the need to set up a separate encoder or decoder for image reconstruction.

[0097] The branch weight determination module is used to determine the branch activity based on the mean statistical response and residual statistical response corresponding to each paired mean residual branch, and to determine the corresponding branch weight based on the branch activity. Thus, paired mean residual branches at different scales and / or directions can obtain corresponding contributions based on the actual image statistical activity, instead of using a fixed average branch weight.

[0098] The fused image generation module aggregates the fusion results of the corresponding branches according to their weights to obtain a fused medical image. In an implementation that employs intensity normalization, the aggregated fused image can also be restricted to a corresponding intensity range according to the aforementioned intensity range constraint relationship, thereby outputting a fused medical image.

[0099] The specific calculation relationships involved in the above modules, such as mean kernel construction, paired mean residual decomposition, mean statistical response, residual statistical response, mean modal weight, residual modal weight, intra-branch fusion, branch activity, branch weight, and final fused medical image, can all adopt the corresponding processing methods described in the aforementioned embodiments, and will not be repeated here.

[0100] This embodiment sequentially completes the acquisition of the medical image to be fused, the pairing decomposition of structure and detail, the independent statistical weighting of the two types of components, the branch fusion, and the cross-scale and / or cross-directional adaptive aggregation through an image acquisition module, a pairing decomposition module, a statistical weighting module, a branch fusion module, a branch weight determination module, and a fused image generation module. This enables the system to achieve image fusion based on pairing mean residual decomposition and statistical weighting without relying on large-scale encoders, decoders, and complex attention networks. The mean kernel, residual components, statistical responses, modal weights, and branch weights all have clear processing meanings. This reduces model complexity, training costs, and computational resource requirements while maintaining the ability to preserve anatomical structure and lesion details, and improves the interpretability, verifiability, and ease of implementation of the image fusion process based on pairing mean residual decomposition and statistical weighting.

[0101] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. An image fusion method based on paired mean residual decomposition and statistical weighting, characterized in that, The method includes: At least two medical images to be fused are acquired. The average of each medical image to be fused is processed based on multiple mean kernels with different scales and / or orientations to obtain the mean components corresponding to each mean kernel. The residual components are determined based on the difference between each medical image to be fused and the corresponding mean components, forming multiple paired mean residual branches corresponding to each mean kernel. The mean statistical response of the mean component and the residual statistical response of the residual component corresponding to different medical images to be fused in each of the paired mean residual branches are determined respectively, and the mean mode weight and the residual mode weight are determined respectively based on the mean statistical response and the residual statistical response. For each of the paired mean residual branches, the mean components corresponding to different medical images to be fused are fused according to the mean mode weights to obtain fused mean components, and the residual components corresponding to different medical images to be fused are fused according to the residual mode weights to obtain fused residual components. The fused mean components and the fused residual components are combined to obtain the corresponding branch fusion result. The branch activity is determined based on the mean statistical response and residual statistical response corresponding to each of the paired mean residual branches, and the corresponding branch weight is determined based on the branch activity. The fusion results of the corresponding branches are aggregated according to the weights of each branch to obtain a fused medical image.

2. The image fusion method based on paired mean residual decomposition and statistical weighting according to claim 1, characterized in that, The acquisition of at least two medical images to be fused includes: Acquire at least two original medical images corresponding to the same examination object and the same anatomical region to obtain a set of original medical images; Spatial registration is performed on each original medical image in the original medical image set to obtain a registered medical image corresponding to the spatial location. The registered medical images are subjected to size unification and intensity normalization to obtain the medical images to be fused.

3. The image fusion method based on paired mean residual decomposition and statistical weighting according to claim 1, characterized in that, The multiple mean kernels with different scales and / or orientations are determined in the following manner: Multiple directional anchor point kernels are constructed based on multiple preset scales and multiple preset directions to obtain a set of directional anchor point kernels; For each of the directional anchor point kernels, the corresponding learnable kernel parameters are non-negatively normalized to obtain the learnable normalized kernel. The mean kernel is obtained by combining each directional anchor kernel and the corresponding learnable normalized kernel according to the adjustment ratio; wherein the adjustment ratio is used to limit the degree of adjustment of the learnable normalized kernel relative to the corresponding directional anchor kernel, the kernel elements in each mean kernel are non-negative, and the sum of the kernel elements in each mean kernel is 1.

4. The image fusion method based on paired mean residual decomposition and statistical weighting according to claim 1, characterized in that, The formation of multiple paired mean residual branches corresponding to each of the mean kernels includes: For any target mean kernel, mean filtering is performed on each of the medical images to be fused using the target mean kernel to obtain the target mean component corresponding to each of the medical images to be fused. The difference between each of the medical images to be fused and the corresponding target mean component is calculated to obtain the target residual component corresponding to each of the medical images to be fused. Each target mean component and each target residual component corresponding to the same target mean kernel are associated according to the corresponding medical image to be fused to form a paired mean residual branch corresponding to the target mean kernel; wherein, for each medical image to be fused, the sum of the pixel values ​​of the target mean component and the target residual component at the same pixel position is equal to the pixel value of the medical image to be fused at that pixel position.

5. The image fusion method based on paired mean residual decomposition and statistical weighting according to claim 4, characterized in that, The mean statistical response and the residual statistical response are determined in the following manner: For each of the mean components, the local mean and local second moment in the corresponding local region are determined, and the local variance is determined based on the local mean and the local second moment to obtain the local statistical information of the mean. The mean statistical response is obtained by statistically combining the local mean and local variance in the local statistical information of the mean. For each residual component, the local response of the absolute value of the residual component and the local quadratic response of the residual component in the corresponding local region are determined to obtain the local statistical information of the residual. The residual statistical response is obtained by statistically combining the local response of the absolute value of the residual component and the local quadratic response of the residual component in the local statistical information of the residual.

6. The image fusion method based on paired mean residual decomposition and statistical weighting according to claim 1, characterized in that, The step of determining the mean mode weight and the residual mode weight based on the mean statistical response and the residual statistical response respectively includes: For the same paired mean residual branch, the mean statistical response corresponding to different medical images to be fused is non-negatively normalized to obtain a set of mean modal weights; wherein, the sum of the mean modal weights in the set of mean modal weights is 1; Non-negative normalization is performed on the residual statistical responses corresponding to different medical images to be fused to obtain a set of residual mode weights; wherein the sum of the residual mode weights in the set of residual mode weights is 1.

7. The image fusion method based on paired mean residual decomposition and statistical weighting according to claim 6, characterized in that, The obtained corresponding branch fusion results include: Based on the mean modality weight set, the mean components corresponding to different medical images to be fused in the same paired mean residual branch are weighted and fused to obtain the fused mean components; Based on the residual modal weight set, the residual components corresponding to different medical images to be fused in the same paired mean residual branch are weighted and fused to obtain the fused residual components. The fusion mean component and the fusion residual component are directly added together to obtain the corresponding branch fusion result.

8. The image fusion method based on paired mean residual decomposition and statistical weighting according to claim 1, characterized in that, The step of determining branch activity based on the mean statistical response and residual statistical response corresponding to each of the paired mean residual branches, and determining the corresponding branch weight based on the branch activity, includes: For each of the paired mean residual branches, the mean statistical responses corresponding to different medical images to be fused are aggregated to obtain the aggregated mean statistical response value. The residual statistical responses corresponding to different medical images to be fused are aggregated to obtain the aggregated value of the residual statistical response. The aggregated value of the mean statistical response and the aggregated value of the residual statistical response are combined to obtain the corresponding branch activity. The branch activity degrees corresponding to the multiple paired mean residual branches are normalized to obtain a set of branch weights; The fusion results of each branch are weighted and aggregated according to the set of branch weights to obtain the aggregated fused image; The intensity range of the aggregated and fused image is constrained to obtain the fused medical image.

9. The image fusion method based on paired mean residual decomposition and statistical weighting according to claim 8, characterized in that, The method further includes: The intensity preservation loss is determined based on the intensity preservation relationship between the fused medical image and each of the medical images to be fused; Multiple directional gradient components corresponding to each of the medical images to be fused are determined to obtain the directional gradient information corresponding to each of the medical images to be fused. The complete gradient activity of each of the medical images to be fused is determined based on the directional gradient information, and the corresponding gradient target mode is determined based on the complete gradient activity to obtain gradient target information; wherein, multiple directional gradient components in the gradient target information come from the same gradient target mode; The directional gradient preservation loss is determined based on multiple directional gradient components of the fused medical image and the gradient target information; The fusion training loss is determined based on the strength preservation loss and the orientation gradient preservation loss; The learnable parameters in the image fusion method based on paired mean residual decomposition and statistical weighting are jointly updated in a single stage according to the fusion training loss to obtain the updated image fusion parameters; wherein, the learnable parameters include mean kernel parameters, statistical combination parameters for determining the mean statistical response and the residual statistical response, and branch aggregation parameters for determining the branch weights.

10. An image fusion system based on paired mean residual decomposition and statistical weighting, characterized in that, The system includes: The image acquisition module is used to acquire at least two medical images to be fused. The pairing decomposition module is used to perform mean processing on each of the medical images to be fused based on multiple mean kernels with different scales and / or directions, to obtain the mean components corresponding to each mean kernel, and to determine the residual components based on the difference between each medical image to be fused and the corresponding mean components, thereby forming multiple paired mean residual branches corresponding to each mean kernel. The statistical weighting module is used to determine the mean statistical response of the mean component and the residual statistical response of the residual component corresponding to different medical images to be fused in each of the paired mean residual branches, and to determine the mean mode weight and the residual mode weight according to the mean statistical response and the residual statistical response, respectively. The branch fusion module is used to fuse the mean components corresponding to different medical images to be fused according to the mean mode weight for each of the paired mean residual branches to obtain fused mean components, and fuse the residual components corresponding to different medical images to be fused according to the residual mode weight to obtain fused residual components. The fused mean components and the fused residual components are combined to obtain the corresponding branch fusion result. The branch weight determination module is used to determine the branch activity degree based on the mean statistical response and residual statistical response corresponding to each of the paired mean residual branches, and to determine the corresponding branch weight based on the branch activity degree. The fused image generation module is used to aggregate the fusion results of the corresponding branches according to the weights of each branch to obtain a fused medical image.

Images