Statistical optimization based medical image fusion method and system

Abstract

The application discloses a medical image fusion method and system based on statistical optimization, and relates to the technical field of medical image processing. The method comprises the following steps: acquiring a first medical image and a second medical image with spatial correspondence and performing frequency band decomposition; performing multi-scale feature extraction on low-frequency components and high-frequency components by using different coding depths; determining statistical representation and fusion weights based on local statistical information of multiple local scales to obtain fused low-frequency features and fused high-frequency features; performing multi-scale decoding on the fused features to obtain fused decoding features; and generating a fused basic component and a fused detail component based on the fused decoding features and combining the two to obtain a medical fusion image. Thus, the method can perform differentiated representation and adaptive fusion on image information of different properties, improve the coordinated reservation capability of effective image information, and take into account the fusion quality, processing efficiency and result stability.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a medical image fusion method and system based on statistical optimization. Background Technology

[0002] With the development of medical imaging technology, various types of medical images can be obtained during clinical diagnosis and treatment. Different medical images have different information characteristics in terms of tissue structure, grayscale distribution, boundary features, and lesion manifestations. Due to the influence of factors such as imaging principles, acquisition conditions, and imaging parameters, the information that a single medical image can present is usually limited. Therefore, medical image fusion technology can be used to process multiple medical images with spatial correspondence, integrating the information contained in different images into a fused image, thereby providing more complete image information for the observation, analysis, and subsequent processing of medical images.

[0003] Currently, medical image fusion can be achieved using various methods such as pixel-based processing, image transformation, or neural networks. Due to differences in information distribution, image contrast, and local feature representation among different medical images, the effective information contribution of different images to the fusion result varies depending on the image region. Existing fusion methods are prone to insufficient retention of some effective information or mutual interference between information from different source images when handling these differences, resulting in problems such as unclear tissue contours, loss of local information, and unstable grayscale representation in the fusion result. Furthermore, with the increasing volume of medical image data and the growing demands for image processing, some fusion methods employ complex processing structures to enhance image information processing capabilities. This may increase computational resource consumption and model training difficulty. In application environments with limited medical image data or constrained computational resources, there is still room for further improvement in fusion effectiveness and processing stability.

[0004] Therefore, in the process of medical image fusion, the significant differences in information between different medical images, the difficulty in fully coordinating and preserving effective image information, and the difficulty in balancing fusion quality, processing efficiency, and result stability have become urgent problems to be solved. Summary of the Invention

[0005] This application provides a statistical optimization-based medical image fusion method and system, aiming to solve the problems in the existing technology of medical image fusion, such as large information differences between different medical images, difficulty in fully coordinating and preserving effective image information, and difficulty in balancing fusion quality, processing efficiency and result stability.

[0006] In a first aspect, a medical image fusion method based on statistical optimization is provided, the method comprising: First and second medical images with spatial correspondence are acquired and frequency band decomposition is performed on them respectively to obtain the low-frequency and high-frequency components corresponding to each medical image. Multi-scale feature extraction is performed on the low-frequency components and the high-frequency components, wherein the low-frequency coding depth is greater than the high-frequency coding depth, to obtain multi-scale low-frequency features and high-frequency features of the two medical images; For the features corresponding to the first and second medical images of the same scale and frequency band, the statistical energy corresponding to each local scale is determined based on local statistical information of multiple local scales. Among them, the low-frequency statistical energy is determined according to the sum of the square of the local mean and the local variance, and the high-frequency statistical energy is determined according to the local variance. The statistical energies corresponding to multiple local scales are aggregated to obtain the statistical energy representation. The corresponding statistical energy representation is normalized and the fusion weight is directly determined. The determination process does not introduce trainable fusion parameters. The corresponding features are fused according to the fusion weight to obtain fused low-frequency features and fused high-frequency features. Multi-scale decoding is performed on the fused low-frequency features and the fused high-frequency features to obtain fused decoded features; Based on the fusion decoding features, fusion basic components and fusion detail components are generated respectively, and then combined to obtain a medical fusion image.

[0007] Optionally, in the above scheme, performing frequency band decomposition on the first medical image and the second medical image respectively includes: The first medical image and the second medical image are smoothed using a preset smoothing kernel to obtain a first low-frequency component and a second low-frequency component. The first medical image is subtracted from the first low-frequency component, and the second medical image is subtracted from the second low-frequency component to obtain a first high-frequency component and a second high-frequency component with positive and negative signs.

[0008] Optionally, in the above scheme, the step of performing multi-scale feature extraction on the low-frequency component and the high-frequency component respectively includes: The low-frequency components are input into a shared encoder, and the low-frequency components are sequentially passed through multiple encoding stages of the shared encoder to obtain low-frequency features at multiple scales. The high-frequency components are input into the shared encoder, and the high-frequency components are encoded through a portion of the pre-encoding stages in the multiple encoding stages before the encoding is terminated, resulting in high-frequency features at multiple scales.

[0009] Optionally, in the above scheme, determining the statistical representation based on local statistical information at multiple local scales includes: For the features corresponding to the first and second medical images of the same scale and frequency band, the local mean and local variance are determined according to multiple local windows of different sizes to obtain multi-window local statistical information; When the corresponding feature is a low-frequency feature, the low-frequency statistical energy corresponding to each local window is determined based on the sum of the square of the local mean and the local variance. When the corresponding feature is a high-frequency feature, the high-frequency statistical energy corresponding to each local window is determined based on the local variance. The statistical energy corresponding to multiple local windows is aggregated to obtain the statistical energy representation of the first medical image and the second medical image at the corresponding scale and frequency band.

[0010] Optionally, in the above scheme, determining the fusion weights based on the statistical energy representation and performing feature fusion according to the fusion weights includes: The statistical energy representations of the first medical image and the second medical image at the same scale and frequency band are normalized to obtain the first spatial weight and the second spatial weight, wherein the sum of the first spatial weight and the second spatial weight at the same spatial location is 1. Based on the first spatial weight and the second spatial weight, the features of the first medical image and the second medical image at the same scale and the same frequency band are weighted and fused to obtain the fused features at the corresponding scale and the corresponding frequency band. Based on the fusion features corresponding to different scales and frequency bands, the fused low-frequency features and the fused high-frequency features are obtained.

[0011] Optionally, in the above scheme, the multi-scale decoding of the fused low-frequency features and the fused high-frequency features includes: Channel adaptation is performed on the deepest-scale fused low-frequency features to obtain the initial decoding features; The initial decoding features are upsampled to obtain the upsampled features corresponding to the current decoding scale; Channel adaptation is performed on at least one of the fused low-frequency features and fused high-frequency features corresponding to the current decoding scale to obtain the current scale adaptation features; The current scale adaptation features are added element-wise to the upsampled features to obtain the current scale decoding features; The current scale decoding features are used as the upsampling features for the next decoding scale. Upsampling, channel adaptation, and element-by-element addition are performed step by step according to the decoding scale to obtain the fused decoding features.

[0012] Optionally, in the above scheme, the step of generating fusion base components and fusion detail components based on the fusion decoding features, and combining the fusion base components and the fusion detail components, includes: The fused decoding features are input to the basic component output path and boundedly mapped to obtain the fused basic component; The fused decoding features are input into the detail component output path and then mapped in a signed and amplitude-limited manner to obtain the fused detail component. The fusion base component and the fusion detail component are superimposed, and the intensity range of the superposition result is limited to obtain the medical fusion image.

[0013] Optionally, in the above scheme, before performing medical image fusion, a self-supervised pre-training is performed on the medical image fusion network used to perform the multi-scale feature extraction and the multi-scale decoding. The self-supervised pre-training includes: Acquire a single-modal medical training image and perform frequency band decomposition on the single-modal medical training image to obtain training low-frequency components and training high-frequency components; Independent random block masks are applied to the training low-frequency component and the training high-frequency component respectively to obtain masked low-frequency component and masked high-frequency component; Differential multi-scale feature extraction is performed on the low-frequency component and the high-frequency component of the mask, and multi-scale decoding is performed on the extracted features to obtain the prediction basic component and the prediction detail component. The predicted basic components and the predicted detail components are combined to obtain the predicted reconstructed image; The joint reconstruction loss is determined based on the differences between the predicted reconstructed image, the predicted basic components, the predicted detail components, and the corresponding training target. The network parameters of the medical image fusion network are updated based on the joint reconstruction loss to obtain the pre-trained medical image fusion network.

[0014] Optionally, the above scheme further includes using the pre-trained medical image fusion network for cross-medical image fusion training, wherein the cross-medical image fusion training includes: Load the encoder parameters, decoder parameters, basic component output parameters, and detail component output parameters of the pre-trained medical image fusion network, set the encoder to parameter frozen state, and retain the basic component output parameters and the detail component output parameters to obtain the medical image fusion network to be trained. Medical images with spatial correspondence are fused into the medical image fusion network to be trained to obtain a training fused image; Based on the medical images, intensity constraint targets and gradient constraint targets are determined, resulting in a fusion training target. The fusion training loss is determined based on the difference between the fused training image and the fusion training target; While keeping the encoder parameters frozen, the decoder parameters, basic component output parameters, and detail component output parameters are updated according to the fusion training loss to obtain the trained medical image fusion network.

[0015] Secondly, a medical image fusion system based on statistical optimization, the system comprising: The image processing module is used to acquire a first medical image and a second medical image that have a spatial correspondence and perform frequency band decomposition on them respectively to obtain the low-frequency component and high-frequency component corresponding to each medical image; The feature extraction module performs multi-scale feature extraction on the low-frequency components and the high-frequency components, wherein the low-frequency coding depth is greater than the high-frequency coding depth, thereby obtaining multi-scale low-frequency features and high-frequency features of the two medical images. The statistical fusion module is used to determine the statistical energy corresponding to each local scale based on local statistical information at multiple local scales for features corresponding to the first and second medical images of the same scale and frequency band. Specifically, low-frequency statistical energy is determined based on the sum of the square of the local mean and the local variance, and high-frequency statistical energy is determined based on the local variance. The statistical energy corresponding to multiple local scales is aggregated to obtain a statistical energy representation. The corresponding statistical energy representation is normalized and the fusion weight is directly determined without introducing trainable fusion parameters. The corresponding features are then fused according to the fusion weight to obtain fused low-frequency features and fused high-frequency features. A multi-scale decoding module is used to perform multi-scale decoding on the fused low-frequency features and the fused high-frequency features to obtain fused decoded features; The fused image generation module is used to generate fused basic components and fused detail components based on the fused decoding features, and combine them to obtain a medical fused image.

[0016] Compared with the prior art, this application has at least the following beneficial effects: Based on further analysis and research of existing technical problems, this application recognizes the significant information differences between different medical images, the difficulty in fully coordinating and preserving effective image information, and the challenge in simultaneously achieving fusion quality, processing efficiency, and result stability during medical image fusion. This application addresses these issues by performing frequency band decomposition on a first and second medical image with spatial correspondence. This allows the overall structural information and local detail information in the medical image to be respectively transformed into corresponding low-frequency and high-frequency components. Furthermore, by employing a relatively deep coding path for the low-frequency components and a relatively shallow coding path for the high-frequency components, multi-scale feature extraction is performed. This enables image information of different properties to be specifically represented according to their respective characteristics, thereby reducing information interference caused by using a uniform processing method. Further, for the features of two medical images of the same scale and frequency band, statistical representations are formed based on local statistical information at multiple local scales, and based on the aforementioned statistical representations… The method uses characterization to determine fusion weights, enabling the fusion contribution of two medical images at different spatial locations to adaptively adjust with local information features, thereby improving the ability to coordinate and preserve effective information in different medical images. After obtaining the low-frequency and high-frequency fusion features, multi-scale decoding is used to form fusion decoding features, and then the fusion basic component and fusion detail component are generated and combined to maintain the overall grayscale and tissue structure in the fused medical image, while also taking into account the expression of local edges and details. Therefore, this application can improve the fusion and preservation of effective image information while adapting to the differences in information of different medical images, and take into account the fusion quality, processing efficiency and result stability of medical image fusion. This solves the problems of large information differences between different medical images, difficulty in fully coordinating and preserving effective image information, and difficulty in balancing fusion quality, processing efficiency and result stability in the existing medical image fusion process. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a statistical optimization-based medical image fusion method provided in one embodiment of this application; Figure 2 This is a schematic diagram of the overall framework of a statistically optimized medical image fusion method provided in one embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] The following provides a detailed description of each implementation method. It should be understood that the following implementation methods are used to explain the corresponding technical solutions, and the network layers, number of channels, window size, and parameter values ​​given are specific implementation methods. Under the condition of satisfying the corresponding processing relationship, they can be configured according to the size of the medical image, the imaging type, and the computing resources.

[0020] In one embodiment, such as Figure 1 As shown, this embodiment provides a statistically optimized medical image fusion method for fusing complementary information contained in a first medical image and a second medical image with spatial correspondence into a single fused medical image. The first and second medical images can be medical images corresponding to the same examination object and the same anatomical region, and can be obtained from different medical imaging modalities, such as any two of computed tomography (CT) images, magnetic resonance imaging (MRI) images, positron emission tomography (PET) images, single-photon emission computed tomography (SPECT) images, or ultrasound images; the first and second medical images can also be obtained from the same imaging modality using different scanning sequences or different imaging parameters. The first and second medical images are denoted as:

[0021] in, and Indicates the spatial dimensions of a medical image.

[0022] Before fusion processing, spatial correspondence processing is performed on the first and second medical images to ensure that the same spatial locations represent the same or substantially the same anatomical regions. Based on the data characteristics of medical images, resampling, grayscale truncation, intensity normalization, and size adjustment can also be performed to ensure that the first and second medical images have the same spatial dimensions and a preset intensity range. In one specific implementation, the input medical image is normalized to... Range: When the size of the input medical image is not divisible by the maximum downsampling factor in the subsequent encoding process, the image edges are padded, and the resulting fused medical image is cropped to its original size after fusion. Specifically, the maximum downsampling factor can be 16.

[0023] Frequency band decomposition was performed on the first and second medical images respectively, resulting in corresponding low-frequency and high-frequency components for each image. The low-frequency components mainly characterize the overall grayscale, tissue contours, regional contrast, and large-scale anatomical structures of the medical image, while the high-frequency components, retaining their positive and negative signs, mainly characterize tissue edges, fine textures, and local intensity variations. By forming frequency band components with different properties before feature extraction, subsequent processing can perform feature representation on both overall structural information and local detail information.

[0024] Multi-scale feature extraction was performed on the low-frequency and high-frequency components of the first and second medical images, respectively. This multi-scale feature extraction was implemented using a shared encoder, with network parameters shared across different medical images and frequency bands. Specifically, low-frequency components were extracted along a relatively deep coding path, while high-frequency components were extracted along a relatively shallow coding path. Therefore, the coding depth for low-frequency components was greater than that for high-frequency components. This yielded low-frequency and high-frequency features of the first and second medical images at multiple scales.

[0025] For features corresponding to first and second medical images of the same scale and frequency band, local statistical information is extracted at multiple local scales, and corresponding statistical representations are formed based on this local statistical information. Different local scales can be achieved through local windows of different sizes, so that the statistical representations can simultaneously adapt to large-scale tissue structures and small-scale edge details. Furthermore, fusion weights are determined based on the statistical representations of the first and second medical images at the same scale, frequency band, and corresponding spatial locations. The features of the two images are then weighted and fused according to these fusion weights to form fused low-frequency features and fused high-frequency features, respectively.

[0026] Multi-scale decoding is performed on fused low-frequency and fused high-frequency features at different scales. During the multi-scale decoding process, spatial resolution is restored step by step starting from the fused features at deeper scales, and corresponding fused low-frequency and / or fused high-frequency features are introduced according to the current decoding scale, so that structural and detail information at different scales is transmitted along the decoding path, and finally the fused decoded features are obtained.

[0027] The fusion decoding features are used to generate fusion base components and fusion detail components. The fusion base components primarily represent the overall grayscale, tissue contours, and large-scale regional structures in the medical fusion image, while the fusion detail components supplement or correct edges, textures, and local intensity variations in the fusion base components. The fusion base components and fusion detail components are then combined, and the intensity range of the combined result is limited to obtain the medical fusion image.

[0028] After training the medical image fusion network, during actual fusion inference, the first and second medical images, after spatial correspondence and intensity processing, are input. The input size is padded to an integer multiple of the maximum downsampling factor of the shared encoder as needed. Then, the low-frequency and high-frequency components of the two medical images are obtained respectively. Multi-scale low-frequency and high-frequency features are obtained using the shared encoder. The corresponding fusion weights are calculated using statistical information at multiple local scales, and fusion low-frequency and high-frequency features are obtained. Then, fusion decoding features are obtained through multi-scale decoding, and fusion basic components and fusion detail components are generated respectively. Finally, the fusion basic components and fusion detail components are combined and intensity-limited. The fusion result is cropped to the original size of the input medical image to obtain the final fused medical image.

[0029] Through the above implementation method, differentiated multi-scale feature extraction is performed based on the different characteristics of overall structural information and local detail information in medical images. Statistical representations at multiple local scales are used to determine the fusion contribution of different medical images at various spatial locations, enabling adaptive fusion of medical images with different information characteristics. On this basis, through multi-scale decoding and joint generation of fusion basic components and fusion detail components, the preservation of overall grayscale, tissue structure, and local edge details is taken into account, thereby improving the ability to coordinate and retain effective image information, while also considering fusion quality, processing efficiency, and output stability.

[0030] In this embodiment, when performing frequency band decomposition on the first medical image and the second medical image, a preset smoothing kernel is used to smooth the first medical image and the second medical image respectively.

[0031] For either the first medical image or the second medical image ,in, Convolution is performed using a preset smoothing kernel to obtain the corresponding low-frequency components:

[0032] in, The scale parameter is represented as The normalized Gaussian kernel, This represents the convolution operation. By processing the first and second medical images in the manner described above, the first low-frequency component and the second low-frequency component can be obtained.

[0033] After obtaining the first low-frequency component and the second low-frequency component, the first medical image is subtracted from the first low-frequency component, and the second medical image is subtracted from the second low-frequency component, respectively, to obtain the first high-frequency component and the second high-frequency component. For any medical image, the high-frequency component is represented as:

[0034] The first and second high-frequency components obtained in this way retain both positive and negative signs, so that both positive and negative local changes generated relative to the low-frequency structure can be preserved.

[0035] In one specific implementation, the size of the normalized Gaussian kernel is... The scale parameter is 1.2, and image boundaries are handled using reflection filling. The same parameters are used for band decomposition in both the pre-training and fusion phases of the medical image fusion network. The normalized Gaussian kernel is fixed and does not participate in the gradient update of network parameters. By maintaining consistency in the band decomposition parameters across different processing phases, the frequency information learned in the pre-training phase can be kept in correspondence with the frequency information processed in the fusion phase.

[0036] The first low-frequency component and the second low-frequency component are used to characterize the overall grayscale, tissue contour, regional contrast and large-scale anatomical structure in the corresponding medical image, while the first high-frequency component and the second high-frequency component are used to characterize the tissue edges, fine textures and local intensity changes in the corresponding medical image.

[0037] Through the above implementation method, low-frequency components reflecting the overall intensity and regional structure are obtained by using fixed smoothing processing, and high-frequency components that retain positive and negative signs are obtained by using the difference between the original medical image and the corresponding low-frequency components. This allows structural information and local detail information with different properties to be processed separately in subsequent processes. At the same time, the fixed frequency band decomposition method that remains consistent in different processing stages does not require additional learning of frequency band decomposition parameters, which helps to reduce the training burden and improve the consistency of frequency band information.

[0038] In this embodiment, multi-scale feature extraction of low-frequency and high-frequency components is achieved by a shared encoder. The shared encoder reuses the first medical image and the second medical image, as well as their corresponding different frequency bands, eliminating the need to configure independent encoding parameters for different medical images. This allows the features of different medical images to be mapped to a unified feature representation space, while reducing the number of network parameters.

[0039] A shared encoder may include an input stem, multiple lightweight residual encoding stages, and multiple downsampling units. Each lightweight residual unit may sequentially include depthwise convolution, group normalization, nonlinear activation, first pointwise convolution, nonlinear activation, second pointwise convolution, channel scaling, and residual summation. Specifically, depthwise convolution is used to extract spatial information, the first and second pointwise convolutions are used to facilitate information exchange between different feature channels, group normalization reduces the network's dependence on the training batch size during feature extraction, and residual summation preserves input information while performing feature transformation.

[0040] In one specific implementation, the shared encoder includes five coding scales: E0, E1, E2, E3, and E4. The number of channels corresponding to each coding scale is 16, 24, 40, 64, and 96, respectively, and the corresponding space reduction factors are 1, 2, 4, 8, and 16, respectively.

[0041] For low-frequency components, the low-frequency components are sequentially processed through multiple coding stages corresponding to E0 to E4, thereby obtaining low-frequency features at multiple scales, which can be expressed as:

[0042] Among them, the low-frequency features at shallower scales maintain relatively high spatial resolution, while the low-frequency features at deeper scales form a larger range of structural representations to represent the overall grayscale, tissue contours, and regional structures from different scales.

[0043] For high-frequency components, the high-frequency components only undergo a portion of the pre-coding stages in the multiple coding stages, and the coding terminates after reaching a preset intermediate coding scale. In one specific implementation, the high-frequency components sequentially pass through E0, E1, and E2, and the coding terminates after stage E2, without continuing through E3 and E4, thus obtaining:

[0044] The high-frequency components of both the first and second medical images are processed according to the aforementioned shallow coding path. Since high-frequency components mainly carry tissue edges, local textures, and local intensity variations, terminating coding at a shallower coding scale can maintain relatively high spatial resolution of high-frequency features and reduce edge and detail loss during continuous downsampling.

[0045] Through the above implementation method, the shared encoder enables different medical images and different frequency bands to share feature extraction parameters. At the same time, different coding depths are configured according to the information characteristics of low-frequency components and high-frequency components. Low-frequency components form structural representations of different ranges through deeper multi-scale coding, while high-frequency components retain local edges and textures through shallower multi-scale coding. This reduces the possibility of mutual interference between different types of information caused by using a uniform coding depth, and reduces computational overhead by reducing unnecessary deep coding of high-frequency branches.

[0046] In this embodiment, for the features corresponding to the first and second medical images of the same scale and frequency band, the local mean and local variance are determined by using multiple local windows of different sizes, thereby forming multi-window local statistical information.

[0047] With the first The first scale, the first Feature maps corresponding to medical images For example, at pixel position The selected size is local area And calculate the local mean:

[0048] in, Indicated by position Centered on, with dimensions of Local area, This indicates the characteristic location within the local region. This represents the local mean within the corresponding local window.

[0049] Simultaneously determine the local variance within the same local region, and denote the local variance as... Local variance is used to characterize the degree of variation of feature data within a corresponding local region. In the specific calculation process, to avoid negative variance caused by floating-point calculation errors, the obtained local variance is limited to not less than 0.

[0050] In one specific implementation, respectively adopt , and Local statistical information is calculated using a local window. When the size of the current feature map is smaller than the preset local window size, the corresponding local window is adjusted to the largest odd-numbered window that is no larger than the current feature map size, so that effective local statistics can be performed on features at different encoding scales.

[0051] For low-frequency features, the low-frequency statistical energy corresponding to each local window is determined based on the sum of the squares of the local means and the local variances:

[0052] The local mean square is used to reflect the overall intensity of a local area, and the local variance is used to reflect the structural changes within a local area. Therefore, this low-frequency statistical energy can simultaneously consider both the gray level of the area and the structural changes within the area.

[0053] For high-frequency features, the high-frequency statistical energy corresponding to each local window is determined based on the local variance:

[0054] High-frequency components have residual characteristics with near-zero mean, so local variance can reflect the strength of changes in tissue edges, texture, and local contrast.

[0055] For each local window and each feature channel, the corresponding statistical energy can be denoted as: ,in, Indicates low frequency or high frequency. This represents the feature channel. The statistical energy of different local windows is averaged, and further averaged along the feature channel dimension to obtain the corresponding single-channel spatial statistical energy representation:

[0056] in, This represents the set of local windows used. By performing the above processing on the first and second medical images respectively, we can obtain the statistical energy representations of the two images at their corresponding scales, frequency bands, and spatial locations.

[0057] By using the above implementation method to statistically analyze medical image features using multiple local windows of different sizes, the dependence of statistical results on the scale of a single window can be reduced, and the scale adaptability to different sizes of tissue structures and local lesion details can be improved. At the same time, statistical energy is constructed according to the different information attributes of low-frequency and high-frequency features, so that the low-frequency statistical representation takes into account both local intensity and internal structural changes, and the high-frequency statistical representation highlights edge and texture changes, thereby improving the stability and pertinence of subsequent fusion decisions.

[0058] In this embodiment, the fusion weight is determined based on the statistical energy representation obtained from the first medical image and the second medical image at the same scale and frequency band.

[0059] The first medical image and the second medical image are in the... The first scale, the first Frequency band and spatial location The statistical energy at each location is denoted as follows: and Based on the statistical energy of both, normalization is performed, and the first spatial weight is:

[0060] The weights of the second space are:

[0061] in, This represents a stability parameter used to prevent the denominator from being zero. In one specific implementation, The first spatial weights and the second spatial weights satisfy:

[0062] Therefore, when the first medical image has a large statistical energy at the corresponding spatial location, its first spatial weight increases accordingly; when the second medical image has a large statistical energy at the corresponding spatial location, its second spatial weight increases accordingly, so that the contribution of different medical images to the fusion result can be adjusted according to local image features.

[0063] Based on the first spatial weight and the second spatial weight, the features of the first medical image and the second medical image at the same scale and frequency band are weighted and fused:

[0064] in, This indicates element-wise multiplication. and The first and second medical images respectively represent the first and second medical images in the 1st and 2nd medical images. The first scale and the first Features of each frequency band.

[0065] By processing features of different scales and frequency bands in the manner described above, and considering that the shared encoder includes five low-frequency coding scales and three high-frequency coding scales, fused low-frequency features of five scales can be obtained:

[0066] And the fused high-frequency features across three scales:

[0067] The first and second spatial weights are directly determined by their respective statistical energy representations, without setting any additional trainable fusion parameters. Therefore, during the training of the medical image fusion network, it is not necessary to use an additional attention network or a cross-medical image interaction network to learn the fusion ratio between the first and second medical images.

[0068] Through the above implementation method, the statistical energy of the first and second medical images at different scales, frequency bands and spatial locations is directly converted into normalized spatial weights, so that the effective information in the two medical images can adaptively participate in the fusion according to the strength of local features. At the same time, the determination of the fusion weights does not introduce additional trainable fusion parameters, which can reduce the number of model parameters, training dependence and overfitting risk, and make the fusion basis of different medical images at corresponding spatial locations have clear statistical meaning and good interpretability.

[0069] In this embodiment, the fused low-frequency features and fused high-frequency features are input into an additive multi-scale decoder for multi-scale decoding. The additive multi-scale decoder may include five decoding stages: D4, D3, D2, D1, and D0. In one specific implementation, the number of channels in each decoding stage is 96, 64, 40, 24, and 16, respectively.

[0070] First, the deepest-scale fusion of low-frequency features Channel adaptation is performed through a channel adaptation layer to obtain the initial decoding features corresponding to the D4 stage. The D4 stage does not require additional complex bottleneck modules, thereby reducing the network parameters and computational overhead of the deepest decoding stage.

[0071] Subsequently, the initial decoded features of stage D4 are upsampled by a factor of two to obtain upsampled features that match the next decoding scale; simultaneously, the fused low-frequency features corresponding to the current decoding scale are channel-adapted and then added element-wise with the upsampled features. The processing for the decoding scale corresponding to E3 is as follows:

[0072] in, This indicates a double upsampling. This represents the channel adaptation layer corresponding to the corresponding scale and frequency band. This indicates the corresponding decoding unit.

[0073] Because the encoding of high-frequency components terminates at a shallower scale, the decoding stages corresponding to E2, E1, and E0 simultaneously exhibit fused low-frequency and fused high-frequency features at the corresponding scales. At the decoding scale corresponding to E2, the following approach is adopted:

[0074] At the decoding scale corresponding to E1, the following is adopted:

[0075] At the decoding scale corresponding to E0, the following is adopted:

[0076] Each channel adapter layer can be adopted Convolution ensures that the fused features from different sources have a number of channels matching the current decoding stage before element-wise addition. When the current decoding scale only has fused low-frequency features, channel adaptation is performed on the corresponding fused low-frequency features, and then they are added element-wise with the upsampled features. When the current decoding scale has both fused low-frequency and fused high-frequency features, channel adaptation is performed on both separately, and then they are added element-wise with the upsampled features to obtain the current scale's decoding features. The current scale's decoding features continue to be used as the upsampled features for the next decoding scale until the final fused decoding features are obtained. .

[0077] The fusion features at different scales are passed to the decoding path by adding them element by element, rather than by concatenating channels. Therefore, the number of feature channels will not increase continuously due to the repeated concatenation of features at different scales.

[0078] Through the above implementation method, spatial resolution is restored step by step starting from the fusion of low-frequency features at the deepest scale, and fused low-frequency features and fused high-frequency features are introduced as needed at the corresponding scales, so that the overall structural information and local detail information are transmitted along the multi-scale decoding path. At the same time, the element-wise addition after channel adaptation is used instead of channel splicing, which can control the number of channels in each decoding stage while retaining multi-scale information, thereby reducing the number of decoder parameters, memory usage and computational complexity, and improving the processing efficiency of medical image fusion.

[0079] In this embodiment, the finally obtained fused decoding features Input the output paths for the basic components and the detailed components respectively.

[0080] The basic component output path includes the basic component output convolutional layer. The fusion fundamental components are generated using bounded mappings. In one specific implementation, sigmoid activation is used for the bounded mapping:

[0081] Sigmoid activation can restrict the fusion fundamental component to... Within this range, the basic components of the fusion are primarily responsible for restoring the overall grayscale, tissue contours, main structure, and regional contrast of the medical fusion image.

[0082] The detail component output path includes detail component output convolutional layers. This method generates fused detail components using signed and amplitude-constrained mappings. In one specific implementation, a hyperbolic tangent activation function is used to perform the signed mapping, and detail amplitude constraint coefficients are utilized. Limit output amplitude:

[0083] Among them, the detail amplitude constraint coefficient satisfies:

[0084] Since hyperbolic tangent activation can generate both positive and negative values, the blended detail components can enhance or correct local edges and textures in the blended base components respectively. By limiting the range of variation of the blended detail components through detail amplitude limiting coefficients, the possibility of excessive enhancement of local details and noise amplification can be reduced.

[0085] The basic and detail components are superimposed, and the superposition result is limited to a preset range of medical image intensity. The final medical fused image is as follows:

[0086] in, This indicates that the output will be limited to a preset range of medical image intensity.

[0087] Through the above implementation method, the overall image intensity restoration and local detail restoration are undertaken by the fusion base component and the fusion detail component, respectively. This allows the fusion base component to stably maintain tissue grayscale, main outline and large-scale structure, while the fusion detail component, which has positive and negative signs and limited amplitude, supplements or corrects local edges and textures. This reduces the risk of excessive detail enhancement, noise amplification and overall intensity distortion while preserving and enhancing local details, thereby improving the grayscale stability and detail preservation ability of medical fusion images.

[0088] like Figure 2 As shown, before performing medical image fusion, the medical image fusion network used for multi-scale feature extraction and multi-scale decoding can be pre-trained in a self-supervised manner, enabling the medical image fusion network to obtain the ability to restore the overall structure and local details of medical images in advance before cross-medical image fusion.

[0089] Acquiring single-modal medical training images The single-modal medical training images were then decomposed using the aforementioned frequency band decomposition method to obtain the training low-frequency components. and training high-frequency components Single-modal medical training images do not require spatially corresponding training image pairs with other medical images; therefore, different types of medical images can each be used as independent self-supervised pre-training samples.

[0090] Generate independent low-frequency masks for training low-frequency components and training high-frequency components respectively. and high-frequency mask The low-frequency components and high-frequency components of the mask are obtained:

[0091]

[0092] in, This indicates element-wise multiplication; positions with a mask value of 1 retain the original information from the corresponding training component, while positions with a mask value of 0 occlude the corresponding information. Since the low-frequency and high-frequency masks are generated independently, the medical image fusion network can learn the restoration relationships of the occluded overall structure and the occluded local details separately.

[0093] In one specific implementation, both the low-frequency and high-frequency masks employ block-based random masks. During each training iteration, a mask ratio is randomly selected from the following preset set of mask ratios:

[0094] And randomly select a block size from the following set of preset block sizes:

[0095] This allows for the creation of varying frequency band recovery tasks using different occlusion ratios and block sizes.

[0096] The low-frequency components of the mask are input into the complete encoding path of the shared encoder, while the high-frequency components are input into the shallow encoding path of the shared encoder. Differential multi-scale feature extraction is then performed on both. The obtained multi-scale features are then input into the multi-scale decoder to obtain the prediction base components. and predict detailed components .

[0097] The predicted fundamental component and predicted detail component are combined, and intensity range is limited to obtain the predicted reconstructed image:

[0098] During self-supervised pre-training, a joint reconstruction loss is determined simultaneously based on the predicted reconstructed image, the predicted fundamental components, and the predicted detail components. The joint reconstruction loss includes overall image restoration loss, low-frequency restoration loss, and high-frequency restoration loss.

[0099] The loss for restoring the entire image is:

[0100] The low-frequency recovery loss is:

[0101] The high-frequency recovery loss is:

[0102] in, Indicates the mean absolute error loss. Represents structural similarity loss. This represents the gradient consistency loss. This represents the Charbonnier loss. In one specific implementation, the weights corresponding to the high-frequency recovery loss are:

[0103] To improve the ability of medical image fusion networks to restore edges and local details.

[0104] The network parameters of the shared encoder and multi-scale decoder are updated simultaneously based on the joint reconstruction loss. For single-modal medical training images from different sources, the corresponding self-supervised pre-training loss can be calculated separately, and the parameters are updated after taking an equally weighted average of the pre-training losses for medical images from different sources. After pre-training, a pre-trained medical image fusion network is obtained.

[0105] Through the above implementation method, before performing cross-medical image fusion training, the shared encoder and multi-scale decoder learn the overall structure recovery capability and local detail recovery capability respectively by using a random block mask recovery task where low-frequency components and high-frequency components are independent of each other. This allows pre-training using single-modality medical images without relying on paired inputs or manual fusion labels, thereby improving the stability of feature learning under the condition of limited medical image samples or annotations, and providing pre-training priors for the stable generation of subsequent fusion basic components and fusion detail components.

[0106] In this embodiment, a pre-trained medical image fusion network is used for cross-medical image fusion training.

[0107] After completing self-supervised pre-training, the encoder parameters, decoder parameters, basic component output parameters, and detail component output parameters of the pre-trained medical image fusion network are loaded, and the shared encoder is set to a parameter-frozen state. During cross-medical image fusion training, the shared encoder remains in inference state, and its parameters do not participate in gradient updates; at the same time, the basic component output parameters and detail component output parameters obtained during pre-training are retained, and the basic component output paths and detail component output paths are not randomly reinitialized, thus obtaining the medical image fusion network to be trained.

[0108] Since the fusion process used to determine the first and second spatial weights in the statistical fusion process itself does not contain trainable fusion parameters, cross-medical image fusion training mainly updates the parameters corresponding to the multi-scale decoder, the basic component output path, and the detail component output path.

[0109] In one specific implementation, the shared encoder can be jointly verified to ensure it remains frozen by utilizing encoder parameter hash values, parameter gradient states, and the training state. At the start of cross-medical image fusion training, the hash values ​​corresponding to the shared encoder parameters are recorded. During training, changes in the shared encoder parameters, the acquisition of gradients by frozen parameters, and the maintenance of the shared encoder's inference state are monitored. If a change in shared encoder parameters, the acquisition of gradients by frozen parameters, or the shared encoder entering the training state are detected, the current training is terminated, thereby preventing the feature representations formed by self-supervised pre-training from being unexpectedly altered during subsequent training.

[0110] A first medical image and a second medical image with spatial correspondence are combined into a medical image pair, which is then input into the medical image fusion network to be trained for fusion processing to obtain the training fused image. .

[0111] Determine the intensity constraint target based on the first and second medical images:

[0112] Thus, intensity constraints are formed at corresponding spatial locations using the significant intensities in the first and second medical images.

[0113] The gradient constraint target is further determined based on the first and second medical images:

[0114] Gradient constraints are formed by utilizing the relatively strong edge information in the first and second medical images.

[0115] In the specific implementation of the final image joint constraint, the fusion training loss is determined based on the training fused image, the intensity constraint objective, and the gradient constraint objective:

[0116] in, This represents the gradient loss weights. In one specific implementation:

[0117] This allows the training fused image to be constrained by both overall intensity and image edges, in order to maintain significant intensity and strong edges in both the first and second medical images.

[0118] In another specific implementation, the strength constraint target can also be... Low-frequency and high-frequency decomposition is performed to obtain the basic target. and detailed goals Direct constraints are applied to the fusion of the fundamental components and the fusion of the detail components, respectively. The corresponding training loss can be composed of the following weighted loss terms:

[0119] in, This represents the constraint loss corresponding to the basic components. This represents the constraint loss corresponding to the detail components. Indicates the strength constraint loss. Represents gradient constraint loss; , and These represent the weights of the corresponding loss terms. This represents the gradient constraint weights that change with each training epoch. Gradient constraint weights The gradient can be gradually increased with each training round to avoid applying too strong a gradient constraint in the early stages of training.

[0120] While keeping the shared encoder parameters frozen, the multi-scale decoder parameters, base component output parameters, and detail component output parameters are updated according to the fusion training loss to obtain the trained medical image fusion network. Since the base component output parameters and detail component output parameters are inherited from the self-supervised pre-training stage, the base component output path can continue to utilize the learned overall strength to recover the prior, and the detail component output path can continue to utilize the learned signed details to recover the prior.

[0121] Through the above implementation method, the feature representation formed by self-supervised pre-training is fixed in the shared encoder, and the trainable parameters in cross-medical image fusion training are concentrated in the multi-scale decoder and the corresponding output path. At the same time, the basic component output parameters and detail component output parameters formed by pre-training are retained, so that feature representation, statistical fusion and image reconstruction can each perform their respective functions. This reduces the optimization difficulty and overfitting risk of fusion training, and reduces the possibility of background intensity decrease, overall brightness distortion and fusion result degradation to edge map due to output parameter re-initialization or strong gradient constraints. This improves the stability of the medical image fusion training process and the final output result.

[0122] like Figure 2 As shown, this embodiment provides a medical image fusion system based on statistical optimization, including an image processing module, a feature extraction module, a statistical fusion module, a multi-scale decoding module, and a fused image generation module. Each functional module transmits the corresponding processing results according to the data processing direction of the medical images to complete the fusion of the first medical image and the second medical image.

[0123] The image processing module acquires a first medical image and a second medical image that have a spatial correspondence, and performs frequency band decomposition on the first and second medical images respectively to obtain corresponding low-frequency and high-frequency components. In specific implementations, the image processing module can also perform resampling, grayscale truncation, intensity normalization, size adjustment, and edge filling to ensure that the first and second medical images have consistent spatial dimensions and intensity ranges suitable for subsequent network processing. Frequency band decomposition can utilize a fixed normalized Gaussian kernel to form the low-frequency component, and then use the difference between the original medical image and the low-frequency component to form the high-frequency component.

[0124] The feature extraction module is used to extract features at multiple scales for low-frequency and high-frequency components respectively. The feature extraction module may include a shared encoder multiplexed from a first medical image, a second medical image, and different frequency bands. Low-frequency components are processed through relatively deep coding paths in the shared encoder to form low-frequency features at multiple scales, while high-frequency components are processed through relatively shallow coding paths in the shared encoder to form high-frequency features at multiple scales, ensuring that the coding depth corresponding to low-frequency components is greater than that corresponding to high-frequency components.

[0125] The statistical fusion module is used to determine local statistical information at multiple local scales for features corresponding to first and second medical images of the same scale and frequency band, and to form statistical representations based on this local statistical information. For low-frequency features, the corresponding statistical energy can be formed using the local mean square and local variance; for high-frequency features, the corresponding statistical energy can be formed using the local variance. The statistical fusion module further determines normalized spatial weights based on the statistical energy of the first and second medical images at the same scale, frequency band, and corresponding spatial locations, and then performs weighted fusion of the corresponding features based on these spatial weights, thereby obtaining fused low-frequency features and fused high-frequency features. The statistical fusion process used to determine the fusion weights does not include trainable fusion parameters.

[0126] The multi-scale decoding module is used to perform multi-scale decoding on fused low-frequency and fused high-frequency features. The multi-scale decoding module can start from the fused low-frequency feature at the deepest scale and perform upsampling step by step. After channel adaptation, the fused low-frequency feature and / or fused high-frequency feature corresponding to the current scale are added element by element to the upsampled feature of the previous decoding stage to form the current scale decoding feature, and the final fused decoding feature is obtained step by step.

[0127] The fused image generation module generates fused basic components and fused detail components based on fused decoding features. The module can include basic component output paths and detail component output paths. The basic component output paths are formed through bounded mapping to represent the overall grayscale and region structure, while the detail component output paths are formed through signed and amplitude-limited mapping to represent local edges and texture variations. The module combines the fused basic components and fused detail components, limiting the combined result to a preset medical image intensity range to obtain a fused medical image.

[0128] Before being used for fusion inference, the system can undergo self-supervised pre-training using single-modal medical training images. By applying independent random block masks to the low-frequency and high-frequency training components, the feature extraction and multi-scale decoding modules learn the overall structure recovery and local detail recovery capabilities. During subsequent cross-medical image fusion training, the shared encoder can be frozen, and the pre-trained basic component output parameters and detail component output parameters can be retained, thus forming the network parameter configuration for actual medical image fusion. Figure 2 The overall framework shown reflects the processing relationship between the first and second medical images, from frequency band decomposition, differential feature extraction, statistical fusion, multi-scale decoding to the joint generation of a medical fusion image from basic components and detail components, and also reflects the corresponding pre-training and parameter freezing relationships.

[0129] Through the above implementation methods, frequency band decomposition, differentiated multi-scale feature extraction, multi-local scale statistical fusion, multi-scale decoding, and joint generation of basic and detail components are respectively configured in corresponding functional modules, enabling the overall structural information and local detail information in different medical images to be adaptively fused along a unified processing chain. At the same time, by using a shared encoder, statistical fusion processing without additional trainable fusion parameters, and additive multi-scale decoding, the number of network parameters and computational complexity are controlled. Furthermore, by utilizing the division of labor between the fusion of basic components and the fusion of detail components, the strength stability and detail preservation ability of the fusion result are improved, thereby taking into account the fusion quality, processing efficiency, and interpretability of the fusion process in medical images.

[0130] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A medical image fusion method based on statistical optimization, characterized in that, The method includes: First and second medical images with spatial correspondence are acquired and frequency band decomposition is performed on them respectively to obtain the low-frequency and high-frequency components corresponding to each medical image. Multi-scale feature extraction is performed on the low-frequency components and the high-frequency components, wherein the low-frequency coding depth is greater than the high-frequency coding depth, to obtain multi-scale low-frequency features and high-frequency features of the two medical images; For the features corresponding to the first and second medical images of the same scale and frequency band, the statistical energy corresponding to each local scale is determined based on local statistical information of multiple local scales. Among them, the low-frequency statistical energy is determined according to the sum of the square of the local mean and the local variance, and the high-frequency statistical energy is determined according to the local variance. The statistical energies corresponding to multiple local scales are aggregated to obtain the statistical energy representation. The corresponding statistical energy representation is normalized and the fusion weight is directly determined. The determination process does not introduce trainable fusion parameters. The corresponding features are fused according to the fusion weight to obtain fused low-frequency features and fused high-frequency features. Multi-scale decoding is performed on the fused low-frequency features and the fused high-frequency features to obtain fused decoded features; Based on the fusion decoding features, fusion basic components and fusion detail components are generated respectively, and then combined to obtain a medical fusion image.

2. The medical image fusion method based on statistical optimization according to claim 1, characterized in that, The step of performing frequency band decomposition on the first medical image and the second medical image respectively includes: The first medical image and the second medical image are smoothed using a preset smoothing kernel to obtain a first low-frequency component and a second low-frequency component. The first medical image is subtracted from the first low-frequency component, and the second medical image is subtracted from the second low-frequency component to obtain a first high-frequency component and a second high-frequency component with positive and negative signs.

3. The medical image fusion method based on statistical optimization according to claim 1, characterized in that, The step of extracting multi-scale features from the low-frequency components and the high-frequency components includes: The low-frequency components are input into a shared encoder, and the low-frequency components are sequentially passed through multiple encoding stages of the shared encoder to obtain low-frequency features at multiple scales. The high-frequency components are input into the shared encoder, and the high-frequency components are encoded through a portion of the pre-encoding stages in the multiple encoding stages before the encoding is terminated, resulting in high-frequency features at multiple scales.

4. The medical image fusion method based on statistical optimization according to claim 1, characterized in that, The determination of statistical representations based on local statistical information at multiple local scales includes: For the features corresponding to the first and second medical images of the same scale and frequency band, the local mean and local variance are determined according to multiple local windows of different sizes to obtain multi-window local statistical information; When the corresponding feature is a low-frequency feature, the low-frequency statistical energy corresponding to each local window is determined based on the sum of the square of the local mean and the local variance. When the corresponding feature is a high-frequency feature, the high-frequency statistical energy corresponding to each local window is determined based on the local variance. The statistical energy corresponding to multiple local windows is aggregated to obtain the statistical energy representation of the first medical image and the second medical image at the corresponding scale and frequency band.

5. The medical image fusion method based on statistical optimization according to claim 4, characterized in that, The step of determining the fusion weights based on the statistical energy representation and performing feature fusion according to the fusion weights includes: The statistical energy representations of the first medical image and the second medical image at the same scale and frequency band are normalized to obtain the first spatial weight and the second spatial weight, wherein the sum of the first spatial weight and the second spatial weight at the same spatial location is 1. Based on the first spatial weight and the second spatial weight, the features of the first medical image and the second medical image at the same scale and the same frequency band are weighted and fused to obtain the fused features of the corresponding scale and the corresponding frequency band. Based on the fusion features corresponding to different scales and frequency bands, the fused low-frequency features and the fused high-frequency features are obtained.

6. The medical image fusion method based on statistical optimization according to claim 1, characterized in that, The multi-scale decoding of the fused low-frequency features and the fused high-frequency features includes: Channel adaptation is performed on the deepest-scale fused low-frequency features to obtain the initial decoding features; The initial decoding features are upsampled to obtain the upsampled features corresponding to the current decoding scale; Channel adaptation is performed on at least one of the fused low-frequency features and fused high-frequency features corresponding to the current decoding scale to obtain the current scale adaptation features; The current scale adaptation features are added element-wise to the upsampled features to obtain the current scale decoding features; The current scale decoding features are used as the upsampling features for the next decoding scale. Upsampling, channel adaptation, and element-by-element addition are performed step by step according to the decoding scale to obtain the fused decoding features.

7. The medical image fusion method based on statistical optimization according to claim 1, characterized in that, The process of generating fusion base components and fusion detail components based on the fusion decoding features, and combining the fusion base components and the fusion detail components, includes: The fused decoding features are input to the basic component output path and boundedly mapped to obtain the fused basic component; The fused decoding features are input into the detail component output path and then mapped in a signed and amplitude-limited manner to obtain the fused detail component. The fusion base component and the fusion detail component are superimposed, and the intensity range of the superposition result is limited to obtain the medical fusion image.

8. The medical image fusion method based on statistical optimization according to claim 1, characterized in that, Prior to performing medical image fusion, the process also includes self-supervised pre-training of the medical image fusion network used to perform the multi-scale feature extraction and the multi-scale decoding, the self-supervised pre-training comprising: Acquire a single-modal medical training image and perform frequency band decomposition on the single-modal medical training image to obtain training low-frequency components and training high-frequency components; Independent random block masks are applied to the training low-frequency component and the training high-frequency component respectively to obtain masked low-frequency component and masked high-frequency component; Differential multi-scale feature extraction is performed on the low-frequency component and the high-frequency component of the mask, and multi-scale decoding is performed on the extracted features to obtain the prediction basic component and the prediction detail component. The predicted basic components and the predicted detail components are combined to obtain the predicted reconstructed image; The joint reconstruction loss is determined based on the differences between the predicted reconstructed image, the predicted basic components, the predicted detail components, and the corresponding training target. The network parameters of the medical image fusion network are updated based on the joint reconstruction loss to obtain the pre-trained medical image fusion network.

9. The medical image fusion method based on statistical optimization according to claim 8, characterized in that, It also includes cross-medical image fusion training using the pre-trained medical image fusion network, wherein the cross-medical image fusion training includes: Load the encoder parameters, decoder parameters, basic component output parameters, and detail component output parameters of the pre-trained medical image fusion network, set the encoder to parameter frozen state, and retain the basic component output parameters and the detail component output parameters to obtain the medical image fusion network to be trained. Medical images with spatial correspondence are fused into the medical image fusion network to be trained to obtain a training fused image; Based on the medical images, intensity constraint targets and gradient constraint targets are determined, resulting in a fusion training target. The fusion training loss is determined based on the difference between the fused training image and the fusion training target; While keeping the encoder parameters frozen, the decoder parameters, basic component output parameters, and detail component output parameters are updated according to the fusion training loss to obtain the trained medical image fusion network.

10. A medical image fusion system based on statistical optimization, characterized in that, The system includes: The image processing module is used to acquire a first medical image and a second medical image that have a spatial correspondence and perform frequency band decomposition on them respectively to obtain the low-frequency component and high-frequency component corresponding to each medical image; The feature extraction module performs multi-scale feature extraction on the low-frequency components and the high-frequency components, wherein the low-frequency coding depth is greater than the high-frequency coding depth, thereby obtaining multi-scale low-frequency features and high-frequency features of the two medical images. The statistical fusion module is used to determine the statistical energy corresponding to each local scale based on local statistical information at multiple local scales for features corresponding to the first and second medical images of the same scale and frequency band. Specifically, low-frequency statistical energy is determined based on the sum of the square of the local mean and the local variance, and high-frequency statistical energy is determined based on the local variance. The statistical energy corresponding to multiple local scales is aggregated to obtain a statistical energy representation. The corresponding statistical energy representation is normalized and the fusion weight is directly determined without introducing trainable fusion parameters. The corresponding features are then fused according to the fusion weight to obtain fused low-frequency features and fused high-frequency features. A multi-scale decoding module is used to perform multi-scale decoding on the fused low-frequency features and the fused high-frequency features to obtain fused decoded features; The fused image generation module is used to generate fused basic components and fused detail components based on the fused decoding features, and combine them to obtain a medical fused image.

Images