A medical image fusion method based on shared multi-dimensional component tensor dictionary learning

Abstract

The application discloses a medical image fusion method based on shared multi-dimensional component tensor dictionary learning, and belongs to the technical field of medical image fusion. The method is as follows: medical images are acquired from a whole brain atlas database, a multi-modal fusion experimental group is constructed, and color images in the multi-modal fusion experimental group are subjected to YUV processing; the medical images are processed by using NSST transformation to obtain corresponding high-frequency tensors and low-frequency tensors; a shared multi-dimensional component tensor dictionary learning method is used to obtain corresponding sparse coefficient tensors for tensors that need to be cross-fused; the fusion frequency information obtained through tensor dictionary learning is subjected to filtering enhancement to obtain an enhanced coefficient tensor of the fusion frequency; and the enhanced coefficient tensor of the fusion frequency is subjected to dictionary reconstruction and inverse NSST operation to obtain a final fusion image. The medical images are processed by using NSST transformation, so that the frequency tensors of the images retain more edge information, and a large amount of redundant information contained in the dictionary is solved; the generalization characteristics of the dictionary are improved by using a p-Laplacian operator, the problem of image quality influenced by an over-complete dictionary is solved, and the quality and accuracy of the fusion result image are improved.

Description

Technical Field

[0001] This invention belongs to the field of medical image fusion technology, specifically relating to a medical image fusion method based on learning a shared multidimensional component tensor dictionary. Background Technology

[0002] Medical image fusion is a technique that combines medical images from different modalities to provide a more comprehensive and accurate fused image. This research has significant applications in clinical diagnosis, medical research, and surgical planning. However, despite the existence of various medical image fusion methods, some challenges remain.

[0003] First, traditional medical image fusion methods can lead to problems such as loss of detail, texture distortion, and blurred edges. This is because traditional methods typically use simple weighted averaging or pixel-level operations to fuse images, failing to fully consider local features and structural information, resulting in the loss of important details. Second, medical images of different modalities have different features and properties. Traditional methods lack adaptability when processing multi-scale and multi-modal images, struggling to fully consider the characteristics of each image type, leading to less than ideal fusion results. Furthermore, traditional methods have high computational complexity, requiring significant computational resources and time.

[0004] To address the problems of traditional medical image fusion methods, researchers have proposed several improved and innovative approaches. For example, multi-scale decomposition-based medical image fusion methods decompose images into sub-bands of different frequencies and then fuse them at different frequencies to preserve image details and structural information as much as possible. Ming Yin et al. proposed a convolutional neural network (NSST-PAPCNN) combining non-subsampling Shearlet transform and perceptual attention mechanisms as an improved method. However, these methods have high computational complexity. To improve computational speed, some methods decompose medical images based on tensor decomposition to obtain tensor information, and then map the tensors into vector data for image fusion. Anastasia Aidini proposed a tensor decomposition-based image fusion method (TDFusion). However, this method may destroy the structure of tensor data by vectorizing tensor information, leading to the loss of edge information. Another type of method is based on dictionary learning, which improves the edge information of the fused image and increases the computational efficiency of tensors by learning an overcomplete dictionary. Kaidong Wang et al. proposed a density peak clustering-based dictionary learning method (LDPC) as an improved method. However, the dictionary learned by this method may contain a large amount of redundant information, thus reducing the quality of the fused image.

[0005] In summary, current medical image fusion methods still have some problems when processing multi-order tensor information. Therefore, how to solve the large amount of redundant information contained in the dictionary, thereby improving the quality and accuracy of the fused image, is the technical problem that this invention aims to solve. Summary of the Invention

[0006] The purpose of this invention is to provide a medical image fusion method based on shared multidimensional component tensor dictionary learning, so as to solve the problems mentioned in the background art.

[0007] The objective of this invention is achieved as follows: a medical image fusion method based on shared multidimensional component tensor dictionary learning, characterized by the following steps:

[0008] Step S1: Obtain medical images from the whole brain atlas database, construct a multimodal fusion experimental group, and perform YUV processing on the color images therein;

[0009] Step S2: Process the medical image using NSST transform to obtain the corresponding high-frequency tensor and low-frequency tensor;

[0010] Step S3: Use the shared multidimensional component tensor dictionary learning method to obtain the corresponding sparse coefficient tensors from the tensors that need to be cross-fused;

[0011] Step S4: Filter and enhance the fused frequency information obtained through tensor dictionary learning to obtain the coefficient tensor of the enhanced fused frequency;

[0012] Step S5: The coefficient tensor of the enhanced fusion frequency is reconstructed by a dictionary and subjected to inverse NSST operation to obtain the final fused image.

[0013] In the preferred step S1, medical images are obtained from the whole-brain atlas database, a multimodal fusion experimental group is constructed, and the color images therein are processed using YUV, specifically as follows:

[0014] The medical images include computed tomography (CT) images, magnetic resonance imaging (MRI) images, positron emission tomography (PET) images, and single-photon emission tomography (SPECT) images. Different weights are applied to the MRI images to obtain T1, T2, and PD.

[0015] A multimodal fusion experimental group was constructed, specifically: T1-T2, T2-PD, CT-MRI, MRI-PET, and MRI-SPECT.

[0016] Preferably, the NSST transform includes multi-scale decomposition and multinomial representation. The NSST transform is used to process medical images, specifically as follows:

[0017] The NSST transform includes multi-scale decomposition and multinomial representation. Specifically, the NSST transform is used to process medical images as follows:

[0018] Step S2-1: Apply non-subsampled pyramidal filters (NSPFs) to the source image. The image is processed to obtain a multi-scale representation of the source image.

[0019] The source image is decomposed into K high-frequency images by using K non-subsampled pyramidal filters (NSPFs). and a low-frequency image

[0020] Step S2-2: Use shift-invariant shear wave filter banks (SFBs) to process high-frequency images and obtain multi-directional representations of high-frequency images in multiple directions;

[0021] The k-th high-frequency graph H k In D k The polynomial representation of each direction is as follows

[0022] Step S2-3: Map the components of the D high-frequency images into a three-dimensional tensor. in Low-frequency images are mapped into a three-dimensional tensor

[0023] Preferably, the processing of medical images using NSST transform is expressed as follows:

[0024]

[0025] Where I is the source image, For a three-dimensional tensor, It is a two-dimensional tensor;

[0026] Given source image and Their high-frequency tensors were obtained by NSST transformation. and low-frequency tensors

[0027] Preferably, in step S3, a shared multidimensional component tensor dictionary learning method is used to obtain the corresponding sparse coefficient tensors from the tensors that need to be cross-fused, specifically as follows:

[0028] Step S3-1: Using linear combination of tensors, obtain a dictionary with the same dimension as the input information and sparsely represent the input tensor;

[0029] High-frequency tensors The dictionary tensor linear representation is as follows:

[0030] in, The dictionary tensor expands into a matrix along three dimensions.

[0031] Low-frequency tensors The dictionary tensor linear representation is as follows:

[0032] in, and It is a two-dimensional matrix, and also a third-order tensor with a third-dimensional space of size 1;

[0033] Step S3-2: Optimize the input tensor dictionary representation;

[0034] To ensure that no information is lost from the input tensor representation after tensor dictionary learning, the input tensor representation is fed into the objective function, which is:

[0035]

[0036] in, It is a high-frequency tensor. It is a low-frequency tensor;

[0037] Step S3-3: Optimize tensor dictionary learning using a shared multidimensional component tensor dictionary;

[0038] To improve the generalization ability of the dictionary, a method is used to learn a shared multidimensional component tensor dictionary from data of different frequencies, capturing common structures and features in the data; the objective function is:

[0039]

[0040] Among them, shared dictionary Where R <= R1 + R2;

[0041] Step S3-4: Use the discretized p-Laplacian operator to constrain the shared dictionary to obtain the overall objective function;

[0042] Step S3-5: Based on the overall objective function, solve the sparse tensor using the alternating minimization algorithm. sparse tensor and dictionaries

[0043] initialization A third-order tensor with elements of 0, and a dictionary initialized. It is a random three-dimensional tensor;

[0044] Using an alternating minimization algorithm to fix the sparse tensor and dictionary Solving sparse tensors Fixed sparse tensor and sparse tensors Dictionary solving Fixed sparse tensor Solving sparse tensors dictionary Finally, we obtain the sparse tensor that minimizes the overall objective function. sparse tensor and dictionaries

[0045] Step S3-6: Take a weighted average of the corresponding elements of different sparse coefficient tensors to obtain the sparse coefficient tensor of low-frequency information fusion.

[0046] Preferably, in step S3-3, a discretized p-Laplacian operator constraint shared dictionary is used to obtain the overall objective function, specifically:

[0047] By using the shared dictionary of p-Laplacian regularization terms in the p-Laplacian operator, the expression for the p-Laplacian regularization term is:

[0048] in, It is a dictionary The expansion matrix in the third dimension The Laplace matrix, It is the p of this matrix i Norm;

[0049] Introducing the p-Laplacian operator constraint into tensor dictionary learning, the objective function becomes:

[0050]

[0051] Where β is the regularization parameter, It is a high-frequency tensor. λ1 is a low-frequency tensor; λ2 is a sparse tensor. The sparsity parameter, λ2 is the sparse tensor The sparsity parameter.

[0052] Preferably, in steps S3-4, the sparse tensor is solved using an alternating minimization algorithm based on the overall objective function. sparse tensor and dictionaries Specifically:

[0053] Step S3-4-1: Solve the sparse tensor using the alternating minimization algorithm. fixed and Iterative updates using gradient descent are used to solve the problem. The specific steps are as follows:

[0054] according to and Calculate the residual tensor:

[0055] Complete minimization Objective function:

[0056] Calculate the gradient of the residual term, i.e.

[0057] Calculate the gradient of the regularization term, i.e.

[0058] renew Right now

[0059] right A soft threshold function is applied to each element to achieve sparsity updates. Right now:

[0060]

[0061] Where the learning rate α1 is known, the sparse tensor is finally obtained.

[0062] Step S3-4-2: Fix and Iterative updates using gradient descent are used to solve the problem.

[0063] according to and Calculate the residual tensor:

[0064] Complete minimization Objective function:

[0065] Calculate the gradient of the residual term, i.e.

[0066] Calculate the gradient of the regularization term, i.e.

[0067] renew Right now

[0068] right A soft threshold function is applied to each element to achieve sparsity updates. Right now:

[0069]

[0070] Where the learning rate α2 is known, the sparse tensor is finally obtained.

[0071] Preferably, in steps S3-4, the sparse tensor is solved using an alternating minimization algorithm based on the overall objective function. sparse tensor and dictionaries Specifically:

[0072] Fixed sparse tensor sparse tensor Update dictionary Minimize dictionary Objective function:

[0073]

[0074] Step S3-4-3: Iterate and update using gradient descent until the residual is less than 0.001.

[0075] Calculate the gradient of the residual term, i.e.:

[0076]

[0077]

[0078] Calculate the gradient of the regularization term P(D) by taking the derivative with respect to each element:

[0079] in,

[0080]

[0081] The total gradient Convert to and For a third-order tensor with the same dimension, update the dictionary. Right now

[0082] The learning rate α3 is known, and the dictionary tensor is ultimately obtained.

[0083] Preferably, in step S4, the fused frequency information obtained through tensor dictionary learning is filtered and enhanced to obtain the coefficient tensor of the enhanced fused frequency, specifically as follows:

[0084] The idea of ​​filtering GIFs using local filter maps: using tensors To guide the filtering of tensors, Filtering will result in an enhanced output tensor. It is a guiding tensor In window W ε The linear transformation of (p′) is expressed as:

[0085]

[0086] Among them, window W ε (p′), with radius ε and center p′, p∈W ε (p′), a p′ and b p′ It is a constant;

[0087]

[0088]

[0089] Where ⊙ represents a tensor and Element product, Tensor and Elements in window W ε (p′) is the average of the element-wise products. Tensor Elements in window W ε (p′) is the average value of the elements. Tensor Elements in window W ε (p′) is the average value of the elements. Tensor In window W ε The mean squared error of element (p′).

[0090] Preferably, in step S5, the coefficient tensor of the enhanced fusion frequency undergoes dictionary reconstruction and inverse NSST operation, specifically as follows:

[0091] Low-frequency fusion coefficient tensor Low-frequency fusion frequency information is obtained by reconstructing from the shared dictionary.

[0092] Coefficient tensor of fusion frequency The fusion frequency information is obtained by reconstructing from the shared dictionary.

[0093] The obtained low-frequency fusion frequency information and fusion frequency information As input to the inverse NSST, the final fused image I is obtained. res ,

[0094] Compared with the prior art, the present invention has the following improvements and advantages:

[0095] 1. By using NSST transform to process medical images, the frequency tensor of the images is obtained to retain more edge information; by using the tensor dictionary learning method with shared multidimensional components and using the p-Laplacian operator to constrain the shared dictionary, the problem of a large amount of redundant information contained in the overcomplete dictionary is solved, thereby improving the quality and accuracy of the fused image.

[0096] 2. By using a local filter map filtering method, the sparse tensor of the fused frequency information obtained through shared dictionary learning is filtered and enhanced, which solves the generalization problem in dictionary learning, preserves important edge information in the source image, and improves the applicability of the fused image method. At the same time, dictionary reconstruction and inverse NSST operation are performed on the coefficient tensor of the enhanced fused frequency to further improve the quality and accuracy of the fused image, providing stronger support for doctors' diagnosis and analysis. Attached Figure Description

[0097] Figure 1 This is an overall flowchart of the method of the present invention.

[0098] Figure 2 This is a general framework diagram of the inventive method.

[0099] Figure 3 This is an intuitive visual comparison diagram of the fused images obtained through experiments using three comparison methods and the method proposed in this patent, representing two different modalities: CT and T2 source images.

[0100] Figure 4 An intuitive visual comparison diagram of the fused images obtained through experiments using three comparison methods and the method proposed in this patent, representing two different modalities, T1 and T2.

[0101] Figure 5 This is an intuitive visual comparison diagram of the fused images obtained through experiments using three comparison methods and the method proposed in this patent, representing two different modalities, T2 and PD. Detailed Implementation

[0102] The invention will be further summarized below with reference to the accompanying drawings.

[0103] like Figure 1 , 2 As shown, a medical image fusion method based on shared multidimensional component tensor dictionary learning is presented, which includes the following steps:

[0104] Step S1: Obtain medical images from the whole brain atlas database, construct a multimodal fusion experimental group, and perform YUV processing on the color images therein;

[0105] Four types of medical images were obtained from the whole brain atlas database, including: computed tomography (CT) images, magnetic resonance imaging (MRI) images, positron emission tomography (PET) images, and single photon emission tomography (SPECT) images. Different weights were applied to the MRI images to obtain T1, T2, and PD. The spatial resolution of all medical images was 256*256.

[0106] In medical image fusion, color PET and SPECT images are originally presented in grayscale form, but in order to provide richer visual information, they are presented as color images by applying pseudo-color mapping;

[0107] This invention aims to obtain high-quality medical fusion images, with a focus on edge information. Therefore, a YUV processing method is used to convert color images into YUV images. The YUV space in which the YUV image is located separates luminance and chrominance, where the luminance component Y represents the brightness information of the image, while the chrominance components U and V represent color information. This invention uses the luminance component Y of the color image as input and completes medical image fusion through the following steps.

[0108] Step S2: Process the medical image using NSST transform to obtain the corresponding high-frequency tensor and low-frequency tensor;

[0109] The NSST transform is a multi-scale analysis method that effectively captures multi-dimensional geometric feature information in an image using the Shearlet transform. It generates a result of the same size as the source image through non-downsampling and has translation invariance and stronger robustness. Therefore, the NSST transform is chosen in this invention to perform medical image analysis operations under different modalities.

[0110] The NSST transform comprises two basic steps: multi-scale decomposition and multi-directional representation. The application of the NSST transform to process medical images specifically involves:

[0111] Step S2-1: Apply non-subsampled pyramidal filters (NSPFs) to the source image. The image is processed to obtain a multi-scale representation of the source image.

[0112] The source image is decomposed into K high-frequency images by using K non-subsampled pyramidal filters (NSPFs). and a low-frequency image The frequency of high-frequency images gradually decreases;

[0113] Step S2-2: Use shift-invariant shear wave filter banks (SFBs) to process high-frequency images and obtain multi-directional representations of high-frequency images in multiple directions;

[0114] The k-th high-frequency graph H kIn D k The polynomial representation of each direction is as follows Since the high-frequency image and the source image have the same size, translation invariance can be guaranteed and pseudo-Gibbs phenomenon can be suppressed.

[0115] Step S2-3: Map the components of the D high-frequency images into a three-dimensional tensor. in Low-frequency images are mapped into a three-dimensional tensor

[0116] The processing of medical images using the NSST transform is represented as follows:

[0117]

[0118] Where I is the source image, For a three-dimensional tensor, It is a two-dimensional tensor;

[0119] Given source image and Their high-frequency tensors were obtained by NSST transformation. and low-frequency tensors

[0120] Step S3: Use the shared multidimensional component tensor dictionary learning method to obtain the corresponding sparse coefficient tensors from the tensors that need to be cross-fused;

[0121] To preserve all important information in the high and low frequencies obtained from the source image decomposition, a mixing tensor cross-fusion processing method is proposed; specifically, the high-frequency tensor of the source image I1 obtained in step 2 is... and the low-frequency tensor of I2 Perform cross-processing on the low-frequency tensor of the source image I1 and I2 high-frequency tensors Perform cross-processing;

[0122] The shared multidimensional component tensor dictionary learning method is used to obtain the corresponding sparse coefficient tensors from the tensors that need to be cross-fused, specifically:

[0123] Step S3-1: Using linear combination of tensors, obtain a dictionary with the same dimension as the input information and sparsely represent the input tensor;

[0124] High-frequency tensors The dictionary tensor linear representation is as follows:

[0125] in, The dictionary tensor expands into a matrix along three dimensions.

[0126] Low-frequency tensors Linear representation by dictionary tensors:

[0127] in, and It is a third-order tensor with a size of 1 in the third-dimensional space, and also a two-dimensional matrix;

[0128] Step S3-2: Optimize the input tensor dictionary representation;

[0129] To ensure that no information is lost from the input tensor representation after tensor dictionary learning, the input tensor representation is fed into the objective function, which is:

[0130]

[0131] in, It is a high-frequency tensor. It is a low-frequency tensor;

[0132] To address the issue that overcomplete dictionaries containing redundant information degrade the quality of fused images, the constraint of ensuring the learned dictionary is not overcomplete is transformed into ensuring the sparsity of the resulting coefficient tensor, i.e., minimizing the number of non-zero elements in the coefficient tensor. The L0 norm can be used to calculate the number of non-zero elements in the coefficient tensor, but the L0 norm is difficult to solve. The L1 norm, however, is the optimal convex approximation function of the L0 norm and is easier to optimize. Therefore, this invention uses the L1 norm as the constraint for sparse coefficients to solve the sparsity optimization problem. The constraint formula is shown below:

[0133]

[0134] Step S3-3: Optimize tensor dictionary learning using a shared multidimensional component tensor dictionary;

[0135] To improve the generalization ability of the dictionary, a method is used to learn a shared multidimensional component tensor dictionary from data of different frequencies, capturing common structures and features in the data; the objective function is:

[0136]

[0137] Among them, shared dictionary Where R <= R1 + R2;

[0138] Among the tensors obtained from decomposition at different frequencies, the information we are interested in is the same, and the corresponding feature information is also the same. We propose a method to learn a shared multidimensional component tensor dictionary from data of different frequencies. By learning a shared dictionary, we can capture the common structure and features in the data, which not only improves the model's generalization ability and efficiency but also makes the model better suited to new tasks or data.

[0139] Step S3-4: Use the discretized p-Laplacian operator to constrain the shared dictionary to obtain the overall objective function;

[0140] The p-Laplacian operator is an operator used in graph signal and image processing to describe the relationships and similarities between nodes in a graph. This invention employs a discretized p-Laplacian operator to compute a shared dictionary, thereby improving the dictionary's smoothness. This operator enables the basis vectors to vary gradually in the data space and helps extract a more discriminative and compact dictionary representation. By using the p-Laplacian operator, the generalization ability of dictionary learning is improved, thus reducing dictionary redundancy and enhancing its generalization capability.

[0141] By using the shared dictionary of p-Laplacian regularization terms in the p-Laplacian operator, the expression for the p-Laplacian regularization term is:

[0142] in, It is a dictionary The expansion matrix in the third dimension The Laplace matrix, It is the p of this matrix i Norm;

[0143] Introducing the p-Laplacian operator constraint into tensor dictionary learning, the objective function becomes:

[0144]

[0145] Where β is the regularization parameter, It is a high-frequency tensor. λ1 is a low-frequency tensor; λ2 is a sparse tensor. The sparsity parameter, λ2 is the sparse tensor The sparsity parameter.

[0146] Step S3-5: Based on the overall objective function, solve the sparse tensor using the alternating minimization algorithm. sparse tensor and dictionaries

[0147] initialization A third-order tensor with elements of 0, and a dictionary initialized. It is a random three-dimensional tensor;

[0148] Using an alternating minimization algorithm to fix the sparse tensor and dictionary Solving sparse tensors Fixed sparse tensor and sparse tensors Dictionary solving Fixed sparse tensor Solving sparse tensors dictionary Finally, we obtain the sparse tensor that minimizes the overall objective function. sparse tensor and dictionaries

[0149] Step S3-5-1: Solve the sparse tensor using the alternating minimization algorithm. fixed and Iterative updates using gradient descent are used to solve the problem. The specific steps are as follows:

[0150] according to and Calculate the residual tensor:

[0151] Complete minimization Objective function:

[0152] Calculate the gradient of the residual term, i.e.

[0153] Calculate the gradient of the regularization term, i.e.

[0154] renew Right now

[0155] right A soft threshold function is applied to each element to achieve sparsity updates. Right now:

[0156]

[0157] Where the learning rate α2 is known, the sparse tensor is finally obtained.

[0158] Step S3-5-2: Fix and Iterative updates using gradient descent are used to solve the problem.

[0159] according to and Calculate the residual tensor:

[0160] Complete minimization Objective function:

[0161] Calculate the gradient of the residual term, i.e.

[0162] Calculate the gradient of the regularization term, i.e.

[0163] renew Right now

[0164] right A soft threshold function is applied to each element to achieve sparsity updates. Right now:

[0165]

[0166] Where the learning rate α2 is known, the sparse tensor is finally obtained.

[0167] Fixed sparse tensor sparse tensor Update dictionary Minimize dictionary Objective function:

[0168]

[0169] Step S3-5-3: Iterate and update using gradient descent until the residual is less than 0.001.

[0170] Calculate the gradient of the residual term, i.e.:

[0171]

[0172]

[0173] Calculate the gradient of the regularization term P(D) by taking the derivative with respect to each element:

[0174] in,

[0175]

[0176] The total gradient Convert to and For a third-order tensor with the same dimension, update the dictionary. Right now

[0177] The learning rate α3 is known, and the dictionary tensor is ultimately obtained.

[0178] Step S3-6: Take a weighted average of the corresponding elements of different sparse coefficient tensors to obtain the sparse coefficient tensor of low-frequency information fusion;

[0179] The sparse coefficient tensor obtained by learning the low-frequency information of image decomposition through a shared dictionary By taking a weighted average of the corresponding elements of the sparse coefficient tensors of different low-frequency information, the sparse coefficient tensor of low-frequency information fusion is finally obtained:

[0180]

[0181] in, This represents the summation of corresponding elements of two tensors.

[0182] Step S4: Filter and enhance the fused frequency information obtained through tensor dictionary learning to obtain the coefficient tensor of the fused frequency;

[0183] The idea of ​​local filter map filtering GIF is adopted: the sparse tensor of the fusion frequency obtained from high-frequency and low-frequency information through shared dictionary learning is enhanced to reduce the loss of edge features of dictionary representation in order to improve the genericity of shared dictionary;

[0184] The coefficient tensor fusion is shown in the following equation:

[0185]

[0186]

[0187]

[0188] Use window W ε (p′) in tensor To guide the filtering of tensors, Filtering will result in an enhanced output tensor. It is a guiding tensor In window W ε The linear transformation of (p′) is expressed as:

[0189]

[0190] Among them, window W ε (p′), with radius ε and center p′, p∈W ε (p′), a p′ and b p′ It is a constant;

[0191]

[0192]

[0193] Where ⊙ represents a tensor and Element product, Tensor and Elements in window W ε (p′) is the average of the element-wise products. Tensor Elements in window W ε (p′) is the average value of the elements. Tensor Elements in window W ε (p′) is the average value of the elements. Tensor In window W ε The mean square error of element (p′);

[0194] In this invention, the guiding tensor

[0195] Step S5: The coefficient tensor of the enhanced fused frequency is reconstructed by a dictionary and subjected to inverse NSST operation to obtain the final fused image;

[0196] The final fused image is obtained as follows:

[0197] Low-frequency fusion coefficient tensor Low-frequency fusion frequency information is obtained by reconstructing from the shared dictionary.

[0198] Coefficient tensor of fusion frequency The fusion frequency information is obtained by reconstructing from the shared dictionary.

[0199] The obtained low-frequency fusion frequency information and fusion frequency information As input to the inverse NSST, the final fused image I is obtained. res ,

[0200] To demonstrate the technical effectiveness of the method of the present invention, experiments were conducted for verification.

[0201] The experimental part of this invention selected five of the most common medical image fusion combination types with different modalities: T1-T2, T2-PD, CT-MRI, MRI-PET, and MRI-SPECT.

[0202] Three metrics were selected: ChenBlum calculates the average quality map of the entire image; Multiscale Structural Similarity Measure (MS-SSIM) is an improved version of SSIM. SSIM measures the similarity between two images on a single scale, while MS-SSIM introduces multiscale analysis, which is more suitable for this invention; Standard Deviation (STD) calculates the contrast information of the fused image quality.

[0203] The image fusion effect of the present invention is analyzed from different perspectives using the above three indicators, thus verifying the performance of the proposed method.

[0204] The proposed method was compared with two existing medical image fusion methods, namely, a convolutional neural network combining non-subsampled Shearlet transform and perceptual attention mechanism (NSST-PAPCNN), an image fusion method based on tensor decomposition (TDFusion), and a dictionary learning method based on density peak clustering (LDPC).

[0205] To more objectively compare the fusion results of each method, all parameters in these methods were set to their default values. The quantitative results of the experiments are shown in the attached figure. Figure 3 For source images of two different modalities, CT and T2, Figure 4 The source images are for two different modalities, T1 and T2. Figure 5 Experiments were conducted on source images of two different modalities, T2 and PD, using three comparison methods and the method proposed in this patent, resulting in an intuitive visual comparison image of the fused image.

[0206] Table 1 is... Figure 4 The evaluation index results obtained from the experiment are as follows:

[0207] Table 1

[0208] NSST-PAPCNN 0.6032 0.9576 0.1953 TDFusion 0.5683 0.9046 0.1878 LDPC 0.5975 0.9671 0.2614 This patented method 0.7142 0.9875 0.2697

[0209] Based on the results in Table 1, in terms of the ChenBlum index, the NSST-PAPCNN method, employing non-subsampled Shearlet transform and the adaptive characteristics of PAPCNN, can provide high-quality fused images; LDPC, with its overcomplete dictionary, focuses more on sparse representation, which may sacrifice the overall image quality in some cases; TDFusion may prioritize the integration of overall information when processing multimodal image fusion, lacking the preservation of local details; the method proposed in this patent combines the advantages of these methods, employing NSST transform and p-Laplacian operator-constrained tensor dictionary learning to improve the quality of the fusion results.

[0210] In terms of the MS-SSIM metric, LDPC fully considers the local information of the source image, and the NSST-PAPCNN method uses non-subsampled Shearlet transform to obtain the multi-scale analysis of the source image using different frequency information. TDFusion ignores the multi-scale analysis of the source image and performs poorly. The method proposed in this patent combines the advantages of these methods and analyzes the source image at different scales to obtain a tensor dictionary with shared multi-dimensional components, making the fused image more accurate.

[0211] In terms of STD metrics, LDPC provides a more comprehensive analysis of details, and NSST-PAPCNN is also a method for analyzing details at multiple scales, but TDFusion's research on image contrast is not precise enough.

[0212] In summary, the method of this invention employs NSST transform for multi-scale analysis of medical images, resulting in more accurate fused images. It proposes a tensor dictionary learning method with shared multi-dimensional components to address the problem of excessive redundant information in overcomplete dictionaries, thereby improving the quality of the fused image. Furthermore, it utilizes a local filter guide graph filtering method to address the loss of important edge information in the source image to improve the generalization of dictionary learning. Comparisons across three metrics and intuitive comparison charts demonstrate that the method of this invention produces fused images with better quality, higher accuracy, and wider applicability than the three previously compared methods.

[0213] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A medical image fusion method based on shared multidimensional component tensor dictionary learning, characterized in that: The method includes the following steps: Step S1: Obtain medical images from the whole brain atlas database, construct a multimodal fusion experimental group, and perform YUV processing on the color images therein; Step S2: Process the medical image using NSST transform to obtain the corresponding high-frequency tensor and low-frequency tensor; Step S3: Using a shared multidimensional component tensor dictionary learning method, the tensors that need to be cross-fused are used to obtain the corresponding sparse coefficient tensors, specifically: Step S3-1: Using linear combination of tensors, obtain a dictionary with the same dimension as the input information and sparsely represent the input tensor; High-frequency tensors The dictionary tensor linear representation is as follows: ; in, , , The dictionary tensor is expanded into a matrix along three dimensions. ; Low-frequency tensors The dictionary tensor linear representation is as follows: ; in, , ; and It is a two-dimensional matrix, and also a third-order tensor with a third-dimensional space of size 1; Step S3-2: Optimize the input tensor dictionary representation; To ensure that no information is lost from the input tensor representation after tensor dictionary learning, the input tensor representation is fed into the objective function, which is: ； in, , , , , It is a high-frequency tensor. It is a low-frequency tensor; Step S3-3: Optimize tensor dictionary learning using a shared multidimensional component tensor dictionary; To improve the generalization ability of the dictionary, a method is used to learn a shared multidimensional component tensor dictionary from data of different frequencies, capturing common structures and features in the data; the objective function is: ； Among them, shared dictionary ,in ; Step S3-4: Use the discretized p-Laplacian operator to constrain the shared dictionary to obtain the overall objective function; Step S3-5: Based on the overall objective function, solve the sparse tensor using the alternating minimization algorithm. sparse tensors and dictionaries ; initialization , A third-order tensor with elements of 0, and a dictionary initialized. It is a random three-dimensional tensor; Using an alternating minimization algorithm to fix the sparse tensor and dictionary Solving sparse tensors Fixed sparse tensor and sparse tensors Dictionary solving Fixed sparse tensor Solving sparse tensors dictionary Finally, the sparse tensor that minimizes the overall objective function is obtained. sparse tensors and dictionaries ; Step S3-6: Take a weighted average of the corresponding elements of different sparse coefficient tensors to obtain the sparse coefficient tensor of low-frequency information fusion; Step S4: Filter and enhance the fused frequency information obtained through tensor dictionary learning to obtain the coefficient tensor of the enhanced fused frequency, specifically: The idea of ​​filtering GIFs using local filter maps: using tensors To guide the filtering of tensors, Filtering will result in an enhanced output tensor. It is a guiding tensor In the window The linear transformation is expressed as: ； Among them, window ,by With radius, Centered on, , and It is a constant; ； ； in, Tensor and Element product, Tensor and Elements in window The average of the element-wise products, Tensor Elements in window The average value of the elements Tensor Elements in window The average value of the elements Tensor In the window The mean squared error of the elements; Step S5: The coefficient tensor of the enhanced fusion frequency is reconstructed by a dictionary and subjected to inverse NSST operation to obtain the final fused image.

2. The medical image fusion method based on shared multidimensional component tensor dictionary learning according to claim 1, characterized in that: In step S1, medical images are obtained from the whole-brain atlas database, a multimodal fusion experimental group is constructed, and the color images are processed using YUV, specifically as follows: The medical images include computed tomography (CT) images, magnetic resonance imaging (MRI) images, positron emission tomography (PET) images, and single-photon emission tomography (SPECT) images. Different weights are applied to the MRI images to obtain T1, T2, and PD. A multimodal fusion experimental group was constructed, specifically: T1-T2, T2-PD, CT-MRI, MRI-PET, and MRI-SPECT.

3. The medical image fusion method based on shared multidimensional component tensor dictionary learning according to claim 1, characterized in that: The NSST transform includes multi-scale decomposition and multinomial representation. Specifically, the NSST transform is used to process medical images as follows: Step S2-1: Apply non-subsampled pyramidal filters (NSPFs) to the source image. The image is processed to obtain a multi-scale representation of the source image. The source image is decomposed into K high-frequency images by using K non-subsampled pyramidal filters (NSPFs). , and a low-frequency image ; Step S2-2: Use shift-invariant shear wave filter banks (SFBs) to process high-frequency images and obtain multi-directional representations of high-frequency images in multiple directions; k-th high-frequency graph exist The polynomial representation of each direction is as follows , ; Step S2-3: ... The components of a high-frequency image are mapped to a three-dimensional tensor. ,in The low-frequency image is mapped into a three-dimensional tensor. .

4. The medical image fusion method based on shared multidimensional component tensor dictionary learning according to claim 3, characterized in that: The processing of medical images using NSST transform is represented as follows: ； in, For the source image, For a three-dimensional tensor, It is a two-dimensional tensor; Given source image and Their high-frequency tensors were obtained through NSST transformation. , and low-frequency tensors , .

5. The medical image fusion method based on shared multidimensional component tensor dictionary learning according to claim 1, characterized in that: In step S3-4, a discretized p-Laplacian operator constraint shared dictionary is used to obtain the overall objective function, specifically: By using the shared dictionary of p-Laplacian regularization terms in the p-Laplacian operator, the expression for the p-Laplacian regularization term is: ; in, It is a dictionary The expansion matrix in the third dimension The Laplace matrix, It is the matrix Norm; Introducing the p-Laplacian operator constraint into tensor dictionary learning, the objective function becomes: ； in, For regularization parameters, , , , , It is a high-frequency tensor. It is a low-frequency tensor; For sparse tensors sparsity parameters For sparse tensors The sparsity parameter.

6. The medical image fusion method based on shared multidimensional component tensor dictionary learning according to claim 1, characterized in that: In steps S3-5, the sparse tensor is solved using an alternating minimization algorithm based on the overall objective function. sparse tensors and dictionaries Specifically: Step S3-5-1: Solve the sparse tensor using the alternating minimization algorithm. ,fixed and Iterative updates using gradient descent are used to solve the problem. The specific operation is as follows: according to and Calculate the residual tensor: ; Complete minimization Objective function: ; Calculate the gradient of the residual term, i.e. ; Calculate the gradient of the regularization term, i.e. ; renew ,Right now ; right A soft threshold function is applied to each element to achieve sparsity updates. ,Right now: ； Among them, learning rate It is known that the sparse tensor is finally obtained. ; Step S3-5-2: Fix and Iterative updates using gradient descent are used to solve the problem. : according to and Calculate the residual tensor: ; Complete minimization Objective function: ; Calculate the gradient of the residual term, i.e. ; Calculate the gradient of the regularization term, i.e. ; renew ,Right now ; right A soft threshold function is applied to each element to achieve sparsity updates. ,Right now: ； Among them, learning rate It is known that the sparse tensor is finally obtained. .

7. The medical image fusion method based on shared multidimensional component tensor dictionary learning according to claim 1, characterized in that: In steps S3-5, the sparse tensor is solved using an alternating minimization algorithm based on the overall objective function. sparse tensors and dictionaries Specifically: Fixed sparse tensor sparse tensors Update the dictionary Minimize dictionary Objective function: ； Step S3-5-3: Iterate and update using gradient descent until the residual is less than 0.

001. Calculate the gradient of the residual term, i.e.: ； ； Calculate regularization terms The gradient of the derivative with respect to each element: ; in, ; ； Total gradient Convert to and For a third-order tensor with the same dimension, update the dictionary. :Right now ; Among them, learning rate It is known that the final result is a dictionary tensor. .

8. The medical image fusion method based on shared multidimensional component tensor dictionary learning according to claim 1, characterized in that: In step S5, the coefficient tensor of the enhanced fusion frequency undergoes dictionary reconstruction and inverse NSST operation, specifically as follows: Low-frequency fusion coefficient tensor Low-frequency fusion frequency information is obtained by reconstructing from the shared dictionary. , ; Coefficient tensor of fusion frequency The fusion frequency information is obtained by reconstructing from the shared dictionary. , ; The obtained low-frequency fusion frequency information and fusion frequency information As input to the inverse NSST, the final fused image is obtained. , .

Images