A three-map multimodal medical image fusion method

Abstract

The application discloses a three-map multimodal medical image fusion method, first, using a method based on rolling guided filtering to cartoon texture decomposition of the image; second, according to the characteristics of the texture component containing more detailed information, and aiming at the medical features of the image to be fused, a sparse representation method using medical texture image as training dictionary sample is proposed for texture component fusion, and meanwhile, according to the high energy characteristics of the cartoon component, an adaptive absolute value taking large and Laplace energy sum method is proposed to fuse the cartoon component; finally, the final fusion image is obtained through reconstruction. The application solves the problem of obvious brightness information loss and edge blur in the fusion result of the multimodal medical image using the existing image fusion method.

Description

Technical Field

[0001] This invention relates to the field of image processing data fusion technology, and in particular to a method for fusion of three-image multimodal medical images. Background Technology

[0002] Image fusion utilizes digital image processing techniques and algorithms to extract features from two or more images of the same object, selecting the parameters that best represent the image information features for fusion and processing to obtain a new image. This achieves complementarity and redundancy elimination between images, thereby improving image clarity and contrast. Image fusion technology plays an important role in fields such as medical imaging, computer vision, and military applications.

[0003] Medical imaging technology plays an irreplaceable role in various clinical applications such as medical diagnosis and organ transplantation. With the development of various imaging devices, multimodal medical image fusion has become an important research topic for obtaining accurate clinical information. However, due to differences in imaging mechanisms, different modalities of medical images have different focuses in describing organs or tissues. Single-modal medical images are insufficient to provide a complete description of lesion information. To provide doctors with more diagnostic information, it is usually necessary to obtain medical images of different modalities, such as magnetic resonance imaging (MRI), single-photon emission computed tomography (SPECT), and positron emission tomography (PET). SPECT can accurately detect dense structures, such as bones and implants, and is often used to reflect the body's metabolic information, which is of great significance for improving medical technologies such as vascular disease diagnosis and tumor detection. Compared with low spatial resolution functional images, MR images obtained based on magnetic resonance technology can provide high-resolution soft tissue anatomical information, but they cannot reflect the characteristics of skeletal regions, nor can they reflect information on human metabolic activities. Among them, MR-T1 images can accurately reflect anatomical structure information to a certain extent, while MR-T2 images can reflect detailed information on tissue lesions. In clinical medicine, repeatedly analyzing lesion information from a single modality of medical images can significantly reduce diagnostic efficiency. It is worth noting that the useful information in some individual modalities is complementary. This makes it possible to fuse aligned medical images of the same target from different modalities into a single image containing comprehensive and accurate lesion information, thus helping doctors to diagnose and make decisions more easily and efficiently.

[0004] There are some advanced image fusion methods, such as Convolutional Sparse Coding and Morphological Component Analysis (CSMCA) and Convolutional Sparse Representation (CSR). However, the fusion results obtained by these methods have obvious problems of brightness information loss and edge blurring. Summary of the Invention

[0005] To address the aforementioned shortcomings, this invention proposes a three-image multimodal medical image fusion method, which aims to solve the problems of significant loss of brightness information and blurred edges in the fusion results obtained by existing image fusion methods for multimodal medical images.

[0006] To achieve this objective, the present invention adopts the following technical solution:

[0007] A three-image multimodal medical image fusion method includes the following steps:

[0008] Step S1: Input three multimodal medical images, namely a first source image, a second source image, and a third source image; wherein, the first source image and the second source image are grayscale images, and the third source image is a color image;

[0009] Step S2: Use rolling guided filtering to perform cartoon texture decomposition on the first source image and the second source image respectively, to obtain the cartoon component and texture component of the first source image, the cartoon component and texture component of the second source image;

[0010] Step S3: Use sparse representation to fuse the texture components of the first source image and the texture components of the second source image to obtain intermediate fused texture components;

[0011] Step S4: The cartoon components of the first source image and the second source image are fused using adaptive absolute value maximization and Laplacian energy to obtain an intermediate fused cartoon component;

[0012] Step S5: Overlay the intermediate blended texture component and the intermediate blended cartoon component to obtain the intermediate blended result image;

[0013] Step S6: Decompose the third source image into three channels: Y, U, and V. Then fuse the Y channel image of the third source image and the intermediate fusion result image according to the methods in steps S1 to S5 to obtain the fused image of the intermediate fused image and the Y channel image of the third source image.

[0014] Step S7: Combine the Y-channel image of the intermediate fused image and the third source image with the U-channel image and the V-channel image of the third source image to obtain the final fused result image.

[0015] Preferably, in step S2, the rolling guided filtering includes two sub-steps: a small structure removal sub-step and an edge recovery sub-step.

[0016] The small structure removal sub-step includes the following steps:

[0017] Substituting the input image I into the small structure removal function F(i), the calculation formula is as follows:

[0018]

[0019] Where F(i) is the small structure removal function, representing the output image, I(j) represents the input image, and K... i Represents the normalized function.

[0020]

[0021] N(i) represents the neighborhood pixel set centered at pixel i, σ s The parameter representing the control space weights, if the scale is less than σ s The filter can completely remove small structures.

[0022] Preferably, the edge recovery sub-step includes the following steps:

[0023] By analyzing the input image I and the guiding image J t By performing guided filtering, the output J of the t-th iteration can be obtained. t+1 The calculation formula is as follows:

[0024]

[0025] Among them, K i Represents the normalized function.

[0026]

[0027] σ s σ represents the parameter used to control the spatial weights. r J represents the parameter used to control the range weights. t+1 Indicated by J t The output image obtained after smoothing the input image I is the cartoon component of the source image, J. t The expression is used to calculate the approximation between pixels. N(i) represents the set of neighboring pixels centered at pixel i, where i represents the x-coordinate of a single pixel and j represents the y-coordinate of a single pixel.

[0028] Preferably, step S3 specifically includes the following steps:

[0029] Step S31: For the two texture components I to be fused in the first source image and the second source image... t1 and I t2 Each image block set is obtained by using a sliding window technique to overlap by S pixels, with each block being 8×8 pixels, and the blocks are extracted sequentially from the top left to the bottom right. and Where G represents the number of blocks, p represents the image block, and r represents the position of the image block;

[0030] Step S32: Draw each image patch into a corresponding column vector, that is...

[0031] Step S33: Select the sparse representation coefficients of the texture components using the rule of taking the largest l1 norm, and determine the sparse representation coefficients of the fused image. The calculation formula is as follows:

[0032]

[0033] F t1 This represents the fused texture component image of the first and second source images;

[0034] Step S34: Use the complete dictionary D. By performing linear representation, we obtain a vector. The calculation formula is as follows:

[0035]

[0036] Step S35: For and Steps S32-S34 are performed on each image patch to obtain a vector of fused texture components.

[0037] Step S36: Convert each vector into an 8×8 image patch and place them sequentially in their original positions to obtain the fused texture component F. t .

[0038] Preferably, step S4 specifically includes the following steps:

[0039] The difference between the two cartoon components Ic1 and Ic2 to be merged is compared with a set threshold; if the difference is greater than the threshold, then the cartoon component F is merged. c1 Select the pixel with the largest absolute value; if the difference is less than a threshold, compare the improved Laplacian energy of Ic1 and Ic2, and then fuse the cartoon component F. c1 Select the improved Laplacian energy and the largest pixel; the specific fusion calculation formula is as follows:

[0040]

[0041] Among them, F c1 (i,j) represents the cartoon component pixels of the fused image of I1 and I2, I c1 and I c2 These refer to the cartoon components of the two source images I1 and I2 to be fused, respectively, and ξ is a threshold greater than 0. and They represent I respectively c1 (i,j) and Ic2 The SML resolution of (i,j), where l represents scale and k represents direction.

[0042] Preferably, in step S5, the specific calculation formula for fusing the intermediate result image is as follows:

[0043] F m =F t1 +F c1

[0044] Among them, F m To merge the intermediate result images, F t1 For intermediate blending of texture components, F c1 It incorporates cartoon elements.

[0045] Preferably, the Orthogonal Matching Pursuit (OMP) algorithm is used. and Optimize the solution to obtain and Corresponding sparse representation coefficients and The calculation formula is as follows:

[0046]

[0047]

[0048] Where D is the overcomplete dictionary, ε is the sparse reconstruction error, r represents the position of the image patch, and I t1 and I t2 These represent the texture components of the two source images, respectively.

[0049] The technical solutions provided in this application embodiment may include the following beneficial effects:

[0050] This scheme employs a rolling guided filter to perform cartoon texture decomposition on the source image, addressing the issue of easily lost detail information in multi-scale decomposition. Then, based on the medical features of the images to be fused and to maintain spatial consistency, a sparse representation method using medical texture images as training dictionary samples is proposed for texture component fusion. Simultaneously, to preserve the energy and sharpness of the source images in the fused image, the cartoon components are treated with an adaptive absolute value maximization and Laplacian energy sum rule, demonstrating advantages over traditional transform domain algorithms. Compared to some advanced image fusion methods, this scheme effectively preserves the energy, structural, and detail information in multimodal medical images, exhibiting good performance in both visual quality and quantitative evaluation. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the steps of a three-image multimodal medical image fusion method.

[0052] Figure 2 This is a schematic diagram of one embodiment. Detailed Implementation

[0053] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0054] A three-image multimodal medical image fusion method includes the following steps:

[0055] Step S1: Input three multimodal medical images, namely a first source image, a second source image, and a third source image; wherein, the first source image and the second source image are grayscale images, and the third source image is a color image;

[0056] Step S2: Use rolling guided filtering to perform cartoon texture decomposition on the first source image and the second source image respectively, to obtain the cartoon component and texture component of the first source image, the cartoon component and texture component of the second source image;

[0057] Step S3: Use sparse representation to fuse the texture components of the first source image and the texture components of the second source image to obtain intermediate fused texture components;

[0058] Step S4: The cartoon components of the first source image and the second source image are fused using adaptive absolute value maximization and Laplacian energy to obtain an intermediate fused cartoon component;

[0059] Step S5: Overlay the blended texture component and the blended cartoon component to obtain the intermediate blending result image;

[0060] Step S6: Decompose the third source image into three channels: Y, U, and V. Then fuse the Y channel image of the third source image and the fusion intermediate result image according to the method of steps S1 to S5 to obtain the fused image of the intermediate image and the Y channel image of the third source image.

[0061] Step S7: Combine the fused image of the Y channel of the intermediate image and the third source image with the U channel of the third source image and the V channel of the third source image to obtain the final fused result image.

[0062] In this embodiment, as Figure 1-2 First, a rolling guided filter is used to perform cartoon texture decomposition on source images I1 and I2, decomposing the cartoon component I of source image I1. c1 and texture component It1 And the cartoon component I of the source image I2 c2 and texture component I t2 Cartoon elements I of source image I1 and source image I2 c1 and I c2 The fusion was performed using an adaptive method of maximizing absolute values ​​and summing Laplace energy to obtain the cartoon fusion component F. c1 Texture components I of source image I1 and source image I2 t1 and I t2 Using sparse representation for fusion, we obtain the texture fusion component F. t1 Finally, regarding the cartoon fusion component F c1 and texture blending component F t1 The intermediate fusion result image F is obtained by overlaying. m , merge intermediate result image F m Decomposed into F by rolling guided filtering m Cartoon elements F c2 and F m Texture component F t2 Simultaneously, the source image I3 is decomposed into three channels: Y, U, and V. The Y channel image of the source image I3 is then decomposed into cartoon texture components in the I3Y channel and texture components in the I3Y channel. m Cartoon elements F c2 The cartoon component of the I3Y channel is fused using an adaptive method of maximizing absolute values ​​and summing Laplace energy to obtain the final cartoon fusion component F. c3 ;F m Texture component F t2 The texture components of the I3Y channel are fused using sparse representation to obtain the final texture fusion component F. t3 Finally, the final cartoon fusion component F c3 Final texture blending component F t3 The U-channel image and V-channel image of source image I3 are combined to obtain the final fusion result image F.

[0063] This scheme employs a rolling guided filter to perform cartoon texture decomposition on the source image, addressing the issue of easily lost detail information in multi-scale decomposition. Then, based on the medical features of the images to be fused and to maintain spatial consistency, a sparse representation method using medical texture images as training dictionary samples is proposed for texture component fusion. Simultaneously, to preserve the energy and sharpness of the source images in the fused image, the cartoon components are treated with an adaptive absolute value maximization and Laplacian energy sum rule, demonstrating advantages over traditional transform domain algorithms. Compared to some advanced image fusion methods, this scheme effectively preserves the energy, structural, and detail information in multimodal medical images, exhibiting good performance in both visual quality and quantitative evaluation.

[0064] Preferably, in step S2, the rolling guided filtering includes two sub-steps: a small structure removal sub-step and an edge recovery sub-step.

[0065] The small structure removal sub-step includes the following steps:

[0066] Substituting the input image I into the small structure removal function F(i), the calculation formula is as follows:

[0067]

[0068] Where F(i) represents the small structure removal function, I(j) represents the input image, and K i This represents the standardized function.

[0069]

[0070] N(i) represents the set of neighboring pixels centered at pixel i, σ s The parameter representing the control space weights, if the scale is less than σ s The filter can completely remove small structures.

[0071] Specifically, removing small structures is part of the rolling guided filtering process, which facilitates the processing of the input image.

[0072] In this embodiment, three multimodal medical images are input: two medical MR anatomical images I1 and I2, and one medical SPECT / PET functional image I3. A rolling guided filter is used to first decompose the two source images I1 and I2 into cartoon textures. The rolling guided filter includes two steps: small structure removal and edge restoration. An image I can be considered as a cartoon component I. c and texture component I t The superposition of, i.e., I = I c +I t .

[0073] Preferably, the edge recovery sub-step includes the following steps:

[0074] By analyzing the input image I and the guiding image J t By performing guided filtering, the output J of the t-th iteration can be obtained. t+1 The calculation formula is as follows:

[0075]

[0076] Among them, K i Represents the normalized function.

[0077]

[0078] σs σ represents the parameter used to control the spatial weights. r J represents the parameter used to control the range weights. t+1 Indicated by J t The output image obtained after smoothing the input image I is the cartoon component of the source image, J. t The expression is used to calculate the approximation between pixels. N(i) represents the neighborhood pixel set centered at pixel i, where i represents the x-coordinate of a single pixel and j represents the y-coordinate of a single pixel.

[0079] Specifically, the edge restoration process mainly includes two steps: an iterative process and joint filtering. Guided filtering is chosen as the joint filtering method because it not only has high computational efficiency but also good edge preservation performance. During the iterative process, image I is continuously updated. J... 1 The output F, set as a Gaussian filter, is obtained by applying the input image I and the guiding image J. t By performing guided filtering, the output J of the t-th iteration can be obtained. t+1 .

[0080] In one embodiment, if t is set to 5, then through 5 rounds of guided filtering, the cartoon and texture components of the source image can be separated. 6 The images of the separated cartoon elements are shown. Let I1 and I2 be the images of the separated cartoon elements. c1 and I c2 Subtract the cartoon elements from I1 and I2 respectively to obtain the corresponding texture element image I. t1 and I t2 .

[0081] Preferably, step S3 specifically includes the following steps:

[0082] Step S31: For the two texture components I to be fused in the first source image and the second source image... t1 and I t2 Each image block set is obtained by using a sliding window technique to overlap by S pixels, with each block being 8×8 pixels, and the blocks are extracted sequentially from the top left to the bottom right. and Where G represents the number of blocks, p represents the image block, and r represents the position of the image block;

[0083] Step S32: Draw each image patch into a corresponding column vector, that is...

[0084] Step S33: Select the sparse representation coefficients of the texture components using the rule of taking the largest l1 norm, and determine the sparse representation coefficients of the fused image. The calculation formula is as follows:

[0085]

[0086] F t1 This represents the fused texture component image of the first and second source images;

[0087] Step S34: Use the complete dictionary D. By performing linear representation, we obtain a vector. The calculation formula is as follows:

[0088]

[0089] Step S35: For and Steps S32-S34 are performed on each image patch to obtain a vector of fused texture components.

[0090] Step S36: Convert each vector into an 8×8 image patch and place them sequentially in their original positions to obtain the fused texture component F. t .

[0091] In this embodiment, two texture components I to be fused are... t1 and I t2 Each image block is extracted using a sliding window technique, overlapping by S pixels (S is 6). The block size is 8×8, and blocks are extracted sequentially from the top left to the bottom right, resulting in their respective image block sets. and G represents the number of blocks. Taking a 256×256 image as an example, a total of 61504 image blocks are extracted, and then each image is stretched into a corresponding column vector, i.e. Normalization is not used here to avoid some pixels that were originally non-zero becoming zero after the difference between the original value and the mean, which would cause loss of medical image block information.

[0092] The absolute value of the sparse coefficients can effectively reflect the information of active pixels in the image. Here, the rule of taking the larger L1 norm is used to select the sparse representation coefficients of the texture components, thus determining the sparse representation coefficients of the final fused image. get Then, a linear representation of it is obtained using a complete dictionary D, resulting in a vector. Where D is obtained by training with medical texture images. and The above operation is performed on each image patch to obtain a vector of fused texture components. Each vector is converted into an 8×8 image patch and placed sequentially in its original position to obtain the fused texture component F.t .

[0093] Preferably, step S4 specifically includes the following steps:

[0094] The difference between the two cartoon components Ic1 and Ic2 to be merged is compared with a set threshold; if the difference is greater than the threshold, then the cartoon component F is merged. c1 Select the pixel with the largest absolute value; if the difference is less than a threshold, compare the improved Laplacian energy of Ic1 and Ic2, and then fuse the cartoon component F. c1 Select the improved Laplacian energy and the largest pixel; the specific fusion calculation formula is as follows:

[0095]

[0096] Among them, F c1 (i,j) represents the cartoon component pixels of the fused image of I1 and I2, I c1 and I c2 These refer to the cartoon components of the two source images I1 and I2 to be fused, respectively, and ξ is a threshold greater than 0. and They represent I respectively c1 (i,j) and I c2 The SML resolution of (i,j), where l represents scale and k represents direction.

[0097] In this embodiment, the cartoon components Ic1 and Ic2 of the source images I1 and I2 are subjected to a difference operation. This difference is compared with a set threshold ξ, which is set to 0.01 here. If the difference is greater than 0.01, the cartoon component F is fused. c1 Select the pixel with the largest absolute value; if the difference is less than 0.01, compare the improved Laplacian energy of Ic1 and Ic2, and then fuse the cartoon component F. c1 Choose the improved Laplace energy and the largest pixel. Similarly, F can be obtained. c3 (i,j), F c3 (i,j) represents the intermediate fusion result image F. m Cartoon-style pixels after being merged with I3.

[0098] To further explain, SML is characterized by measuring the degree of change in pixels. Pixels with larger metric values ​​are considered to originate from parts of the source image containing more useful information and are selected as pixels in the fused image. The metric SML at point (i,j) is calculated as the improved Laplacian energy sum, which, within a small window around (i,j), exceeds a threshold θ, as expressed below:

[0099]

[0100] Where θ is usually set to 0, Ω and The window size is determined here, with both parameters set to 5. γ and μ represent the horizontal and vertical pixel distances from the center point, respectively.

[0101] To further clarify, the Laplace operator is a second-order derivative operator, often used for extracting edge information. To overcome the fact that the second derivatives in the i and j directions may have opposite signs and easily cancel each other out, an improved Laplace operator (ML) is introduced, defined as follows:

[0102] ML l,k (i,j)=|2I l,k (i,j)-I l,k (i-step,j)-I l,k (i+step,j)|+|2I l,k (i,j)-I l,k (i,j-step)-I l,k (i,j+step)|

[0103] Use I l,k (i,j) represents the coefficient at point (i,j) along the k-axis at scale l, and step represents the variable spacing between coefficients, which is set to 1 here.

[0104] Preferably, in step S5, the specific calculation formula for fusing the intermediate result image is as follows:

[0105] F m =F t1 +F c1

[0106] Among them, F m To merge the intermediate result images, F t1 For intermediate blending of texture components, F c1 It incorporates cartoon elements.

[0107] In this embodiment, the fused texture component F t1 and the fusion of cartoon elements F c1 The images are overlaid to obtain the intermediate fusion result image F. m This is beneficial for the formation of the final fusion result image.

[0108] Preferably, the Orthogonal Matching Pursuit (OMP) algorithm is used. and Optimize the solution to obtain and Corresponding sparse representation coefficients and The calculation formula is as follows:

[0109]

[0110]

[0111] Where D is the overcomplete dictionary, ε is the sparse reconstruction error, r represents the position of the image patch, and I t1 and I t2 These represent the texture components of the two source images, respectively.

[0112] Specifically, OMP refers to the Orthogonal Matching Pursuit algorithm, which has a fast convergence speed. It further improves the... and The optimization level of solving the sparse representation vector. In this embodiment, the sparse reconstruction error ε is set to 0.01.

[0113] To further explain, there are two main methods for dictionary learning: the first is a fixed dictionary based on analytical expressions, and the second is a dictionary learned from samples. This application uses the second method to obtain the dictionary, using K-SVD to obtain an overcomplete dictionary D. The overcomplete dictionary determines the signal representation capability of sparse coding.

[0114] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0115] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for fusion of three-image multimodal medical images, characterized in that, Includes the following steps: Step S1: Input three multimodal medical images, namely a first source image, a second source image, and a third source image; wherein, the first source image and the second source image are grayscale images, and the third source image is a color image; Step S2: Use rolling guided filtering to perform cartoon texture decomposition on the first source image and the second source image respectively, to obtain the cartoon component and texture component of the first source image, the cartoon component and texture component of the second source image; Step S3: Use sparse representation to fuse the texture components of the first source image and the texture components of the second source image to obtain intermediate fused texture components; Step S4: The cartoon components of the first source image and the second source image are fused using adaptive absolute value maximization and Laplacian energy to obtain an intermediate fused cartoon component; Step S5: Overlay the intermediate blended texture component and the intermediate blended cartoon component to obtain the intermediate blended result image; Step S6: Decompose the third source image into three channels: Y, U, and V. Then fuse the Y channel image of the third source image and the intermediate fusion result image according to the method of steps S1 to S5 to obtain the fused image of the intermediate fused image and the Y channel image of the third source image. Step S7: Combine the Y-channel image of the intermediate fused image and the third source image with the U-channel image and the V-channel image of the third source image to obtain the final fused result image; In step S2, the rolling guided filtering includes two sub-steps: a small structure removal sub-step and an edge recovery sub-step. The small structure removal sub-step includes the following steps: Substitute the input image I into the small structure removal function F( i The calculation formula is as follows: Among them, F( i ) is a small structure removal function, used to represent the output image, I( j ) represents the input image. Represents the normalized function. N(i) represents the pixel i The central domain pixel set, The parameter representing the control of spatial weights, if the scale is smaller than... The filter can completely remove small structures; The edge recovery sub-step includes the following steps: By analyzing the input image I and the guiding image J t By performing guided filtering, we can obtain the first... t The output J of the next iteration process t+1 The calculation formula is as follows: ； ； This represents the parameters used to control spatial weights. J represents the parameter used to control the range weights. t+1 Indicated by J t The output image obtained after smoothing the input image I is the cartoon component of the source image, J. t N( represents the approximation used to calculate the approximation between pixels) i ) indicates in pixels i The set of pixels in the neighborhood of the center, i Represents the x-coordinate of a single pixel. j This represents the ordinate of a single pixel.

2. The method for fusion of three-image multimodal medical images according to claim 1, characterized in that: Step S3 specifically includes the following steps: Step S31: For the two texture components I to be fused in the first source image and the second source image... t1 and I t2 Each employs a sliding window block extraction technique to achieve overlap. S Each pixel is represented by an 8×8 block. Blocks are extracted sequentially from the top left to the bottom right to obtain their respective image block sets. and Where G represents the number of blocks. p Represents an image patch. r Indicates the position of the image patch; Step S32: Draw each image patch into a corresponding column vector, that is... , ; Step S33: Use The rule of selecting the largest norm is used to choose the sparse representation coefficients of the texture components, thus determining the sparse representation coefficients of the fused image. The calculation formula is as follows: This represents the fused texture component image of the first and second source images; Step S34: Use the complete dictionary D. By performing linear representation, we obtain a vector. The calculation formula is as follows: =D Step S35: For and Steps S32-S34 are performed on each image patch to obtain a vector of fused texture components. ; Step S36: Convert each vector into an 8×8 image patch and place them sequentially in their original positions to obtain the fused texture component F. t .

3. The method for fusion of three-image multimodal medical images according to claim 1, characterized in that: Step S4 specifically includes the following steps: For the two cartoon elements I to be merged c 1 and I c The difference between 2 and the set threshold is compared; if the difference is greater than the threshold, the cartoon elements are merged. Select the pixel with the largest absolute value; if the difference is less than the threshold, then compare I. c 1 and I c 2's improved Laplace energy and, incorporating cartoon elements Select the improved Laplacian energy and the largest pixel; the specific fusion calculation formula is as follows: in, Represents the cartoon component pixels of the fused I1 and I2 image, I c1 and I c2 These refer to the cartoon components of the two source images I1 and I2 to be fused. It is a threshold greater than 0. and They represent and The clarity of SML, l Indicates scale; k Indicates direction.

4. The method for fusion of three-image multimodal medical images according to claim 1, characterized in that: In step S5, the specific calculation formula for fusing the intermediate result image is as follows: = in, To merge the intermediate result images, For intermediate blending of texture components, It incorporates cartoon elements.

5. The method for fusion of three-image multimodal medical images according to claim 2, characterized in that: Using the Orthogonal Matching Pursuit (OMP) algorithm and Optimize the solution to obtain and Corresponding sparse representation coefficients and The calculation formula is as follows: arg arg Where D is an overcomplete dictionary. For sparse reconstruction error, r Indicates the position of an image patch, I t1 and I t2 These represent the texture components of the two source images, respectively.

Images