A three-modality medical image fusion method and system based on double models

Abstract

This invention discloses a trimodal medical image fusion method and system based on a dual-model approach. The method includes: performing a three-layer decomposition process on a multimodal medical image using a bitonicX filter; sequentially fusing and adding the decomposed multimodal medical images to obtain a preliminary multimodal medical image fusion result; decomposing the preliminary multimodal medical image fusion result and the original multimodal medical image to obtain corresponding multimodal medical images to be fused; and sequentially fusing and adding the images to be fused to obtain the final multimodal medical image fusion result. The system includes a first decomposition module, a first fusion module, a second decomposition module, and a second fusion module. By using this invention, useful information from different modalities of medical images can be integrated, providing high-quality fusion results. As a trimodal medical image fusion method and system based on a dual-model approach, this invention can be widely applied in the field of image fusion technology.

Description

Technical Field

[0001] This invention relates to the field of image fusion technology, and in particular to a three-modal medical image fusion method and system based on a dual-model approach. Background Technology

[0002] Medical imaging is diverse. Anatomical medical images, such as computed tomography (CT), can accurately reflect the structural information of tissues or organs with small density differences. Magnetic resonance imaging (MRI) has good soft tissue contrast. MR-T1 imaging can reflect the core area of ​​a tumor, while MR-T2 imaging can highlight the tumor area and peritumoral edema. Functional medical images, such as single-photon emission computed tomography (SPECT) and positron emission tomography (PET), can reflect physiological information such as metabolism and blood flow. However, a single imaging method can only provide limited diagnostic information for the intended purpose. For doctors to diagnose diseases, they need to analyze images of different modalities of the same site one by one, which limits the diagnostic efficiency and accuracy. Summary of the Invention

[0003] To address the aforementioned technical problems, the present invention aims to provide a trimodal medical image fusion method and system based on a dual-model approach, which can integrate useful information from different modalities of medical images and provide high-quality fusion results.

[0004] The first technical solution adopted in this invention is: a three-modal medical image fusion method based on a dual-model approach, comprising the following steps:

[0005] The multimodal medical image is decomposed into three layers using a bitonicX filter to obtain the decomposed multimodal medical image to be fused.

[0006] Based on the preset fusion rules, the decomposed multimodal medical images to be fused are sequentially fused and added to obtain preliminary multimodal medical image fusion results;

[0007] The preliminary multimodal medical image fusion results and the multimodal medical image are decomposed based on the bitonicX filter to obtain the corresponding multimodal medical image to be fused.

[0008] The multimodal medical images to be fused are sequentially fused and added to obtain the final multimodal medical image fusion result.

[0009] Furthermore, the step of performing a three-layer decomposition process on the multimodal medical image using a bitonicX filter to obtain the decomposed multimodal medical image to be fused specifically includes:

[0010] Acquire multimodal medical images, which include a first multimodal medical image, a second multimodal medical image, and a third multimodal medical image;

[0011] The first and second multimodal medical images are filtered using a bitonicX filter to obtain the corresponding energy layer and detail layer to be fused.

[0012] The secondary energy layer to be fused is obtained by performing a secondary filtering process on the energy layer to be fused using a bitonicX filter.

[0013] The energy layer to be fused is subtracted from the secondary energy layer to be fused to obtain the texture layer to be fused.

[0014] By integrating the sub-energy layer, detail layer, and texture layer to be fused, a decomposed multimodal medical image to be fused is constructed.

[0015] Furthermore, the step of sequentially fusing and adding the decomposed multimodal medical images to be fused based on preset fusion rules to obtain preliminary multimodal medical image fusion results specifically includes:

[0016] Considering the gradient information of multimodal medical images, the fused detail layers in the decomposed multimodal medical images are fused by combining multi-scale morphological gradient operators and parameter adaptive coupled neural networks to obtain the fused detail layers.

[0017] A fusion strategy based on the coupled neural P system is used to fuse the texture layers in the decomposed multimodal medical images to be fused, resulting in fused texture layers.

[0018] The energy layers to be fused in the decomposed multimodal medical images are fused based on the fusion rule of "taking the largest absolute value" to obtain the fused energy layers.

[0019] The fused detail layer, fused texture layer, and fused energy layer are added together to obtain preliminary multimodal medical image fusion results.

[0020] Furthermore, the step of considering the gradient information of multimodal medical images and fusing the detail layers to be fused in the decomposed multimodal medical images by combining multi-scale morphological gradient operators and parameter adaptive coupled neural networks to obtain the fused detail layers specifically includes:

[0021] Calculate the multi-scale morphological gradients of the first multimodal medical image and the second multimodal medical image and take their absolute values ​​to obtain the first gradient value and the second gradient value;

[0022] The first gradient value and the second gradient value are input into the parameter adaptive coupling neural network for adaptive calculation to obtain the first dynamic threshold and the second dynamic threshold.

[0023] The first dynamic threshold and the second dynamic threshold are compared, and a first decision map and a second decision map are generated based on the comparison result.

[0024] The first decision map and the second decision map are multiplied with the first multimodal medical image and the second multimodal medical image, respectively, to obtain the fused detail layer.

[0025] Furthermore, the specific expression of the fusion strategy based on the coupled neural P system is as follows:

[0026]

[0027]

[0028]

[0029] In the above formula, I t M represents the texture layer to be blended. t (i,j) represents the initial decision graph. This represents the decision graph obtained after consistency verification. Representing neuron σ i,j exist and The number of ignitions in the middle, and These represent the texture layers of the first and second multimodal medical images, respectively. In the corresponding CNP system, Ω represents the consistency verification window size, and a and b represent the horizontal and vertical pixel distances from (i,j), respectively.

[0030] Furthermore, the specific expression for the fusion rule of "taking the larger absolute value" is as follows:

[0031]

[0032]

[0033] In the above formula, I represents the decision map obtained by comparing the pixel sizes of the sub-energy layers of the first and second multimodal medical images. e1 (i,j) represents the fusion energy layer of the first and second multimodal medical images. This represents the energy layer to be fused in the first multimodal medical image. This represents the energy layer to be fused in the second multimodal medical image.

[0034] Furthermore, the multimodal medical images to be fused include a pre-fusion detail layer of the preliminary multimodal medical image fusion result, a pre-fusion energy layer of the preliminary multimodal medical image fusion result, a pre-fusion texture layer of the preliminary multimodal medical image fusion result, a pre-fusion detail layer of the third multimodal medical image, and a pre-fusion energy layer of the third multimodal medical image. The step of sequentially fusing and adding the multimodal medical images to be fused to obtain the final multimodal medical image fusion result specifically includes:

[0035] A gradient energy clarity operator is introduced to fuse the pre-fusion detail layer of the multimodal medical image to be fused with the pre-fusion detail layer of the preliminary multimodal medical image fusion result, so as to obtain the fused detail layer.

[0036] The pre-fusion energy layer of the multimodal medical image to be fused is fused based on the fusion rule of "taking the largest absolute value" to obtain the fused energy layer;

[0037] The fused detail layer and the fused energy layer are added together to obtain the final multimodal medical image fusion result.

[0038] Furthermore, the step of introducing the gradient energy clarity operator to fuse the pre-fused detail layer of the multimodal medical image to be fused with the pre-fused detail layer of the preliminary multimodal medical image fusion result to obtain the fused detail layer specifically includes:

[0039] The gradient energy clarity operator is used to measure pixel activity in the pre-fusion detail layer of the multimodal medical image to be fused and the pre-fusion detail layer of the preliminary multimodal medical image fusion result, so as to obtain the saliency map of the multimodal medical image to be fused and the saliency map of the preliminary multimodal medical image fusion result.

[0040] The pixels of the saliency map of the preliminary multimodal medical image fusion result and the saliency map of the multimodal medical image to be fused are compared and consistent with each other in turn to obtain the decision map of the detail layer of the multimodal medical image to be fused.

[0041] Based on the preset detail layer fusion rules, the preliminary multimodal medical image fusion results, the decision map of the detail layer of the third multimodal medical image and the multimodal medical image to be fused are fused to obtain the fused detail layer.

[0042] The expression for the preset detail layer fusion rule is as follows:

[0043]

[0044]

[0045]

[0046] In the above formula, I d2 (i,j) represents the merged detail layer. M represents the improved decision graph obtained after consistency verification. d2 (i,j) represents the result obtained through GEC. mid (i,j) and GEC C The detail layer of the preliminary multimodal medical image fusion result obtained by comparing pixels (i,j) is compared with the initial decision map of the detail layer of the multimodal medical image, GEC. mid (i,j) and GEC C (i,j) represent the pairs of gradient energy clear operators designed for each other. and The saliency map obtained from the detection, Ω represents a local region of size r×r centered at (i,j).

[0047] The second technical solution adopted in this invention is: a trimodal medical image fusion system based on a dual-model approach, comprising:

[0048] The first decomposition module is used to perform three-level decomposition processing on the multimodal medical image through the bitonicX filter to obtain the decomposed multimodal medical image to be fused.

[0049] The first fusion module, based on preset fusion rules, sequentially performs fusion and addition processing on the decomposed multimodal medical images to be fused, to obtain preliminary multimodal medical image fusion results;

[0050] The second decomposition module decomposes the preliminary multimodal medical image fusion result and the multimodal medical image based on the bitonicX filter to obtain the corresponding multimodal medical image to be fused.

[0051] The second fusion module is used to sequentially fuse and add the multimodal medical images to be fused to obtain the final multimodal medical image fusion result.

[0052] The beneficial effects of the method and system of this invention are as follows: By introducing a bitonicX filter into image fusion, this invention can effectively decompose the energy and detail information in the source medical image, allowing for more targeted and effective fusion rules for the energy and detail layers. It can also remove noise signals, effectively solving the problem of noisy image fusion. Furthermore, the gradient energy clarity operator designed in this invention, along with the newly proposed operator combining multi-scale morphological gradients and parameter adaptive coupled neural networks, can effectively detect the edge structure, detail information, and energy information of tissues or organs in the source image and transfer them to the fused image. This helps clinicians determine the location and type of disease. Furthermore, the designed coupled neural P system rules for the fusion texture layer can effectively extract texture information and preserve energy, while avoiding the loss of brightness and contrast information during the fusion process, improving the visual perception of the fused image and increasing the accuracy of clinical diagnosis. Attached Figure Description

[0053] Figure 1 This is a flowchart of the steps of a three-modal medical image fusion method based on a dual-model according to the present invention;

[0054] Figure 2 This is a structural block diagram of a trimodal medical image fusion system based on a dual-model according to the present invention;

[0055] Figure 3 This is a structural block diagram of the trimodal medical image fusion method and system based on a dual-model approach of the present invention;

[0056] Figure 4 These are multimodal medical images in specific embodiments of the present invention;

[0057] Figure 5 This is a schematic diagram comparing the fusion results obtained by the image fusion algorithm of this invention with those obtained by existing methods. Detailed Implementation

[0058] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

[0059] Reference Figure 1 and Figure 3 This invention provides a trimodal medical image fusion method based on a dual-model approach, which includes the following steps:

[0060] S1. Acquire multimodal medical images, including a first multimodal medical image, a second multimodal medical image, and a third multimodal medical image, namely, anatomical medical image I.A ,I B and functional medical images I C ;

[0061] S21. Using the bitonicX filter to process I A and I B Filtering yields the corresponding energy layer and detail layer;

[0062] Specifically, its expression is as follows:

[0063]

[0064]

[0065]

[0066]

[0067] In the above formula, R δ,c Let represent the rank filter, used to arrange pixels within a range of δ. c represents the percentage; c = 0% returns the minimum value, and c = 100% returns the maximum value. When c is 5%, the algorithm is more robust to noise. δ represents the filter window, whose shape at each x position is unconstrained and controls the pixels to be processed in forward and reverse arrangement operations. Each element y in the filter window δ(x) is the offset from the center pixel x to a pixel within a filter region of size l × l. C δ,c and O δ,c These represent the close and open operations, respectively.

[0068] Furthermore, the expressions for the energy layers of the first and second multimodal medical images are as follows:

[0069]

[0070]

[0071]

[0072] Similarly, we can obtain in, Represents Gaussian filtering, ρ and ε represents the parameters controlling the window size of the Gaussian filter as (2ρ+1)×(2ρ+1) and the Gaussian smoothness, respectively. O and ε C This indicates removing C from the original signal. δ,c and O δ,c The smoothed version of the detected bitonic signal, and the expressions for the detail layers of the first and second multimodal medical images are shown below:

[0073]

[0074]

[0075] S22. Using the bitonicX filter to process I A and I B Energy layer and Filtering can yield the corresponding sub-energy level. and Texture layers can be obtained by subtracting the energy layer and the sub-energy layer of the source image;

[0076] Specifically, its expression is:

[0077]

[0078]

[0079] In the above formula, and Represents source image I A and I B Texture layer.

[0080] S31, Targeting I A and I B Detail layer image and A detection operator pair combining multi-scale morphological gradient and parameter adaptive coupling network was designed. and Fusion, resulting in I A and I B The pre-fusion detail layer;

[0081] Specifically, its expression is as follows:

[0082]

[0083]

[0084]

[0085]

[0086] In the above formula, MMG A Indicate I A Detail layer Similarly, the salient image obtained after processing with multi-scale morphological operators can be used to obtain the MMG. B , ⊙ and ⊙ represent the dilation and erosion operations respectively, (i, j) represent the polar coordinates of the pixel, BS1 represents the structuring element, and BS n Representing multi-scale structured elements, gradient features I can be used A Detail layer image The morphological gradient operator representation, where u represents the scale number, w u Represents the gradient weights at the u-th scale;

[0087] Will and MMG obtained by multi-scale morphological gradient operator calculation A and MMG B Take the absolute value and get |MMG A |and|MM B The input to the PA-PCNN network, i.e., a parameter-adaptive coupled neural network, is used until all neurons are activated to obtain the dynamic threshold. and The specific expression is as follows:

[0088]

[0089]

[0090]

[0091]

[0092]

[0093] Similarly, As input to the PA-PCNN network,

[0094] R ij [n] is defined as the input image I during the iterative calculation process. ij The strength, R ij [n] and C ij [n] represents the input and connection input for the nth iteration at (i, j), C ij [n] is connected to eight neighboring neurons through synaptic weights, which can be expressed as: K L Indicates the amplitude of the connected input, internal activity P ij [n] contains a decay term of the previous value. and R ij [n] and C ij The nonlinear modulation term R of [n] ij [n](1+γC ij[n]), γ represents the connection strength, Y ij [n] represents the output value; an output of 1 indicates ignition, and an output of 0 indicates no ignition. K E and β e Representing E respectively ij The amplitude and attenuation coefficient of [n], since the initial value of this model is Y ij [n] = 0, P ij [n] = 0 and E ij [n] = 0, therefore, due to the validity of the ignition condition, neurons with non-zero intensity can be ignited in the first iteration (P). ij [1] = I ij >0);

[0095] The above formula has 5 free parameters, namely β f γ, KL, β e and K E , where γ and K L It is K L ∑ kl W ijlm Y lm The weights [n-1] can be considered as a whole γK in the model. L Processing, assuming λ = γK L Representing the weighted connection strength, all parameters can be adaptively calculated, and their specific expressions are as follows:

[0096] β f =log[σ(I)] -1 ]

[0097]

[0098] K E =e -βf +1+6λ

[0099]

[0100] In the above formula, σ(I) represents the standard deviation of the input image I, I max I and I′ represent the maximum intensity of the input source image and the normalized Ostu threshold, respectively.

[0101] S32, By comparison and Corresponding dynamic threshold and The value can be used to obtain the decision map of the detail layer of the first two source images.

[0102] Specifically, the expression for the decision map of the detail layer of the first two source images is as follows:

[0103]

[0104] Multiplying the decision map of the intermediate fusion result (i.e., the preliminary multimodal medical image fusion result) with the source image yields the pre-fusion result of the detail layer. d1 Its specific expression is as follows:

[0105]

[0106] In the above formula, I d1 Indicates intermediate anatomical images and Detail layer fusion result and The dynamic threshold is obtained by the parameter adaptive coupling neural P system. Indicates comparison and The resulting detailed decision diagram.

[0107] S4, targeting I A and I B Detail layer image and A fusion strategy based on the coupled neural P system was utilized;

[0108] Specifically, the expression for the fusion strategy based on the coupled neural P system is as follows:

[0109]

[0110]

[0111] In the above formula, and The corresponding CNP systems with local topology are respectively and These are used as external inputs to the two systems, respectively. The CNP system starts from the initial state and runs until it reaches the maximum number of iterations S. and Representing neuron σ i，j exist and The number of ignitions in the middle, For the initial decision graph M t The final decision graph of the texture layer obtained after consistency verification can be corrected by consistency verification. t To improve the accuracy of pixel selection in the fused structural layer, pixels that may make incorrect decisions are identified. Ω is the size of the consistency verification window, and a and b represent the horizontal and vertical pixel distances (i, j), respectively. Their specific expressions are shown below:

[0112]

[0113] Will I A and I B Decomposed texture layers and Multiplying the result by the decision map yields the fusion result of the texture layers from the first two anatomical medical images. t .

[0114] S5. Apply the rule of taking the largest absolute value to the sub-energy layer I of the first two medical images. A and I B To merge;

[0115] Specifically, the expression for the fusion rule that takes the larger absolute value is as follows:

[0116]

[0117]

[0118] In the above formula, Indicates to I A and I B The decision map obtained by comparing the size of pixels in the sub-energy layer, I e1 This represents the fusion of the first two anatomical medical images into a sub-energy layer image.

[0119] S6. The pre-fused detail layer, texture layer, and energy layer calculated from the first two anatomical medical images are added together to obtain the intermediate fused image I. mid ;

[0120] Specifically, the expression for the intermediate fused image is as follows:

[0121] I mid (i, j) = I d1 (i, j) + I t (i, j) + I e1 (i, j)

[0122] S7. Use bitonicX filtering to process the intermediate fused image I mid And the third functional medical image I C Decomposition yields the energy layer of the intermediate fused image. And the energy layer of the third functional medical image I mid and I C The detail layer can be obtained by subtracting the corresponding energy layer from the source image;

[0123] Specifically, its expression is as follows:

[0124]

[0125]

[0126] S81. A novel gradient energy clarity operator was designed to fuse the detail layers of the intermediate fused image and the third functional medical image, resulting in a pre-fused detail layer I. d2 ;

[0127] Specifically, its expression is as follows:

[0128]

[0129] Similarly, we can obtain GE c In the above formula, GE mid This indicates that the newly designed gradient energy clarity operator is applied to the intermediate fused image I. mid The saliency map obtained from pixel activity measurement, similarly, for I C GE can be obtained using the gradient energy clear operator detection. c (i, j) represents pixel coordinates, and α, β, and γ are the pixel coordinates in GEC. mid Adjustment of STO mid , and LCCM mid The weight parameters, by adjusting their magnitudes, control the proportions of the gradient, energy, and sharpness features in the detail layer activity detection process. mid (i, j) is the saliency map obtained by the structure tensor operator detection, which contains rich gradient features and can clearly represent the texture and structure of fibrous tissue or organs in the source medical image. Here, LCCM is used as an energy detection sub-operator. mid The sub-operator representing local sharpness change can effectively detect image contrast information, and its expression is shown below:

[0130]

[0131]

[0132] In the above formula, CCM(i,j) represents the unoptimized sharpness variation index, Θ represents the size of the k×k local region centered at (i,j), LCCM(i,j) is the local sharpness variation index, and (2p+1)×(2q+1) is the neighborhood size. Compared with CCM(i,j), LCCM(i,j) enhances the correlation between pixels in the neighborhood of (i,j) to a certain extent.

[0133] S82, the blended detail layer;

[0134] Specifically, the fusion detail layer can be represented as:

[0135]

[0136]

[0137]

[0138] In the above formula, M d2 By analyzing the saliency map GEC mid (i,j) and GEC C The intermediate fused image obtained by comparing pixels (i, j) is compared with the initial decision map of the detail layer of the third functional medical image, GEC. mid (i,j) and GEC C (i, j) represent the pairs of newly designed gradient energy clear operators. and The saliency map obtained from the detection, The improved decision graph obtained after consistency verification can reduce M to some extent. d2 In the erroneous decision pixel, Ω represents a local region of size r×r centered at (i, j).

[0139] S9, Energy layer for intermediate fused images And the energy layer of the third image The fusion still uses the simple and effective rule of taking the largest absolute value for fusion;

[0140] Specifically, the expression for the absolute value fusion rule is as follows:

[0141]

[0142]

[0143] In the above formula, This indicates a comparison of the intermediate fused image energy layer and... The decision graph obtained from the comparison, I e2 This represents an image of energy layer fusion.

[0144] S10, merge the intermediate images I mid And the third functional medical image I C The pre-fusion detail layer and pre-fusion energy layer can be used to obtain the final fused image F, which specifically includes;

[0145] Specifically, the expression for the final fused image is as follows:

[0146] F(i,j)=I d2(i, j) + I e2 (i, j)

[0147] In the above formula, F(i,j) represents the final multimodal medical image fusion result, and I d2 (i, j) represents the fusion detail layer of the preliminary multimodal medical image fusion result and the multimodal medical image, I e2 (i, j) represents the initial multimodal medical image fusion result and the fusion energy layer of the multimodal medical image.

[0148] The simulation experiment of this invention is shown in detail below:

[0149] This invention was compared with five state-of-the-art image fusion algorithms in a series of experiments, and the advantages and disadvantages of each algorithm were analyzed from a subjective visual evaluation perspective. Figure 4 (a1), (a2), and (a3) ​​are three multimodal medical images (each 256×256 pixels). Figure 5 Images (a1) to (a6) show the fusion results and difference maps of the following methods in the non-subsampled contour wave transform domain: National Standard Medical Image Fusion (NSCT) based on phase consistency and local Laplacian energy, National Standard Medical Image Fusion (NSST) based on edge-preserving filtering and multi-scale morphology, image fusion based on convolutional neural networks (CNN), real-time image fusion based on compression and decomposition networks (SDNet), image fusion based on unsupervised networks (U2Fusion), and the fusion algorithm proposed in this scheme. It is clear that the fusion result obtained by the U2Fusion method suffers some loss in color. Figure 5 (a5)), the white information in the anatomical image source obtained by the NSST method is almost completely covered by the functional information of the third multimodal medical image. Figure 5 (a2)) The fusion result obtained by the SDNet method almost completely loses the texture information in the anatomical source image of the part covered by the functional information of the third multimodal medical image. Although the fusion results obtained by the NSCT and CNN methods are better than NSST, SDNet and U2Fusion in terms of color and detail preservation, they still have a certain gap compared with the fusion results obtained by the fusion method proposed in this scheme in terms of edge details and color preservation. In summary, the fusion performance of the present invention is in a leading position compared with the existing five comparative methods.

[0150] Reference Figure 2 A trimodal medical image fusion system based on a dual-model approach includes:

[0151] The first decomposition module is used to perform three-level decomposition processing on the multimodal medical image through the bitonicX filter to obtain the decomposed multimodal medical image to be fused.

[0152] The first fusion module, based on preset fusion rules, sequentially performs fusion and addition processing on the decomposed multimodal medical images to be fused, to obtain preliminary multimodal medical image fusion results;

[0153] The second decomposition module decomposes the preliminary multimodal medical image fusion result and the multimodal medical image based on the bitonicX filter to obtain the corresponding multimodal medical image to be fused.

[0154] The second fusion module is used to sequentially fuse and add the multimodal medical images to be fused to obtain the final multimodal medical image fusion result.

[0155] In summary, the model-based trimodal medical image fusion method of this invention plays an important role in the differential diagnosis of various diseases, disease assessment, clinical patient management, and new drug development.

[0156] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0157] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A trimodal medical image fusion method based on a dual-model approach, characterized in that, Includes the following steps: The multimodal medical image is decomposed into three layers using a bitonicX filter to obtain the decomposed multimodal medical image to be fused. The step of performing a three-layer decomposition process on the multimodal medical image using a bitonicX filter to obtain the decomposed multimodal medical image to be fused specifically includes: Acquire multimodal medical images, which include a first multimodal medical image, a second multimodal medical image, and a third multimodal medical image; The first and second multimodal medical images are filtered using a bitonicX filter to obtain the corresponding energy layer and detail layer to be fused. The secondary energy layer to be fused is obtained by performing a secondary filtering process on the energy layer to be fused using a bitonicX filter. The energy layer to be fused is subtracted from the secondary energy layer to be fused to obtain the texture layer to be fused. Integrate the sub-energy layer, detail layer, and texture layer to be fused to construct the decomposed multimodal medical image to be fused; Based on the preset fusion rules, the decomposed multimodal medical images to be fused are sequentially fused and added to obtain preliminary multimodal medical image fusion results; The preliminary multimodal medical image fusion result and the third multimodal medical image are decomposed based on the bitonicX filter to obtain the corresponding multimodal medical image to be fused. The corresponding multimodal medical images to be fused are sequentially fused and added to obtain the final multimodal medical image fusion result; The step of performing sequential fusion and addition processing on the decomposed multimodal medical images to be fused based on preset fusion rules to obtain preliminary multimodal medical image fusion results specifically includes: Considering the gradient information of multimodal medical images, the fused detail layers in the decomposed multimodal medical images are fused by combining multi-scale morphological gradient operators and parameter adaptive coupled neural networks to obtain the fused detail layers. A fusion strategy based on the coupled neural P system is used to fuse the texture layers in the decomposed multimodal medical images to be fused, resulting in fused texture layers. The fusion process is performed on the sub-energy layers to be fused in the decomposed multimodal medical images based on the fusion rule of "taking the largest absolute value" to obtain the fused energy layer. The fused detail layer, fused texture layer, and fused energy layer are added together to obtain preliminary multimodal medical image fusion results.

2. The trimodal medical image fusion method based on a dual-model approach according to claim 1, characterized in that, The step of considering the gradient information of multimodal medical images and fusing the detail layers to be fused in the decomposed multimodal medical images by combining multi-scale morphological gradient operators and parameter adaptive coupled neural networks to obtain the fused detail layers specifically includes: Calculate the multi-scale morphological gradients of the first multimodal medical image and the second multimodal medical image and take their absolute values ​​to obtain the first gradient value and the second gradient value; The first gradient value and the second gradient value are input into the parameter adaptive coupling neural network for adaptive calculation to obtain the first dynamic threshold and the second dynamic threshold. The first dynamic threshold and the second dynamic threshold are compared, and a first decision map and a second decision map are generated based on the comparison result. The first decision map and the second decision map are multiplied with the first multimodal medical image and the second multimodal medical image, respectively, to obtain the fused detail layer.

3. The trimodal medical image fusion method based on a dual-model approach according to claim 2, characterized in that, The specific expression for the fusion strategy based on the coupled neural P system is as follows: In the above formula, This represents the texture layer to be merged. Represents the initial decision graph. This represents the decision graph obtained after consistency verification. , They represent neurons respectively. exist and Number of ignitions in the middle, and These represent the texture layers of the first and second multimodal medical images, respectively. , The corresponding CNP system, Indicates the size of the consistency verification window. , They represent distances respectively. The horizontal and vertical pixel distance.

4. The trimodal medical image fusion method based on a dual-model approach according to claim 3, characterized in that, The specific expression for the "larger absolute value" fusion rule is as follows: In the above formula, This represents a decision map obtained by comparing the pixel sizes of sub-energy layers in the first and second multimodal medical images. This represents the fusion energy layer of the first and second multimodal medical images. This represents the energy layer to be fused in the first multimodal medical image. This represents the energy layer to be fused in the second multimodal medical image.

5. The trimodal medical image fusion method based on a dual-model approach according to claim 4, characterized in that, The multimodal medical images to be fused include a pre-fusion detail layer of the preliminary multimodal medical image fusion result, a pre-fusion energy layer of the preliminary multimodal medical image fusion result, a pre-fusion texture layer of the preliminary multimodal medical image fusion result, a pre-fusion detail layer of the third multimodal medical image, and a pre-fusion energy layer of the third multimodal medical image. The step of sequentially fusing and adding the multimodal medical images to be fused to obtain the final multimodal medical image fusion result specifically includes: A gradient energy clarity operator is introduced to fuse the pre-fusion detail layer of the multimodal medical image to be fused with the pre-fusion detail layer of the preliminary multimodal medical image fusion result, so as to obtain the fused detail layer. The pre-fusion energy layer of the multimodal medical image to be fused is fused based on the fusion rule of "taking the largest absolute value" to obtain the fused energy layer; The fused detail layer and the fused energy layer are added together to obtain the final multimodal medical image fusion result.

6. The trimodal medical image fusion method based on a dual-model approach according to claim 5, characterized in that, The step of introducing the gradient energy clarity operator to fuse the pre-fusion detail layer of the multimodal medical image to be fused with the pre-fusion detail layer of the preliminary multimodal medical image fusion result to obtain the fused detail layer specifically includes: The gradient energy clarity operator is used to measure pixel activity in the pre-fusion detail layer of the multimodal medical image to be fused and the pre-fusion detail layer of the preliminary multimodal medical image fusion result, so as to obtain the saliency map of the multimodal medical image to be fused and the saliency map of the preliminary multimodal medical image fusion result. The pixels of the saliency map of the preliminary multimodal medical image fusion result and the saliency map of the multimodal medical image to be fused are compared and consistent with each other in turn to obtain the decision map of the detail layer of the multimodal medical image to be fused. Based on the preset detail layer fusion rules, the preliminary multimodal medical image fusion results, the decision map of the detail layer of the third multimodal medical image and the multimodal medical image to be fused are fused to obtain the fused detail layer.

7. The trimodal medical image fusion method based on a dual-model approach according to claim 6, characterized in that, The expression for the preset detail layer fusion rule is as follows: In the above formula, This represents the merged detail layer. This represents the improved decision graph obtained after consistency verification. Indicates through the and The detail layer of the preliminary multimodal medical image fusion result obtained by pixel comparison is compared with the initial decision map of the detail layer of the multimodal medical image. and These are respectively represented by the designed gradient energy clear operator pairs. and The saliency map obtained from the detection, Indicates the size of the consistency verification window.

8. A trimodal medical image fusion system based on a dual-model approach, characterized in that, Includes the following modules: The first decomposition module is used to perform three-level decomposition processing on the multimodal medical image through the bitonicX filter to obtain the decomposed multimodal medical image to be fused. The step of performing a three-layer decomposition process on the multimodal medical image using a bitonicX filter to obtain the decomposed multimodal medical image to be fused specifically includes: acquiring multimodal medical images, which include a first multimodal medical image, a second multimodal medical image, and a third multimodal medical image; filtering the first and second multimodal medical images using a bitonicX filter to obtain corresponding energy layers and detail layers to be fused; performing a secondary filtering process on the energy layer to be fused using a bitonicX filter to obtain a secondary energy layer to be fused; subtracting the energy layer to be fused from the secondary energy layer to be fused to obtain a texture layer to be fused; and integrating the secondary energy layer, detail layer, and texture layer to be fused to construct the decomposed multimodal medical image to be fused. The first fusion module, based on preset fusion rules, sequentially performs fusion and addition processing on the decomposed multimodal medical images to be fused, to obtain preliminary multimodal medical image fusion results; The step of performing sequential fusion and addition processing on the decomposed multimodal medical images to be fused based on preset fusion rules to obtain preliminary multimodal medical image fusion results specifically includes: considering the gradient information of the multimodal medical images, performing fusion processing on the detail layers to be fused in the decomposed multimodal medical images by combining multi-scale morphological gradient operators and parameter adaptive coupled neural networks to obtain fused detail layers; performing fusion processing on the texture layers to be fused in the decomposed multimodal medical images to be fused based on the fusion strategy of coupled neural P systems to obtain fused texture layers; performing fusion processing on the secondary energy layers to be fused in the decomposed multimodal medical images to be fused based on the fusion rule of "taking the larger absolute value" to obtain fused energy layers; and adding the fused detail layers, fused texture layers, and fused energy layers to obtain preliminary multimodal medical image fusion results. The second decomposition module decomposes the preliminary multimodal medical image fusion result and the multimodal medical image based on the bitonicX filter to obtain the corresponding multimodal medical image to be fused. The second fusion module is used to sequentially fuse and add the corresponding multimodal medical images to be fused to obtain the final multimodal medical image fusion result.

Images