Anatomical medical image fusion method based on texture perception and pixel intensity correlation

Abstract

The application discloses an anatomic medical image fusion method based on texture perception and pixel intensity correlation, which comprises the following steps: acquiring MRI images and CT images of brain diseases and performing registration; constructing an image texture information energy perception function; decomposing the registered MRI images and CT images based on the image texture information energy perception function to obtain respective corresponding basic layers and structure layers; establishing an image fusion rule; processing the obtained basic layers and structure layers based on the image fusion rule to obtain a fusion basic layer image and a fusion structure layer image; and performing image reconstruction based on the fusion basic layer image and the fusion structure layer image to obtain a fusion image, thereby realizing anatomic medical image fusion. The application effectively solves the problem of losing image contrast information in the fusion process and alleviates the image information loss phenomenon.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, and in particular relates to an anatomical medical image fusion method based on texture perception and pixel intensity correlation. Background Technology

[0002] Medical images are inextricably linked to human health examinations. Utilizing specific imaging principles, they reflect the body's health status from different angles, making it difficult for doctors to obtain sufficient descriptive information from single-modality medical images. Medical image fusion leverages the redundancy and complementarity of multi-source information to integrate multiple medical images into a single image, providing a more comprehensive and accurate description of disease attributes. This assists doctors in medical diagnosis and treatment planning.

[0003] Anatomical images primarily include magnetic resonance imaging (MRI) and computed tomography (CT) images. Fusion MRI-CT images can describe the extent of damage to surrounding tissues caused by the lesion, aiding in skull base tumor surgery.

[0004] Currently, medical image fusion methods are mainly divided into traditional methods and deep learning methods. Among traditional methods, those based on multi-scale image decomposition, such as pyramid methods, filtering methods, and frequency domain transform methods, have achieved relatively good fusion results. These methods decompose images into different scales to preserve image features of different modalities. However, the number of decomposition layers is difficult to fix, generally requiring multiple trials to obtain a satisfactory result, and lacks universality. When changing the decomposition tool or the decomposition object, the number of decomposition layers still needs to be determined again. Furthermore, the fusion performance of the above methods largely depends on the inherent properties of the decomposition function, and cannot be adaptively adjusted according to the content of the medical image, further limiting the performance of the fusion methods. Unlike the approach of manually designing image feature extraction algorithms, deep learning-based methods learn from a large amount of labeled data to acquire the ability to adaptively represent image features, resulting in more stable performance of the image fusion model. However, given the special nature of medical images, the publicly available and labeled medical image datasets are relatively small, making it difficult to support complex network models to fully learn the features of medical images, posing a challenge to overcoming the current research bottleneck. Summary of the Invention

[0005] The purpose of this invention is to provide an anatomical medical image fusion method based on texture perception and pixel intensity correlation to solve the problems existing in the prior art.

[0006] To achieve the above objectives, this invention provides an anatomical medical image fusion method based on texture perception and pixel intensity correlation, comprising:

[0007] Acquire MRI and CT images of brain diseases and perform registration;

[0008] An image texture information energy perception function is constructed, and the registered MRI and CT images are decomposed based on the image texture information energy perception function to obtain their respective basic and structural layers.

[0009] An image fusion rule is established, and the base layer and structural layer obtained by decomposition are processed according to the image fusion rule to obtain a fused base layer image and a fused structural layer image.

[0010] Image reconstruction is performed based on the fused base layer image and the fused structural layer image to obtain a fused image, thereby realizing the fusion of anatomical medical images.

[0011] Optionally, the registration process includes:

[0012] Using either the MRI image or the CT image as a reference image and the other as a floating image, the mutual information between the two images is calculated.

[0013] Given a spatial transformation, the pixels in the floating image are mapped to the coordinate system of the base image, and grayscale interpolation is performed on the pixels with non-integer coordinates after the mapping.

[0014] Establish the relationship between the mutual information and the parameters of the spatial transformation;

[0015] By optimizing the algorithm, based on the relationship, the parameter values ​​of the spatial transformation are changed to obtain the maximum value of mutual information and the parameter values ​​of the spatial transformation corresponding to the maximum value, thus completing image registration.

[0016] Optionally, the process of constructing the image texture information energy perception function includes: obtaining the global energy change of the image by analyzing the relationship between the target pixel in the output image and the neighboring pixels at the corresponding position in the input image; and analyzing the local energy change of the image by measuring the relationship between the target pixel and its neighboring pixels in the output image. The image texture information energy perception function is then constructed based on the global energy change and the local energy change.

[0017] Optionally, the process of obtaining the base layer of MRI and CT images includes: sequentially inputting the MRI and CT images into the image texture information energy sensing function, solving for the minimum value of the function, and using the output image when the function reaches the minimum value as the base layer of the MRI and CT images.

[0018] Optionally, the structural layers corresponding to the MRI and CT images can be obtained by performing linear subtraction operations on the base layers corresponding to the images.

[0019] Optionally, the fusion rules include base layer image fusion rules and structural layer image fusion rules.

[0020] Optionally, a fused base layer image is obtained based on the base layer image fusion rules, the process of which includes:

[0021] Based on the energy similarity between the base layer of the image and its original image, an objective function is designed to calculate the pixel intensity maps of the MRI base layer image and the CT base layer image respectively. By comparing the intensity values ​​of the MRI image and the CT image pixel by pixel, the fused base layer image is obtained.

[0022] Optionally, the fused structural layer image is obtained based on the structural layer image fusion rules, and the process includes:

[0023] Based on the pixel intensity of the base layer, the structural intensity of the MRI and CT structural layer images is calculated, and the fused structural layer image is obtained by comparing the magnitude of the absolute values ​​of the structural intensity.

[0024] Optionally, the objective function is as follows:

[0025]

[0026]

[0027] Where ⊙ represents the Hadamard product, α and β are two nonnegative parameters, and B * Image I * The base layer, E * B * The corresponding pixel intensity map, where m and n represent image I * The length and width; W = [W h W v ]and The values ​​represent binary information, where h represents the horizontal information of the image and v represents the vertical information of the image.

[0028] Optionally, the pixel intensity values ​​at corresponding positions of the fused base layer image and the fused structure layer image are added together to obtain the fused image.

[0029] The technical effects of this invention are as follows:

[0030] This invention provides an anatomical medical image fusion method based on texture perception and pixel intensity correlation. The image decomposition process makes full use of the image content information and pixel intensity distribution characteristics, and obtains the base layer and structural layer of the corresponding image by solving the minimum value of the texture energy perception function.

[0031] To address the issue of lost contrast features in fused images, a fusion decision graph for the base layer and structural layer is constructed using the theory that pixel intensity across subbands is correlated. This maintains the consistency of intensity information, effectively alleviates the loss of image information in existing medical image fusion algorithms, and improves the visual perception quality of the fusion results. Attached Figure Description

[0032] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0033] Figure 1 This is a flowchart of the anatomical medical image fusion method based on image texture perception and pixel intensity correlation in an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of the fusion effect of MRI-CT test images in an embodiment of the present invention. (a) is the MRI image before fusion, (b) is the CT image before fusion, and (c) is the fusion result. Detailed Implementation

[0035] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0036] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0037] Example 1

[0038] like Figure 1-2 As shown, this embodiment provides an anatomical medical image fusion method based on texture awareness and pixel intensity correlation, including:

[0039] S1. Acquire MRI and CT images of different brain diseases and perform registration;

[0040] S2. Input the MRI and CT images into the image texture information energy perception function, solve for the minimum value of the objective function, and use it as the base layer B of the source image I* (*=1,2). * The structural layer S of the corresponding image is obtained by linear subtraction. * ;

[0041] S3. Merge B1 and B2 using the designed base layer fusion rules to obtain the fused base layer B.F By fusing S1 and S2 according to the designed structural layer fusion rules, the fused structural layer S is obtained. F ;

[0042] S4. Through image reconstruction, obtain B F and S F Linear addition yields a fused MRI-CT image.

[0043] In some embodiments, the input multimodal image pairs in step S1 have not been preprocessed, so image registration is required.

[0044] In this embodiment, the image texture information energy perception function in step S2 can adaptively smooth the original image according to its content. For an input image I... in and a guide image I g Its output smooth image I u The minimum value can be obtained by solving the following objective function:

[0045]

[0046] The first item is the data item, which measures the input image I. in With output image I u The global energy change between them. The second term is the smoothing term, which calculates the output image I. u Internal local texture variations. i and j represent pixel indices in the image, and pixel j is located in the neighborhood N of pixel i. i Within this invention, the neighborhood size is set to 3×3. (I) u,i -I in,j ) represents a smoothed image I u Pixel i and input image I in The intensity difference between pixels j. (I) u,i -I u,j The smoothed image I was calculated. u The intensity difference between pixel i and its neighboring pixel j. The parameter λ controls the intensity of image smoothing; in this invention, λ = 0.5. It is the Gaussian kernel function, defined as For guiding weights, it is defined as follows: Among them, the guiding image I g The sensitive edge in the equation is determined by the parameter k, which is 0.5 in this invention. ò is a very small constant term to avoid division by zero; here, ò is set to 10. -3 Furthermore, in this invention, the original image is used as the guiding image, therefore I in =I g Truncating the Huber penalty function hT (·) is defined as follows:

[0047]

[0048] Where a and b are constants, and h(·) is the Huber penalty function, defined as follows:

[0049]

[0050] By setting different parameters, h T (·) can exhibit great flexibility, producing different punishment behaviors. Assume the input intensity value is in [0, I... m If the amplitude of any edge falls within [0, I], then the amplitude of any edge will fall within [0, I]. m [Inside. First, set a = ò, then if b > I...] m h T (·) is actually the same as h(·), because the second condition in equation (2) can never be satisfied. Since a is a sufficiently small value, h T (·) in this case will approach the L1 norm, thus it is a penalty function that preserves the edges without making them sharper. Conversely, when b < I m At that time, h T The truncation in parentheses (·) will be activated. This may result in weak edges being penalized while strong edges are not, thus sharpening the strong edges. In short, parameter b can be seen as determining h. T (·) A switch to enable or disable edge sharpening. Similarly, by setting a = b > I m and a=b<I m h T (·) allows for easy switching between the L2 norm and the truncated L2 norm. In this invention, to better perceive the texture and edge information of the image and achieve image smoothing, the parameter is set to a = 10. -3 b = 1.

[0051] When input image I in When I* (*=1,2), the smoothed result I can be obtained by solving for the minimum value. u and use it as the corresponding image I * Base layer image B * Then, by linear subtraction of S... * =I * -B * Image I was obtained * Structural layer S * .

[0052] In this embodiment, the base layer B obtained in step S2 * It is the original image I* The maximum approximation. To fully preserve the energy and contrast information of the source image, the following function is constructed to obtain B. * Corresponding pixel intensity map E * :

[0053]

[0054] Where ||·||0 represents the L0 norm, and the parameter θ is a nonnegative coefficient used to balance the fidelity term and the sparsity term. Since directly solving equation (4) is difficult, an approximate solution is used, that is, equation (4) is rewritten as:

[0055]

[0056] Where ⊙ represents the Hadamard product, α and β are two nonnegative parameters, and B * Image I * The base layer, E * B * The corresponding pixel intensity map, where m and n represent image I * The length and width. W = [W h W v ]and Representing binary information, h represents the horizontal information of the image, v represents the vertical information of the image, and when W... i =0, When W is true, it means that the element at the corresponding position is located in the structure or edge of the image; otherwise, W is false. i =1, constraint This represents advanced edge properties, such as long links and consistent neighborhood direction. h and v are the indices of the W matrix, where W... h and W v The dimensions of the two matrices are m*n. Merging the two matrices gives W, so its dimension is 2m*n, with W in the horizontal direction. h Obtained by traversing the image using [1-1], vertical direction W v Use [1 -1] Τ B is obtained by traversing the image. B is obtained through an alternating solution method. * Corresponding pixel intensity map E * Then, by comparing pixels one by one, the fusion decision map of the base layer is obtained:

[0057]

[0058] Where (a, b) represent pixel coordinates. The base layer image B for fusion is obtained by weighted Hadamard product. F The calculation method is as follows:

[0059] B F=B1⊙D1+B2⊙(1-D1)

[0060] In the embodiment, the structural layer S obtained in step S2 * It contains the image's texture information. Given the high similarity between the base layer information and texture information of an image, based on image I... * Pixel intensity map of the base layer E * Calculate the implicit structural strength:

[0061] T * =B * -E *

[0062] Therefore, the fusion result of the structural layers is obtained by comparing the structural strength:

[0063]

[0064] In this embodiment, the fused base layer image and the fused structural layer image obtained in step S3 are linearly added together to obtain the fused MRI-CT image F, and the calculation method is as follows:

[0065] F = B F +S F

[0066] The fusion results of MRI-CT test images, such as Figure 2 As shown in (c), the MRI image before fusion is as follows: Figure 2 (a) Pre-fusion CT image as shown Figure 2 (b)

[0067] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An anatomical medical image fusion method based on texture perception and pixel intensity correlation, characterized in that, The method comprises the following steps: obtaining MRI images and CT images of the brain disease and performing registration; constructing an image texture information energy perception function, decomposing the registered MRI images and CT images based on the image texture information energy perception function to obtain respective corresponding basic layers and structure layers; establishing an image fusion rule, processing the basic layers and structure layers obtained by decomposition based on the image fusion rule to obtain a fused basic layer image and a fused structure layer image; performing image reconstruction based on the fused basic layer image and the fused structure layer image to obtain a fused image, thereby realizing anatomical medical image fusion; the fusion rule comprises a basic layer image fusion rule and a structure layer image fusion rule; the fused basic layer image is obtained based on the basic layer image fusion rule, and the process comprises: based on the energy similarity of the basic layer of the image and the original image thereof, a target function is designed to calculate the pixel intensity map of the MRI basic layer image and the CT basic layer image, the intensity values of the MRI image and the CT image are compared pixel by pixel, and the fused basic layer image is obtained; the fused structure layer image is obtained based on the structure layer image fusion rule, and the process comprises: based on the pixel intensity of the basic layer, the structure intensity of the MRI structure layer image and the CT structure layer image is calculated, and the fused structure layer image is obtained by comparing the absolute values of the structure intensity; the target function is as follows: wherein, denotes a Hadamard product, and are two non-negative parameters, denotes a base layer of an image , denotes a corresponding pixel intensity map, and denotes a height and a width of an image ; and denotes binary information, denotes horizontal direction information of an image, denotes vertical direction information of an image.

2. The anatomical medical image fusion method based on texture perception and pixel intensity correlation according to claim 1, wherein the registration process comprises: taking any one of the MRI image and the CT image as a reference image and the other as a floating image, and calculating the mutual information of the two images; given a spatial transformation, mapping the pixel points in the floating image to the basic image coordinate system, and performing gray value interpolation on the pixel points in the non-integer coordinates after the mapping is completed; establishing the relationship between the mutual information and the parameters of the spatial transformation; by an optimization algorithm, changing the parameter value of the spatial transformation based on the relationship, obtaining the maximum value of the mutual information and the parameter value of the spatial transformation corresponding to the maximum value, and completing the image registration.

3. The anatomical medical image fusion method based on texture perception and pixel intensity correlation according to claim 1, wherein the process of constructing the image texture information energy perception function comprises: obtaining the global energy change of the image by analyzing the relationship between the target pixel in the output image and the neighborhood pixels in the corresponding position of the input image; analyzing the local energy change of the image by measuring the relationship between the target pixel in the output image and its neighborhood pixels, and constructing the image texture information energy perception function based on the global energy change and the local energy change.

4. The anatomical medical image fusion method based on texture perception and pixel intensity correlation according to claim 1, wherein the process of obtaining the basic layers of the MRI image and the CT image comprises: sequentially inputting the MRI image and the CT image into the image texture information energy perception function, solving the minimum value of the function, and taking the output image when the function reaches the minimum value as the basic layer of the MRI image and the CT image.

5. The anatomical medical image fusion method based on texture perception and pixel intensity correlation according to claim 1, characterized in that, The structure layer corresponding to the MRI image and the CT image is obtained by performing linear subtraction operation on the basis layer corresponding to the MRI image and the CT image respectively.

6. The anatomical medical image fusion method based on texture perception and pixel intensity correlation according to claim 1, characterized in that, The pixel intensity values of the corresponding positions of the fusion basis layer image and the fusion structure layer image are added to obtain the fusion image.

Images