U-net network multi-parameter MRI image fusion method and device
Multi-parameter fusion of MRI images through U-net network solves the problem of cumbersome and false detection of images in diagnosis, and achieves clearer and more reliable diagnosis of prostate lesions.
Patent Information
- Application Number
- CN202010997120.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-21
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-09-21
AI Technical Summary
In the prior art, doctors need to switch T2W and ADC images when diagnosing prostate lesions, resulting in cumbersome manual switching and easy to miss and miss detection, making it difficult to reliably distinguish prostate cancer from other lesions.
The MRI images of the prostate were fusion by multi-parameters. By combining ADC and T2W images, weighted images were used for decomposition and fusion, local energy matching was calculated to determine the fusion image, avoiding artifacts and energy losses.
It improves doctors' work efficiency, reduces false detection rates, enhances clarity and contrast of prostate areas, and improves overall image visual perception.
Smart Images

Figure CN114255195B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of medical image fusion, and in particular relates to a method and a device for fusing multi-parameter MRI images of a U-net network. Background Art
[0002] Hospital images can effectively help doctors improve their work efficiency, such as the prostate mpMRI prostate imaging reporting and data system (PI-RADS) scoring system established by the European Society of Urogenital Radiology (ESUR). The RAD system helps doctors make clinical decisions quickly, efficiently and reliably, and can also improve the quantitative assessment of the disease. MRI is the best imaging technology for prostate visualization, with excellent resolution and contrast between the prostate and surrounding structures and anatomical regions of the gland. With the development of multiparametric MRI (mpMRI), MRI combines anatomical T2W with functional and physiological assessments, including diffusion-weighted imaging (DWI) and its derived apparent diffusion coefficient (ADC) map.
[0003] In the prior art, T2W images on cross-sections are considered to be the best images for describing the prostate's band-like anatomical structure. However, in many cases, when there are prostate lesions, such as inflammation or cancer, the prostate area often appears black in T2W, resulting in doctors being unable to reliably distinguish PCa from prostatic intraepithelial neoplasia, hemorrhage, changes after radiotherapy, or prostatitis simply by referring to T2W images.
[0004] In the ADC graph, low ADC values indicate restricted diffusion, while higher ADC values come from tissues that diffuse relatively freely. Because the cellularity of tissues and the integrity of cell membranes are inversely proportional to the diffusion of molecules in tissues, the measurement of water diffusion provides information about tissue structure in benign and malignant tissues. Therefore, doctors generally ask patients to refer to T2W images and ask them to do ADC examinations. They also check T2W images and ADC images at the same time and give preliminary examination results based on their own experience.
[0005] Since doctors need to switch between T2W images and ADC images, manual image switching is cumbersome. In addition, when observing images with multiple parameters at the same time, they need to compare the corresponding areas of the T2W images and ADC images one by one, which can easily lead to missed detections and false detections. Summary of the invention
[0006] In order to solve the above problems existing in the prior art, the present invention provides a method and device for fusing multi-parameter MRI images of a U-net network. The technical problem to be solved by the present invention is achieved by the following technical solutions:
[0007] In a first aspect, a method for fusing multi-parameter MRI images of a U-net network provided by an embodiment of the present invention includes:
[0008] Acquire a pair of MRI images of the prostate, wherein the pair of images includes an ADC image and a T2W image;
[0009] Merging the ADC image and the T2W image to obtain a merged image;
[0010] Inputting the merged image into a trained U-net network to obtain a weighted image output by the U-net network with the same size as the merged image, wherein the pixel values in the weighted image represent the weight values of the weighted image in the T2W image or the ADC image;
[0011] Decomposing the ADC image, the T2W image and the weighted image respectively;
[0012] Calculating a first local energy of the decomposed ADC image and a second local energy of the decomposed T2W image;
[0013] When the matching degree between the first local energy and the second local energy reaches an energy threshold, the decomposed ADC image and the decomposed T2W image are fused based on the decomposed weighted image to obtain a fused image.
[0014] Optionally, the step of respectively decomposing the ADC image, the T2W image, and the weighted image includes:
[0015] Performing Laplace pyramid decomposition on the ADC image and the T2W image respectively;
[0016] Perform Gaussian pyramid decomposition on the weighted image.
[0017] Optionally, the step of calculating the first local energy of the decomposed ADC image and the second local energy of the decomposed T2W image includes:
[0018] For each pixel point of the decomposed ADC image, calculate the square sum of pixel values in a preset window centered on the pixel point to obtain a first local energy of the decomposed ADC image;
[0019] For each pixel point in the decomposed T2W image, the square sum of pixel values in a preset window centered on the pixel point is calculated to obtain the second local energy of the decomposed T2W image.
[0020] Optionally, the step of calculating, for each pixel point of the decomposed ADC image, the square sum of pixel values in a preset window centered on the pixel point to obtain the first local energy of the decomposed ADC image includes:
[0021] For each pixel point of the decomposed ADC image, using a first local energy calculation formula, calculate the square sum of pixel values in a preset window centered on the pixel point to obtain a first local energy of the decomposed ADC image;
[0022] The step of calculating, for each pixel point in the decomposed T2W image, the square sum of pixel values in a preset window centered on the pixel point to obtain the second local energy of the decomposed T2W image comprises:
[0023] For each pixel point in the decomposed T2W image, using a second local energy calculation formula, calculate the square sum of pixel values in a preset window centered on the pixel point to obtain a second local energy of the decomposed T2W image;
[0024] The local energy calculation formula is:
[0025]
[0026] E A represents the first local energy, E B represents the second local energy, i represents the horizontal coordinate of the pixel point, j represents the horizontal coordinate and vertical coordinate of the pixel point, m represents the window length centered on the pixel point, n represents the window width centered on the pixel point, L{A}(i+m,j+n) represents the pixel value of the pixel point in the window with a size of m×n and the pixel point (i, j) of the decomposed ADC image as the center, L{B}(i+m,j+n) represents the pixel value of the pixel point in the window with a size of m×n and the pixel point (i, j) of the decomposed T2w image as the center, N represents the length of the image, and M represents the width of the image.
[0027] Optionally, before the step of when the matching degree between the first local energy and the second local energy reaches an energy threshold, the multi-parameter MRI image fusion method based on the U-net network provided by the first aspect of the embodiment of the present invention further includes:
[0028] calculating a matching degree between the first local energy and the second local energy;
[0029] It is determined whether the matching degree reaches a preset energy threshold.
[0030] Optionally, the step of calculating the matching degree between the first local energy and the second local energy includes:
[0031] Using an energy matching degree calculation formula, calculating the matching degree of the first local energy and the second local energy;
[0032] The energy matching calculation formula is:
[0033]
[0034] M AB (i, j) represents the matching degree between the first local energy and the second local energy, E A represents the first local energy, E B represents the second local energy, i represents the horizontal coordinate of the pixel point, j represents the horizontal coordinate and vertical coordinate of the pixel point, and m represents the window length centered on the pixel point.
[0035] n represents the window width centered on the pixel point, L{A}(i+m,j+n) represents the pixel value of the pixel point in the window with a size of m×n and a pixel point (i, j) as the center of the decomposed ADC image, L{B}(i+m,j+n) represents the pixel value of the pixel point in the window with a size of m×n and a pixel point (i, j) as the center of the decomposed T2w image, N represents the length of the image, and M represents the width of the image.
[0036] Optionally, after the step of determining whether the matching degree reaches a preset energy threshold, the fusion method further includes:
[0037] When the matching degree between the first local energy and the second local energy does not reach an energy threshold, comparing the information amount of the pixel points in the decomposed ADC image and the information amount of the pixel points in the decomposed T2W image to determine the pixel points with high information amount;
[0038] The pixel points with high information content are determined as fused pixel points to obtain a fused image.
[0039] Optionally, the step of fusing the decomposed ADC image and the decomposed T2W image based on the decomposed weighted image to obtain a fused image includes:
[0040] For a pixel point, multiply a pixel value of the pixel point in the weighted image by a first local energy of the pixel point in the decomposed ADC image to obtain a first multiplication result;
[0041] For a pixel point, multiply the difference between 1 and the pixel value of the pixel point in the weighted image by the second local energy of the pixel point in the decomposed T2W image to obtain a second multiplication result;
[0042] The sum of the first multiplication result and the second multiplication result is determined as the pixel value of the pixel point in the fused image.
[0043] Optionally, after the step of fusing the decomposed ADC image and the decomposed T2W image based on the decomposed weighted image to obtain a fused image when the matching degree between the first local energy and the second local energy reaches an energy threshold, the multi-parameter MRI image fusion method based on the U-net network provided by the first aspect of the embodiment of the present invention further includes:
[0044] The fused image is reconstructed using an inverse Laplace pyramid transform to obtain a reconstructed medical image.
[0045] In a second aspect, an embodiment of the present invention provides a multi-parameter MRI image fusion device of a U-net network, comprising:
[0046] An acquisition module, used for acquiring an MRI image pair of the prostate, wherein the image pair includes an ADC image and a T2W image;
[0047] A merging module, used for merging the ADC image and the T2W image to obtain a merged image;
[0048] An output module, used for inputting the merged image into a trained U-net network to obtain a weighted image output by the U-net network with the same size as the merged image, wherein the pixel value in the weighted image represents the weight value of the weighted image in the T2W image or the ADC image;
[0049] A decomposition module, used for decomposing the ADC image, the T2W image and the weighted image respectively;
[0050] A calculation module, used for calculating a first local energy of the decomposed ADC image and a second local energy of the decomposed T2W image;
[0051] A fusion module is used to fuse the decomposed ADC image and the decomposed T2W image based on the decomposed weighted image to obtain a fused image when the matching degree between the first local energy and the second local energy reaches an energy threshold.
[0052] An embodiment of the present invention provides a method for fusing multi-parameter MRI images of a U-net network. The method inputs a pair of MRI images of prostate cancer into a U-net network, and the U-net network outputs a weighted image generated based on pixel information. The pixel value in the weighted image represents the weight value of the weighted image in a T2W image or an ADC image. The T2W image and the ADC image are then fused using the weighted image, thereby avoiding the generation of artifacts or energy loss during the fusion process. The fused image is provided to a doctor, thereby improving the doctor's work efficiency and reducing the false detection rate. At the same time, the clarity and contrast of the fused image in the prostate region are improved, thereby improving the visual perception of the fused image in the overall image.
[0053] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 It is a flow chart of a multi-parameter MRI image fusion method of a U-net network provided by an embodiment of the present invention;
[0055] Figure 2 It is a schematic diagram of the structure of a U-net network provided by an embodiment of the present invention;
[0056] Figure 3 is a schematic diagram of the process from S12 to S16 provided in an embodiment of the present invention;
[0057] Figure 4 It is a structural diagram of a multi-parameter MRI image fusion device of a U-net network provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0058] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.
[0059] Embodiment 1
[0060] like Figure 1 As shown, an embodiment of the present invention provides a method for fusing multi-parameter MRI images using a U-net network, comprising:
[0061] S11, obtaining a pair of MRI images of the prostate;
[0062] The image pair includes an ADC image and a T2W image.
[0063] It can be understood that the MRI image pair is composed of an ADC image of the prostate and a T2W image of the prostate of the same person, so the image after image fusion is the image of the prostate of the person.
[0064] S12, merging the ADC image and the T2W image to obtain a merged image;
[0065] It can be understood that after the ADC image and the T2W image are merged, the ADC image and the T2W image are merged into a dual-channel image, which provides an input image for subsequent entry into the U-net.
[0066] S13, inputting the merged image into a trained U-net network to obtain a weighted image output by the U-net network with the same size as the merged image;
[0067] The pixel value in the weighted image represents the weight value of the weighted image in the T2W image or the ADC image. The closer the weight value is to 1, the more pixel intensity the pixel value represents in the ADC image, and the closer the weight value is to 0, the more pixel intensity the pixel value represents in the T2W image.
[0068] Among them, the trained U-net network is trained based on the large-scale training samples synthesized on the VOC 2012 natural image dataset. The training process iteratively trains the preset U-net network based on the relationship between image feature information and training objectives. This process is consistent with the existing training network model process and will not be repeated here.
[0069] It can be understood that the U-net network will automatically extract image information from the input image and output a weighted image of the same size as the input image. The U-net network can be divided into two parts: contraction and expansion. Figure 2 As shown in the figure, after the convolution and max pooling layers, the image is gradually reduced, and each convblock contains two sets of convolution and BatchNormal operations. The main part of the expansion is composed of deconvolution blocks and convblocks, and each expansion layer is followed by a cascade operation to cascade with the corresponding part of the contraction layer.
[0070] S14, decomposing the ADC image, the T2W image and the weighted image respectively;
[0071] The maximum number of decomposition layers during decomposition is determined according to the size of the medical image. In actual operation, it is assumed that the image size is 256 and the maximum number of decomposition layers is 8.
[0072] Combination Figure 2 , Figure 2 Step 2 in FIG. 1 is a schematic diagram of the process of decomposing the ADC image, the T2W image and the weighted image respectively.
[0073] S15, calculating a first local energy of the decomposed ADC image and a second local energy of the decomposed T2W image;
[0074] S16, when the matching degree between the first local energy and the second local energy reaches an energy threshold, based on the decomposed weighted image, the decomposed ADC image and the decomposed T2W image are fused to obtain a fused image.
[0075] Combination Figure 2 , Figure 2 Step 3 in FIG. 1 is a schematic diagram of a process of fusing the decomposed ADC image and the decomposed T2W image based on the decomposed weighted image to obtain a fused image.
[0076] An embodiment of the present invention provides a U-net network multi-parameter MRI image fusion method. By inputting a pair of MRI images of prostate cancer into a U-net network, the U-net network outputs a weighted image generated based on pixel information. The pixel value in the weighted image represents the weight value of the weighted image in the T2W image or the ADC image. The T2W image and the ADC image are then fused through the weighted image, thereby avoiding the generation of artifacts or energy loss during the fusion process. The fused image is provided to a doctor, which can improve the doctor's work efficiency and reduce the false detection rate. At the same time, the clarity and contrast of the fused image in the prostate area are improved, and the visual perception of the fused image in the overall image is improved.
[0077] Embodiment 2
[0078] As an optional embodiment of the present invention, the step S14 includes:
[0079] Step 1: Perform Laplace pyramid decomposition on the ADC image and T2W image respectively;
[0080] Step 2: Perform Gaussian pyramid decomposition on the weighted image.
[0081] Embodiment 3
[0082] As an optional embodiment of the present invention, the step S15 includes:
[0083] Step 1: for each pixel point of the decomposed ADC image, the square sum of the pixel values in a preset window centered on the pixel point is calculated to obtain the first local energy of the decomposed ADC image;
[0084] Step 2: For each pixel point in the decomposed T2W image, the square sum of the pixel values in a preset window centered on the pixel point is calculated to obtain the second local energy of the decomposed T2W image.
[0085] Embodiment 4
[0086] As an optional embodiment of the present invention, for each pixel point of the decomposed ADC image, the step of calculating the square sum of pixel values in a preset window centered on the pixel point to obtain the first local energy of the decomposed ADC image includes:
[0087] For each pixel of the decomposed ADC image, using the first local energy calculation formula, calculate the square sum of pixel values in a preset window centered on the pixel to obtain the first local energy of the decomposed ADC image;
[0088] For each pixel point in the decomposed T2W image, the step of calculating the square sum of pixel values in a preset window centered on the pixel point to obtain the second local energy of the decomposed T2W image includes:
[0089] For each pixel point in the decomposed T2W image, the second local energy calculation formula is used to calculate the square sum of the pixel values in a preset window centered on the pixel point to obtain the second local energy of the decomposed T2W image;
[0090] The local energy calculation formula is:
[0091]
[0092] E A represents the first local energy, E B represents the second local energy, i represents the horizontal coordinate of the pixel point, j represents the horizontal coordinate and vertical coordinate of the pixel point, m represents the window length centered on the pixel point, n represents the window width centered on the pixel point, L{A}(i+m,j+n) represents the pixel value of the pixel point in the window with a size of m×n and the pixel point (i, j) of the decomposed ADC image as the center, L{B}(i+m,j+n) represents the pixel value of the pixel point in the window with a size of m×n and the pixel point (i, j) of the decomposed T2w image as the center, N represents the length of the image, and M represents the width of the image.
[0093] Embodiment 5
[0094] As an optional embodiment of the present invention, before the above step S16, the multi-parameter MRI image fusion method and device of the U-net network provided in the embodiment of the present invention further includes:
[0095] Step 1: Calculate the matching degree between the first local energy and the second local energy;
[0096] Step 2: Determine whether the matching degree reaches a preset energy threshold.
[0097] Embodiment 6
[0098] As an optional embodiment of the present invention, the step S15 includes:
[0099] Using an energy matching degree calculation formula, calculating the matching degree of the first local energy and the second local energy;
[0100] The energy matching calculation formula is:
[0101]
[0102] M AB (i, j) represents the matching degree between the first local energy and the second local energy, E A represents the first local energy, E B represents the second local energy, i represents the horizontal coordinate of the pixel point, j represents the horizontal coordinate and vertical coordinate of the pixel point, m represents the window length centered on the pixel point, n represents the window width centered on the pixel point, L{A}(i+m,j+n) represents the pixel value of the pixel point in the window with a size of m×n and the pixel point (i, j) of the decomposed ADC image as the center, L{B}(i+m,j+n) represents the pixel value of the pixel point in the window with a size of m×n and the pixel point (i, j) of the decomposed T2w image as the center, N represents the length of the image, and M represents the width of the image.
[0103] Embodiment 7
[0104] As an optional embodiment of the present invention, after the step of determining whether the matching degree reaches a preset energy threshold, the multi-parameter MRI image fusion method and device of the U-net network provided in the embodiment of the present invention further includes:
[0105] When the matching degree between the first local energy and the second local energy does not reach the energy threshold, comparing the information amount of the pixel point in the decomposed ADC image and the information amount of the pixel point in the decomposed T2W image to determine the pixel point with high information amount;
[0106] The pixels with high information content are determined as the fused pixels to obtain a fused image.
[0107] If the matching degree between the first local energy and the second local energy does not reach the energy threshold, it means that the difference between the ADC image and the T2W image is relatively large, and the pixels with higher information content in the two images are selected for fusion, as shown in the following formula:
[0108]
[0109] Wherein, {F}(i,j) represents the pixel value of the fused image pixel (i,j).
[0110] Embodiment 8
[0111] As an optional embodiment of the present invention, based on the decomposed weighted image, the decomposed ADC image and the decomposed T2W image are fused to obtain a fused image, the step includes:
[0112] For a pixel point, multiply a pixel value of the pixel point in the weighted image by a first local energy of the pixel point in the decomposed ADC image to obtain a first multiplication result;
[0113] For a pixel point, multiply the difference between 1 and the pixel value of the pixel point in the weighted image by the second local energy of the pixel point in the decomposed T2W image to obtain a second multiplication result;
[0114] The sum of the first multiplication result and the second multiplication result is determined as the pixel value of the pixel point in the fused image.
[0115] Among them, if the matching degree of the first local energy and the second local energy reaches the energy threshold, it means that the image difference is relatively small, and the two images are weighted fused according to the weighted image. The weighted fusion formula is as follows:
[0116] L{F}(i,j)=G{W}(i,j)L{A}(i,j)+(1-G{W}(i,j))L{B}(i,j)
[0117] Wherein, G{W}(i,j) represents the pixel value of the pixel point (i,j) in the weighted image.
[0118] Embodiment 9
[0119] As an optional embodiment of the present invention, after the above step S16, the multi-parameter MRI image fusion method and device of the U-net network provided in the embodiment of the present invention further includes:
[0120] The fused image is reconstructed using the inverse Laplace pyramid transform to obtain a reconstructed medical image.
[0121] The fusion method provided in the embodiment of the present invention is tested below to verify the effect of the fusion method provided in the embodiment of the present invention.
[0122] First, 30 pairs of MR images (axial T2w and ADC images) of 10 patients with prostate cancer were selected as experimental data (3 pairs for each patient), and the image pairs were fused using the fusion method provided by the present invention and the image fusion algorithm of the prior art, and the fusion results were objectively and subjectively evaluated. The fusion effects of other medical image fusion algorithms including weighted average algorithm (AVE), NSCT domain multi-parameter medical image fusion method (NSCT), zero learning fast medical image fusion (ZLF) and fusion method based on deconvolution neural network (CNN) were compared.
[0123] Experiments have shown that the ZLF method can extract sufficient spatial details from the source image, but there are some undesirable artifacts in the fused image, which greatly reduces the visual perception ability. The contrast of certain areas in the NSCT image is reduced, resulting in reduced brightness in certain areas of the image. The main disadvantage of the AVE method is that its ability to retain details is limited, resulting in blurred images. Although the fusion results of the CNN-based fusion method and the fusion method provided by the present invention are similar in the prostate area, the fusion method of the present invention can retain more detailed tissue information from the T2w image. In summary, the fusion method proposed in the present invention shows good results in detail retention without causing too much energy loss or artifact generation.
[0124] Refer to Table 1, which is a table of objective evaluation average scores of various fusion methods. The objective evaluation method of image fusion uses five indicators: information entropy (EN), gradient-based similarity measure (Q_(AB / F)), structural similarity (MS_-SSIM), mutual information measure (MI) and feature mutual information (FMI-w). The higher the value of each indicator, the better the quality of the fused image. The quantitative performance evaluation of different fusion algorithms is shown in Table 3. Except for the Q(AB / F) indicator (our method is second only to the CNN algorithm), the fusion method of the present invention outperforms the other four algorithms in most indicators. The fusion method of the present invention is superior to other algorithms in both visual perception and objective evaluation.
[0125] Table 1
[0126]
[0127] In summary, the pyramid has the advantage of enhancing the significant features of the image (such as contrast and intensity), but it is sensitive to noise, cannot effectively process the edge information of the image, and it is not easy to set a suitable fusion rule to process the semantic information of the image. The U-net network can effectively process and extract image semantic information. The present invention combines the traditional Gaussian pyramid transform fusion framework with the deep learning U-net network to form a set of medical image fusion methods. The present invention applies the fusion method to the fusion of prostate dual-parameter magnetic resonance images, which can help doctors find prostate lesions in the fused MRI images.
[0128] Furthermore, magnetic resonance imaging is the most effective method in the prostate image diagnosis method. The imaging diagnosis of prostate tumors is mainly through multi-parameter magnetic resonance images. However, it is cumbersome for doctors to observe images of multiple parameters at the same time and manually switch images. It is easy to cause missed detection and false detection. The present invention superimposes the effective information in the multi-parameter images on one piece by fusion, which not only increases the contrast and clarity of the prostate area on the image, but also saves the tediousness of doctors switching images with different parameters.
[0129] like Figure 4 As shown, a multi-parameter MRI image fusion device of a U-net network provided by an embodiment of the present invention includes:
[0130] An acquisition module 41 is used to acquire an MRI image pair of the prostate, wherein the image pair includes an ADC image and a T2W image;
[0131] A merging module 42, used for merging the ADC image and the T2W image to obtain a merged image;
[0132] An output module 43 is used to input the merged image into a trained U-net network to obtain a weighted image output by the U-net network with the same size as the merged image, wherein the pixel value in the weighted image represents the weight value of the weighted image in the T2W image or the ADC image;
[0133] A decomposition module 44, used for decomposing the ADC image, the T2W image and the weighted image respectively;
[0134] A calculation module 45, configured to calculate a first local energy of the decomposed ADC image and a second local energy of the decomposed T2W image;
[0135] The fusion module 46 is used to fuse the decomposed ADC image and the decomposed T2W image based on the decomposed weighted image to obtain a fused image when the matching degree between the first local energy and the second local energy reaches an energy threshold.
[0136] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.
[0137] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0138] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification.
[0139] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in a claim. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.
[0140] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.
Claims
1. A method for fusing multi-parameter MRI images using a U-net network, characterized in that: The fusion method comprises: Acquire a pair of MRI images of the prostate, wherein the pair of images includes an ADC image and a T2W image; Merging the ADC image and the T2W image to obtain a merged image; Inputting the merged image into a trained U-net network to obtain a weighted image output by the U-net network with the same size as the merged image, wherein the pixel values in the weighted image represent the weight values of the weighted image in the T2W image or the ADC image; Decomposing the ADC image, the T2W image and the weighted image respectively; Calculating a first local energy of the decomposed ADC image and a second local energy of the decomposed T2W image; When the matching degree between the first local energy and the second local energy reaches an energy threshold, based on the decomposed weighted image, fusing the decomposed ADC image and the decomposed T2W image to obtain a fused image; The step of calculating the first local energy of the decomposed ADC image and the second local energy of the decomposed T2W image comprises: For each pixel point of the decomposed ADC image, calculate the square sum of pixel values in a preset window centered on the pixel point to obtain a first local energy of the decomposed ADC image; For each pixel point in the decomposed T2W image, the square sum of pixel values in a preset window centered on the pixel point is calculated to obtain the second local energy of the decomposed T2W image.
2. The fusion method according to claim 1, characterized in that: The step of respectively decomposing the ADC image, the T2W image and the weighted image comprises: Performing Laplace pyramid decomposition on the ADC image and the T2W image respectively; Perform Gaussian pyramid decomposition on the weighted image.
3. The fusion method according to claim 1, characterized in that: For each pixel point of the decomposed ADC image, the step of calculating the square sum of pixel values in a preset window centered on the pixel point to obtain the first local energy of the decomposed ADC image comprises: For each pixel point of the decomposed ADC image, using a first local energy calculation formula, calculate the square sum of pixel values in a preset window centered on the pixel point to obtain a first local energy of the decomposed ADC image; The step of calculating, for each pixel point in the decomposed T2W image, the square sum of pixel values in a preset window centered on the pixel point to obtain the second local energy of the decomposed T2W image comprises: For each pixel point in the decomposed T2W image, using a second local energy calculation formula, calculate the square sum of pixel values in a preset window centered on the pixel point to obtain a second local energy of the decomposed T2W image; The local energy calculation formula is: E A represents the first local energy, E B represents the second local energy, i represents the horizontal coordinate of the pixel point, j represents the horizontal coordinate and vertical coordinate of the pixel point, m represents the window length centered on the pixel point, n represents the window width centered on the pixel point, L{A}(i+m,j+n) represents the pixel value of the pixel point in the window with a size of m×n and the pixel point (i, j) of the decomposed ADC image as the center, L{B}(i+m,j+n) represents the pixel value of the pixel point in the window with a size of m×n and the pixel point (i, j) of the decomposed T2w image as the center, N represents the length of the image, and M represents the width of the image.
4. The fusion method according to claim 1, characterized in that: Before the step of when the matching degree between the first local energy and the second local energy reaches an energy threshold, the fusion method further includes: calculating a matching degree between the first local energy and the second local energy; It is determined whether the matching degree reaches a preset energy threshold.
5. The fusion method according to claim 4, characterized in that: The step of calculating the matching degree between the first local energy and the second local energy includes: Using an energy matching degree calculation formula, calculating the matching degree of the first local energy and the second local energy; The energy matching calculation formula is: M AB (i, j) represents the matching degree between the first local energy and the second local energy, E A represents the first local energy, E B represents the second local energy, i represents the horizontal coordinate of the pixel point, j represents the horizontal coordinate and vertical coordinate of the pixel point, m represents the window length centered on the pixel point, n represents the window width centered on the pixel point, L{A}(i+m,j+n) represents the pixel value of the pixel point in the window with a size of m×n and the pixel point (i, j) of the decomposed ADC image as the center, L{B}(i+m,j+n) represents the pixel value of the pixel point in the window with a size of m×n and the pixel point (i, j) of the decomposed T2w image as the center, N represents the length of the image, and M represents the width of the image.
6. The fusion method according to claim 4, characterized in that: After the step of determining whether the matching degree reaches a preset energy threshold, the fusion method further includes: When the matching degree between the first local energy and the second local energy does not reach an energy threshold, comparing the information amount of the pixel points in the decomposed ADC image and the information amount of the pixel points in the decomposed T2W image to determine the pixel points with high information amount; The pixel points with high information content are determined as fused pixel points to obtain a fused image.
7. The fusion method according to claim 1, characterized in that: The step of fusing the decomposed ADC image and the decomposed T2W image based on the decomposed weighted image to obtain a fused image comprises: For a pixel point, multiply a pixel value of the pixel point in the weighted image by a first local energy of the pixel point in the decomposed ADC image to obtain a first multiplication result; For a pixel point, multiply the difference between 1 and the pixel value of the pixel point in the weighted image by the second local energy of the pixel point in the decomposed T2W image to obtain a second multiplication result; The sum of the first multiplication result and the second multiplication result is determined as the pixel value of the pixel point in the fused image.
8. The fusion method according to claim 1, characterized in that: After the step of fusing the decomposed ADC image and the decomposed T2W image based on the decomposed weighted image to obtain a fused image when the matching degree between the first local energy and the second local energy reaches an energy threshold, the fusion method further includes: The fused image is reconstructed using an inverse Laplace pyramid transform to obtain a reconstructed medical image.
9. A multi-parameter MRI image fusion device based on a U-net network, characterized in that: The fusion device comprises: An acquisition module, used for acquiring an MRI image pair of the prostate, wherein the image pair includes an ADC image and a T2W image; A merging module, used for merging the ADC image and the T2W image to obtain a merged image; An output module, used for inputting the merged image into a trained U-net network to obtain a weighted image output by the U-net network with the same size as the merged image, wherein the pixel value in the weighted image represents the weight value of the weighted image in the T2W image or the ADC image; A decomposition module, used for decomposing the ADC image, the T2W image and the weighted image respectively; A calculation module, used for calculating a first local energy of the decomposed ADC image and a second local energy of the decomposed T2W image; a fusion module, configured to fuse the decomposed ADC image and the decomposed T2W image based on the decomposed weighted image to obtain a fused image when the matching degree between the first local energy and the second local energy reaches an energy threshold; The step of calculating the first local energy of the decomposed ADC image and the second local energy of the decomposed T2W image comprises: For each pixel point of the decomposed ADC image, calculate the square sum of pixel values in a preset window centered on the pixel point to obtain a first local energy of the decomposed ADC image; For each pixel point in the decomposed T2W image, the square sum of pixel values in a preset window centered on the pixel point is calculated to obtain the second local energy of the decomposed T2W image.
