A method for delineating organs at risk based on adaptive fusion of MRI multi-parameter images
Through MRI multi-parameter image adaptive fusion technology, the optimal contrast fusion image is generated for organ segmentation, solving the problems of low efficiency and inconsistent accuracy in traditional technologies, achieving efficient and accurate organ outlines, reducing model training and reasoning time.
Patent Information
- Application Number
- CN202411171032.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-26
AI Technical Summary
The traditional MRI multi-parameter image-based automatic organ outline technology has problems such as low efficiency, inconsistent accuracy and overlapping boundaries of adjacent organs. It is mainly because a single parameter image cannot provide the best contrast information for all organs, resulting in large workloads in model training and reasoning. The feature information of the multi-parameter image is not fully utilized.
Using the MRI multi-parameter image adaptive fusion method, through joint training of image fusion network and organ-threatening organ segmentation network, a fusion image with optimal contrast is generated for organ segmentation, and automatic outline of all target organs is achieved, reducing the number of model training and Reasoning time to improve segmentation accuracy.
The optimal segmentation performance of all target organs is achieved, the workload and inference time of model training are reduced, the learning efficiency and accuracy are improved, the phenomenon of overlapping organ boundaries is avoided, and the feature information of multi-parameter images is fully utilized.
Smart Images

Figure CN118691934B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical image technology, and in particular to a method for delineating organs at risk based on adaptive fusion of MRI multi-parameter images. Background Art
[0002] MRI (magnetic resonance imaging) plays a vital role in radiotherapy. First, MRI images have excellent soft tissue resolution. This allows doctors to more clearly identify the boundaries of the tumor and the structure of the surrounding normal tissue. For example, for brain tumors, MRI can accurately distinguish the tumor from the surrounding gray matter, white matter and other structures, thereby more accurately defining the target area for radiotherapy and reducing damage to normal brain tissue. Secondly, MRI images can provide multi-parameter information. For example, by measuring parameters such as the diffusion coefficient, it can reflect the activity and degree of infiltration of tumor cells, which helps to assess the malignancy of the tumor and the sensitivity of radiotherapy. Taking breast cancer as an example, combined with MRI's multi-parameter imaging, it is possible to have a more comprehensive understanding of the characteristics of the tumor and provide a basis for formulating personalized radiotherapy plans.
[0003] Automatic organ-at-risk delineation based on MRI multi-parameter imaging is a technology that uses multi-parameter images of magnetic resonance imaging (MRI) to achieve automatic delineation of organs at risk. The multi-parameter images specifically include:
[0004] (1) T1-weighted imaging (T1WI) reflects the difference in the longitudinal relaxation time T1 of tissues. Fat tissue appears as a white high signal on T1WI, while fluids such as cerebrospinal fluid appear as a black low signal.
[0005] (2) T2-weighted imaging (T2WI) reflects the difference in the transverse relaxation time T2 of tissues. Water appears as a white high signal on T2WI, such as cerebrospinal fluid and fluid in cysts.
[0006] (3) Diffusion-weighted imaging (DWI) reflects the Brownian motion of molecules and is manifested by detecting the restricted diffusion of water molecules in tissues, such as bleeding and congestion.
[0007] (4) Diffusion tensor imaging (DTI) reflects the anisotropy of water molecule diffusion. It is mainly used to evaluate the integrity of nerve fiber bundles and is helpful for the diagnosis of white matter lesions and spinal cord injuries.
[0008] (5) Perfusion-weighted imaging (PWI): It reflects the blood perfusion of tissues and is mainly used to evaluate the blood perfusion status of tissues such as the brain and myocardium. It also plays a certain role in evaluating the blood supply of tumors.
[0009] (6) Magnetic resonance spectroscopy (MRS) reflects the content and concentration of specific metabolites in tissues. It is often used in the diagnosis and differential diagnosis of brain tumors and can provide information on tumor metabolism.
[0010] Usually, in manual delineation of organs at risk based on MRI multi-parameter images, doctors need to manually switch between multiple parameter images and confirm the position and accurate boundary information of the organ through repeated comparison. The main reason is that the same organ has different contrast information on different parameter images, and usually has the best contrast on a certain parameter image. Different organs have different contrast information on the same parameter image, and usually there is no parameter image that can obtain the best contrast information for all organs. Different parts of the same organ have different contrast information on the same parameter image, and multiple parameter images are needed to determine the boundary conditions of all parts of the organ.
[0011] Therefore, the traditional automatic delineation technology of organs at risk based on a single parameter image has defects, and it is difficult to achieve a very good segmentation effect for all organs. The current common solution is to divide all organs at risk into multiple groups based on the contrast distribution of all organs on different parameter images, that is, only one group of organs is delineated on each parameter image, so that the group of organs has the optimal contrast distribution on the parameter image.
[0012] However, the above methods require training of multiple groups of models, which increases the workload of model training and the time of organ-at-risk reasoning. At the same time, the above methods do not fully utilize the feature information of all parameter images to efficiently segment organs, and it is difficult to ensure the accuracy of the boundaries of all organs at risk. Therefore, the boundaries of two adjacent organs predicted by different models often overlap.
[0013] Based on this, the present invention designs an organ at risk delineation method based on MRI multi-parameter image adaptive fusion to solve the above problems. Summary of the invention
[0014] In order to make full use of the information of MRI multi-parameter images, the problems of low efficiency of organ delineation, inconsistent organ delineation accuracy, and overlapping of adjacent organ boundaries predicted by different groups of models in traditional single-parameter image delineation or multi-parameter image group delineation are solved. The present invention provides an organ-at-risk delineation method based on adaptive fusion of MRI multi-parameter images. The present invention performs adaptive image fusion on MRI multi-parameter image data of the same patient, and can automatically fuse a fused image with optimal contrast for all target organs, and segment all target organs based on the fused image, thereby realizing automatic delineation of multiple organs at risk, so that all organs at risk have optimal segmentation performance.
[0015] To achieve the above objectives, the present invention is implemented through the following technical solutions:
[0016] An organ-at-risk delineation method based on adaptive fusion of MRI multi-parameter images includes a model training phase and a model reasoning phase;
[0017] The model training phase is used to train the image fusion network and the risk organ segmentation network;
[0018] The steps of the model inference stage are as follows: First, the MRI multi-parameter image has multiple parameter images. The MRI multi-parameter image is input into the image fusion network to obtain a fused image I that fuses all parameter features. fusion , then the fused image I fusion Input into the risk-at-risk organ segmentation network, and finally output the segmentation results of n target risk-at-risk organs.
[0019] Furthermore, the model training phase includes the following steps: data preprocessing, data augmentation, model training, and model validation.
[0020] Furthermore, the data augmentation includes the following steps: performing image flipping, image rotation, image translation and image contrast adjustment on the pre-processed MRI multi-parameter image.
[0021] Furthermore, the model training steps are:
[0022] The pre-processed MRI multi-parameter image of the patient is input and the parameters are used as the input feature channel. Then the image fusion network and the organ-at-risk segmentation network are forward deduced and the network parameter gradients are calculated to obtain the output of the image fusion network, i.e., the fused image I fusion , and obtain the output of the endangered organ segmentation network, i.e., the segmentation results of n target endangered organs;
[0023] In the gradient backpropagation and parameter update stage, first, the parameters of the image fusion network are frozen, the segmentation loss value L1 between the n target organs at risk and the labeled organ mask is calculated, and then the gradient backpropagation and network parameter update of the organ at risk segmentation network are performed; secondly, the parameters of the image fusion network are unfrozen, and the parameters of the organ at risk segmentation network are frozen, and then the fused image I is used to generate the segmentation loss value L1. fusion The mutual information loss is calculated with the input MRI multi-parameter image and then accumulated to obtain the fused image I fusion The loss value L2 is calculated, and the loss value L2 is added to the previously calculated segmentation loss value L1 to obtain the final loss value L3. Then, the image fusion network is gradient-backed and the network parameters are updated.
[0024] Furthermore, the segmentation loss value L1=Dice loss value + CE cross entropy loss value.
[0025] Furthermore, the model inference phase also includes the following steps: data preprocessing and postprocessing.
[0026] Furthermore, the specific steps of data preprocessing are:
[0027] 2.1.1、MRI multi-parameter images will be pre-parsed from dicom format data, and multiple parameter images of the same patient will be uniformly normalized;
[0028] 2.1.2. Merge multiple normalized parameter images in the channel direction to obtain a 4-dimensional matrix (D*H*W*P), where D represents the number of image layers, H represents the image height, W represents the image width, and P represents the number of parameters. Save it in nifti data format.
[0029] Furthermore, the unified normalization includes a unified image origin, a unified image coordinate axis, a unified image resolution, a unified image size, and a range normalization to [0, 1].
[0030] Furthermore, under normal circumstances, an MRI multi-parameter image has 6 parameter images. If a patient has less than 6 parameter images, the missing parameter images are replaced with an image whose values are all zero.
[0031] Furthermore, the main body of the image fusion network is a 3D UNet network. The multi-channel MRI multi-parameter image is input, and the deep features are extracted and decoded into a weight map through the 3D UNet network. The weight map has the same size and number of channels as the input MRI multi-parameter image, and is converted into a weight probability map through a softmax function. Finally, the input MRI multi-parameter image is multiplied by the weight probability map and summed in the channel direction to obtain a new image, which is the fused image I fusion ;
[0032] The main body of the endangered organ segmentation network is a UNet network, which inputs the fusion image I fusion ,After segmentation by the UNet network, a softmax activation function is used to obtain the segmentation results of the n target organs at risk that are finally outputted.
[0033] The present invention has strong robustness. Compared with the traditional single parameter image segmentation, the present invention makes full use of the characteristic information of all parameter images to efficiently segment organs, and has higher learning efficiency and accuracy.
[0034] The present invention has strong applicability. Compared with the traditional segmentation model trained with a single parameter image which can only be used for organ segmentation on the parameter image, the present invention can be applied to organ segmentation of any parameter or any combination of multiple parameter images.
[0035] The invention has high accuracy and enables all target organs to obtain optimal contrast, so that all organs can have optimal segmentation accuracy at the same time, and there will be no boundary overlap of organs.
[0036] The present invention is efficient and fast. Only one model needs to be trained to meet the needs of simultaneous delineation of multiple target organs, which reduces the workload of model training on the one hand and the time of organ prediction on the other hand. The problem of low efficiency of traditional single-parameter image delineation or multi-parameter image grouping delineation of organs is solved. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0038] Figure 1 It is a flow chart of the method for delineating organs at risk based on adaptive fusion of MRI multi-parameter images of the present invention;
[0039] Figure 2 This is a schematic diagram of the model training phase in the present invention;
[0040] Figure 3 It is a structural schematic diagram of the image fusion network in the present invention;
[0041] Figure 4 It is a schematic diagram of the structure of the endangered organ segmentation network in the present invention. DETAILED DESCRIPTION
[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0043] Example 1: Please refer to the attached manual Figure 1 , an organ-at-risk delineation method based on adaptive fusion of MRI multi-parameter images, firstly, the MRI multi-parameter image has 6 parameter images, I0, I1, I2, I3, I4, and I5. The MRI multi-parameter image is input into the image fusion network (Image Fusion Net) to obtain the fusion image I that integrates all parameter features. fusion , then the fused image I fusion Input to the dangerous organ segmentation network (Segment Net), and finally output Oar1, Oar2, ...Oar n Segmentation results of n target organs at risk.
[0044] I0, I1, I2, I3, I4, and I5 represent T1-weighted image (T1WI), T2-weighted image (T2WI), diffusion-weighted image (DWI), diffusion tensor image (DTI), perfusion-weighted image (PWI), and magnetic resonance spectroscopy image (MRS), respectively.
[0045] Embodiment 2: A method for delineating organs at risk based on adaptive fusion of MRI multi-parameter images, specifically including a model training stage and a model reasoning stage.
[0046] Please refer to the instruction manual Figure 2 In the model training phase, the image fusion network (Image Fusion Net) and the endangered organ segmentation network (Segment Net) are trained. The training process is an end-to-end training method, which includes 4 steps: data preprocessing, data augmentation, model training, and model verification.
[0047] The model training phase mainly includes four steps:
[0048] 1.1. Data preprocessing: Preprocess the patient's MRI multi-parameter images (6 parameter images). The preprocessing method refers to step 2.1.
[0049] 1.2. Data augmentation: The preprocessed MRI multi-parameter images are subjected to image flipping, image rotation, image translation, and image contrast adjustment.
[0050] 1.3 Model training:
[0051] The pre-processed MRI multi-parameter image of the patient (6 parameter images) is input, and the parameters are used as the input feature channels. Then, the image fusion network and the organ-at-risk segmentation network are forward deduced and the network parameter gradients are calculated to obtain the output of the image fusion network - the fused image I fusion The output of the organ-at-risk segmentation network - Oar1, Oar2, ..., Oar n Segmentation results of n target organs at risk.
[0052] During the gradient backpropagation and parameter updating stage, the image fusion network and the organ at risk segmentation network are performed alternately.
[0053] First, freeze the parameters of the image fusion network and calculate the n target organs at risk Oar1, Oar2, ..., Oar nThe segmentation loss value L1 between the labeled organ mask (including Dice loss value (Dice Loss) and CE cross entropy loss value (CELoss), segmentation loss value L1 = Dice loss value + CE cross entropy loss value), and then the gradient of the endangered organ segmentation network is backpropagated and the network parameters are updated.
[0054] Secondly, unfreeze the parameters of the image fusion network and freeze the parameters of the endangered organ segmentation network, and then use the fused image I fusion The mutual information loss (MI Loss) is calculated with the input MRI multi-parameter image and then accumulated to obtain the fused image I fusion The loss value L2 is calculated, and the loss value L2 is added to the previously calculated segmentation loss value L1 to obtain the final loss value L3. Then, the image fusion network is gradient-backed and the network parameters are updated.
[0055] When the image fusion network and the organ at risk segmentation network have completed the gradient backpropagation and network parameter update, they enter the next training cycle. Different data are sampled for training in each training cycle.
[0056] 1.4. Model validation: After a specified number of training rounds, use the validation data to test the effectiveness of the current model weights, and select the model weights with the best performance for storage.
[0057] Please refer to the instruction manual Figure 1 ,The model inference stage mainly includes four steps, namely: data preprocessing, adaptive image fusion, multi-organ automatic segmentation, and post-processing;
[0058] 2.1. Data preprocessing: MRI multi-parameter images have 6 parameter images, namely I0, I1, I2, I3, I4 and I5. The 6 parameter images of MRI are preprocessed;
[0059] The specific steps of the pretreatment are:
[0060] 2.1.1. MRI multi-parameter images will be parsed from DICOM format data in advance, and multiple parameter images of the same patient will be standardized, that is, the image origin, image coordinate axis, image resolution, image size, and value range will be normalized to [0, 1].
[0061] 2.1.2. Under normal circumstances, MRI multi-parameter images have a total of 6 parameter images, namely I0, I1, I2, I3, I4, and I5. If a patient has less than 6 parameter images, the missing parameter images will be replaced with an image with all values zero (the size, resolution and other information are consistent with other parameter images).
[0062] 2.1.3. Merge the multiple normalized parameter images (6) in the channel direction to obtain a 4-dimensional matrix (D*H*W*P), where D represents the number of image layers, H represents the image height, W represents the image width, and P represents the number of parameters. Here P=6. Save it in nifti data format.
[0063] 2.2. Adaptive image fusion: The preprocessed 6 parameter images are input into the image fusion network, and the 6 parameter images are adaptively fused through the image fusion network to obtain a fused image I with high boundary contrast for all target organs at risk. fusion ;
[0064] 2.3. Automatic segmentation of multiple organs: Fusion image I fusion Input into the endangered organ segmentation network, the fusion image I fusion Perform multi-organ automatic segmentation to obtain segmentation results of all target organs at risk;
[0065] 2.4. Post-processing: Post-process the segmentation results and output Oar1, Oar2, ...Oar n Segmentation results of n target organs at risk.
[0066] Among them, the method of post-processing the segmentation results is related to the specific organs at risk, and usually includes: retaining the maximum connected domain, retaining the maximum biconnected domain, removing tiny scattered points and internal holes, etc.
[0067] Example 3: Figure 3 As shown in the figure, the main body of the Image Fusion Net is a classic 3DUNet network. The multi-channel MRI multi-parameter image is input, and the deep features are extracted through the 3D UNet network and decoded into a weight map. The weight map has the same size and number of channels as the input MRI multi-parameter image. It is converted into a weight probability map through a softmax function. Finally, the input MRI multi-parameter image and the weight probability map are multiplied pixel by pixel and summed in the channel direction to obtain a new image, which is the fused image I fusion ;
[0068] In particular, the input MRI multi-parameter image can be partially missing, that is, any one of the MRI multi-parameter images (T1WI, T2WI, DWI, DTI, PWI, MRS) can be missing and replaced by a blank image with all values zero.
[0069] Example 4: Figure 4As shown in Figure 2, the main body of the endangered organ segmentation network (Segment Net) is a classic UNet network, with the input fusion image I fusion After UNet network segmentation, a softmax activation function is used to obtain the final output organ segmentation result, namely Oar1, Oar2, ...Oar n Segmentation results of n target organs at risk.
[0070] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for delineating organs at risk based on adaptive fusion of MRI multi-parameter images, characterized in that: Includes model training phase and model inference phase; The model training phase is used to train the image fusion network and the risk organ segmentation network; The steps of the model inference stage are as follows: First, the MRI multi-parameter image has multiple parameter images. The MRI multi-parameter image is input into the image fusion network to obtain a fused image I that fuses all parameter features. fusion , then the fused image I fusion Input into the endangered organ segmentation network, and finally output the segmentation results of n target endangered organs; The model training phase specifically includes the following steps: data preprocessing, data augmentation, model training, and model validation; The model training steps are: The pre-processed MRI multi-parameter image of the patient is input and the parameters are used as the input feature channel. Then the image fusion network and the organ-at-risk segmentation network are forward deduced and the network parameter gradients are calculated to obtain the output of the image fusion network, i.e., the fused image I fusion , and obtain the output of the endangered organ segmentation network, i.e., the segmentation results of n target endangered organs; In the gradient backpropagation and parameter update stage, first, the parameters of the image fusion network are frozen, the segmentation loss value L1 between the n target organs at risk and the labeled organ mask is calculated, and then the gradient backpropagation and network parameter update of the organ at risk segmentation network are performed; secondly, the parameters of the image fusion network are unfrozen, and the parameters of the organ at risk segmentation network are frozen, and then the fused image I is used to generate the segmentation loss value L1. fusion The mutual information loss is calculated with the input MRI multi-parameter image and then accumulated to obtain the fused image I fusion The loss value L2 is added to the previously calculated segmentation loss value L1 to obtain the final loss value L3, and then the image fusion network is gradient-backed and the network parameters are updated; Segmentation loss value L1 = Dice loss value + CE cross entropy loss value; The model inference phase also includes the following steps: data preprocessing and postprocessing; The specific steps of data preprocessing are: 2.1.
1. Unify and normalize multiple parameter images of the same patient; 2.1.
2. Combine multiple normalized parameter images in the channel direction to obtain a 4-dimensional matrix (D*H*W*P), where D represents the number of image layers, H represents the image height, W represents the image width, and P represents the number of parameters. Save it in nifti data format; Normally, MRI multi-parameter images have 6 parameter images. If a patient has less than 6 parameter images, the missing parameter images are replaced with an image with all values 0. The main body of the image fusion network is a 3D UNet network. The multi-channel MRI multi-parameter image is input, and the deep features are extracted and decoded into a weight map through the 3DUNet network. The weight map has the same size and number of channels as the input MRI multi-parameter image. It is converted into a weight probability map through a softmax function. Finally, the input MRI multi-parameter image is multiplied by the weight probability map and summed in the channel direction to obtain a new image, which is the fused image I fusion ; The main body of the endangered organ segmentation network is a UNet network, which inputs the fusion image I fusion ,After segmentation by the UNet network, a softmax activation function is used to obtain the segmentation results of the n target organs at risk that are finally outputted.
2. The method for delineating organs at risk based on adaptive fusion of MRI multi-parameter images according to claim 1, characterized in that: The data augmentation includes the following steps: performing image flipping, image rotation, image translation and image contrast adjustment on the preprocessed MRI multi-parameter image.
3. The method for delineating organs at risk based on adaptive fusion of MRI multi-parameter images according to claim 2, characterized in that: Unified normalization includes unifying the image origin, image coordinate axes, image resolution, image size, and range normalization to [0, 1].
